Luís Martinho
Posts
It was the last day of Codebits 2011 and I still manage to catch a quirky talk by Alcides Fonseca about emerging programming languages and such... Interestingly enough, when the talk ended, I visited the O'Reilly stand and was unexplainably drawn to Seven Languages in Seven Weeks by Bruce A. Tate.
The plan seemed simple enough: to give my best to solve some basic problems in different programming languages. All of this while understanding some key attributes of each language (typing model, programming model, core features, etc.) and getting a feel for the syntax.
So that's what I'll be doing for the next seven weeks: picking up some language, and taking it for a spin. Hope it's worth it... Looking forward to it.
I'm thinking about putting up a tutorial for a dummy linux driver, so you'll be able to cross that off your bucket list. Stay tuned.
Can't wait? Check this tutorial out, mine won't be as good: http://www.freesoftwaremagazine.com/articles/drivers_linux
Unless you earn a living from interest on money you inherited, you probably work for some kind of organization that has to captivate the attention of some kind of audience.
It doesn't matter if its sales, investments, donations, clicks, retweets or whatever your endgame is. I believe we are all in sales, and can improve our end of the job by understanding some basic dynamics of the sale process.
That's why I'll be mixing in, with the usual technical stuff, some sales tips I learn from the work I do with my sales hat on. I'm going to aim for the basics, so let me know if you're bored or just skip these kinds of posts.
- Plugin: the component itself, it provides a façade for the behaviour it encapsulates, as well as all the resources it might need to operate.
- Dependency: plugins may present an explicit request for another plugin id+version combination in order to function. The plugin manager will then take care of injecting a reference to the dependency.
- Capability: plugins may also indicate they can have their functionality extended by other plugins that conform to a specific interface (capability). This way is not about a specific plugin, but rather about a given contract... which means the plugin can be extended in innovative ways, as long as the API contract is kept.
Last week I got a special visit at The Office by the fine folks of Patrulha, a Portuguese web show about internet and technology!
Go check it out to see where Hive Solutions is dreaming up the next generation of web applications.
For those of you who don't speak Portuguese, well you'll miss all the nonsense, but you still get to see how cool the office is.
And here we are at 30.000 thousand views.
Time for a facelift, after all the dubious feedback I've been getting about the theme (which I still like very much).
I've also moved to the new posterous data center which seems to be making the blog load faster (I should probably wait until everyone else is settled in, to call it #win).
- Core clock speed: 1.15 GHz
- 448 cores
- Memory bandwidth: 114 GB/sec
- 515 GFLOPS (double precision, twice as fast with 32-bit floats)
- Price: around $3,600
- Core clock speed: 3.46GHz
- 6 cores
- Memory bandwidth: 21 GB/s (dual-channel mode and assuming DDR3 1333 MT/s)
- 107.55 GFLOPS
- Price: around $1,100
Ever been developing a rich user experience using client side scripting, say JavaScript, and hit a same origin policy wall?
Browsers sandbox access to resources in different domains for security reasons. JSONP used to be the standard procedure for working around this limitation, but it only allowed GET requests and was limited to retrieving, well, JSON.
Sure you can make a local proxy, in the same domain, which lets you forward requests to the remote domain, bypassing the limitation... But what you really want is a clean browser technology specification.
Enter Cross-Origin Resource Sharing, which allows cross site XMLHttpRequests as long as the resource to be retrieved is configured to be shared.
This means your HTTP server only has to add the following response headers (ABNF):
- Access-Control-Allow-Origin = "Access-Control-Allow-Origin" ":" origin-list-or-null | "*"
For example, to allow a given resource to be included from example.com, one would configure the HTTP server providing the resource to add the following response header:
- Access-Control-Allow-Origin : "http://example.com"
Pretty easy, right?
To see learn more about CORS, and how to quickly enable CORS in your particular stack, check out: enable-cors.org.
The W3C working draft from 2010 is here.
How long has it been since you've sat down to read code? Not just to try to fix some bug that guy who quit last week left on some obscure part of the codebase, but to really learn from the masters.
I think writing software is equal parts engineering and creative writing. Yes, we should adhere to standards and strive for consistency, but it is within those norms that we can be more creative.
All the right "Tips For Reading Code", can be found in Ward's wiki.
What you'll find after the jump:
- Build and Run the Program
- Find the High-Level Logic
- Draw Some Flowcharts
- Examine Library Calls
- Search for Key Words
- Leverage the Power of Code Comprehension Tools [common questions you should ask yourself when reading]
- Print the code
- Write UnitTests
- Comment the Code
- Clean Up the Code
But what I'm going to suggest is that you just skim through a bit of code, and find something that catches your eye.
Try to find new ways to do old stuff. Find new idioms in a language you're already familiar with. Or maybe look at a new language to get a difference perspective.
Why don't you start with a project you respect. Use Google Code Search or go to Github or Bitbucket.
Productivity tips are a dime a dozen, but I really wanted to share these 3 which have been kind of useful to me (real world useful).
I haven't invented any of them, just put together from things I've read.
Although magically simple, each of these tips can have a tremendous impact on your daily output, your procrastination habits and your overall mood!
Take a look at each of them, and think about how they could apply to what you're doing right now:
- Eat the Frog First
- Set a Date
- Visualize
Eat the Frog First
If you eat a frog first thing in the morning that will probably be the worst thing you do all day. - Mark Twain
If you start your day with the thing you least want to do, or the thing you've been putting off the most, you'll get a feeling of accomplishment that will stay with you for the rest of the day.
Do the hard stuff first, and you'll feel empowered!
Set a Date
Try to clearly define a date and time slot for your most important tasks.
This tip is actually worth two:
- setting a date to get something done works a lot better than just putting more stuff into your ToDo list;
- and deciding how much time you are going to spend on a task makes you focus on what is really important to get the job done.
Visualize
Next time you try to do something, first imagine doing it in your mind.
Be as detailed as possible, imagine all the finer points of the work you're doing.
Imagine the problems you would have, imagine how you would go around them. Imagine the feeling of sucess at the end of a job well done.
That way, when you actually do the thing, it will be a lot like doing the thing for the second time: free experience, just using your prefrontal cortex
So last week my dad had two laptops that were having some sort of BIOS related issue.
One was an ASUS EEE PC, which kept reloading default settings every boot. I figured it could probably be fixed with a newer BIOS. I ran the ASUS official utility, and it got me a new BIOS version for my motherboard and flashed it all by itself. It rebooted and fixed the problem. Happy.
The other machine was from a Portuguese manufacturer, or was it?
Anyway, every time this laptop would boot, it would prompt for date and time settings. Simple diagnostic, right? Change the CMOS battery! So I popped the hood, but insted of finding a neat battery slot, like so:
I found the battery wrapped in a cocoon-like casing.
I'm sure that to anyone used to messing around with laptops, the battery casing shouldn't present much of a challenge.
(Warning: from this point on it would have been smarter to ask for help from someone who wasn't all thumbs like myself and could actually replace the battery)
But I somehow thought it would be better to leave the battery alone and go after the other option: "maybe I can fix it with a BIOS upgrade". Famous last words, right?
I tried to find a way to upgrade the BIOS in the manufacturer's website, but nothing I couldn't find anything except a generic PDF manual and the some drivers. So I did some research about the laptop and found out the Portuguese assembler brand I my dad had bought it from, was hardly assembling anything at all. That's when I lifted the "Assembled in Portugal" sticker to find a "Made in China" one. I looked up the product code, and found it was produced by an ODM in Taiwan (aren't they all).
These guys actually had several BIOS versions available for the laptop. I downloaded the latest BIOS they customized from AMI, and flashed it successfully onto the chip. I rebooted and got a silent black screen. No beeps, no HD activity, just the fan! Help?
The outsourced hardware support company the Portuguese manufacturer hired, said I had to get a new motherboard (obviously). The thing was the motherboard cost as much as the laptop by now.
So I was stuck, with no EEPROM programmer in sight, I had no option but to dump the machine altogether. But with my replaceable BIOS slot, I thought something could be done:
Is there still hope?
Wii motherboard, as seen on Wiire.org
In my previous post I told you about how was able to easily run homebrew software in my Wii. For me, it was easy as running the canned exploit and the installer, and after that it was clear sailing...
I suspected it wasn't exactly this easy for the hacker themselves, but I was stunned with the ingenuity they displayed, while getting round the Wii's security mechanisms. I learned more about the early parts of the process in this interview on Atomic magazine.
The interview dates back to 2008, and tells the story about the initial experiments done by Bushing, Segher, Tmbinc et al.
These guys found out that Nintendo ships the newest version with almost every game. They were able to get their hands not only on game code but also on the code that does more or less everything on the Wii. They tried to disassemble the PowerPC code (remember Wii's Broadway CPU), only to discover it was actually ARM code. If you look at the motherboard picture above you'll find that no ARM chip is anywhere to be found, so these guys figured out it should be hidden inside the Hollywood GPU (actually a multi-chip module package). No we know it to be a ARM926 core, which they nicknamed Starlet back then. This Starlet core does I/O, but also does security (essencially authenticating software before running).
Home made DVDs could already be run, since the DVD drive used the same security as the Gamecube (which had already been beaten), but homebrew code could only be written for the Wii's Gamecube mode, which didn't use signing. Although sandboxed in this mode (which basically couldn't access anything the Gamecube didn't already have), they were still able a limited amount of memory.
Operating in Gamecube mode they were able to physically short the address lines in the memory chip (U3 in the picture above) using a pair of tweezers, and trick the Starlet into spitting out its decryption key from memory and into a controller port (which then connected to a PC or something). How clever is that?
Now they were able to disassemble and decrypt code and data, and so the analysis began. From there on they were able to read the code and analyse it extensively enough to find exploitable flaws. One early bug, was that Nintendo was using strncmp to compare signatures instead of memcmp... Yes, that meant the comparison only went as far as the first null byte... This meant you could easily fool the SHA verification code, by adjusting your content to make the signature have a zero as early as possible (bytes before the zero would still have to be bruteforced).
Below is more comprehensive timeline I found here:
1) Modchips developed because DVD drive uses exact same (already defeated) security as Gamecube (WTF #1).
2) Team Twiizers uses homebrew in Gamecube mode with Twiizer hack to dump system code from memory that was not supposed to be accessible in that mode.
3) Decryption and analysis of dumped code reveals strncmp() bug (WTF #2).
4) Knowing the signature flaw, they replace the main executable on a copy of a Wii game and successfully boot it. This flaw is not published because Nintendo can easily patch it; instead, they use it to probe for more exploits from the inside.
5) Save game encryption key is found, which leads to discovery of buffer overflow in Zelda (WTF #3). This flaw is released, and homebrew begins.
6) System Menu 3.3 update patches signature flaw, and checks for and removes hacked Zelda saves, but the save file checking routine has several massive bugs that render it useless. (WTF #4)
From then on the homebrew scene flourished, and enough so that a graphical installer and homebrew software launcher are available.
Props to the people at Wiibrew.org
I look at the alternatives: my HDTV IP Set-Top Box (a Thomson DCI8220CAB) - not the kind of thing you would like to mess with if you are short on time - and a Nintendo Wii!
I browsed around for a bit and quickly bumped into a lively homebrew scene. I found someone had that someone had already gone ahead and ported MPlayer to the Wii (a couple of times).
All the homebrew software could be run from an unmodified console, using a set of clever hacks that exploited vulnerabilities in existing games that allow to run code from, say, a modified savegame.
The wiibrew.org wiki helped identify the adequate hack for my System Menu version: Indiana Pwns, clearly a classic in the community. So I set out to buy LEGO Indiana Jones, the Original Adventures. My local GAME store was out of stock on it. Apparently the buffer overflow that "roto" found out made the game become somewhat of a rarity, and made the value skyrocket, reaching 100 EUR in some cases. I ended up buying it for 15 EUR in a Worten store here in Gaia.
Using the hack is extraordinarly simple, and you can see these guys have gone to great lengths to make this really easy to use. You just put your homebrew binary in an SD card together with the hacked saved game, load up the save and play the game a bit to activate the overflow and corresponding code injection.
After that, the homebrew installer was loaded, and it set me up with the Homebrew Channel, a homebrew loader that sits right in your System Menu. All clear sailing from there on, I can just download or build my own homebrew apps, put them on an SD card or a USB drive and the Homebrew Channel will let me load them straight away.
And there I was, ready to use my Wii as a makeshift media center. Thank you so much Wii Homebrew community.
Last week, computer history real came at me from all sides.
I was looking into Fred Brook's Mythical Man-Month and some anectodes about the birth of System/360 and OS/360. I also skimmed through an OS book which gave a pretty good overview of computing history (yes it was Andy Tanenbaum's), from the Analytical Machine up to the Personal Computer Revolution.
The same week I was shown this awesome Revolutions tour (the new exhibit at the Computer History Museum) by Woz himself, as covered by Robert Scoble.
I highly recommend it, and you can find some other stuff about the exhibit after the jump.
While we're talking about history, why not take a look at the operating manual for the Apple I. Wonderful piece of documentation, really makes me wonder how it must of been to live in those times of homebrew awesomeness.
As a side note, the Apple I design was powered by the MOS Technology 6502, which curiously was the base for the first computing system I ever owned: a crummy Famiclone sold here in Europe. The NES was famously run by the Ricoh 2A03 (NTSC) or Ricoh 2A07 (PAL), which were based on a second-source 6502 core and were basically system-on-a-chip. They didn't have the BCD mode like the 6502, but did have 22 memory-mapped registers for I/O as well as support for joypad reading and DMA right on the die.
I'll probably return to this subject in the future. The best way to predict what happens next, is to understand what happened before.
Let's talk about beautiful software. I'll start with MINIX.
This week my boss sold me on MINIX (not your average boss, right? I'm not your average employee either...).
We have probably all heard about MINIX from Andy Tanebaum's OS book, a staple in undergraduate courses in the US and here in Europe.
The microkernel hypothesis does seem to make sense: smaller memory footprint, separation of concerns and overall cleanliness. But there's more... about €2.5 M more. Tanenbaum, back in 2008, was awarded a significant European Research Council Advanced Grant to work on reliable operating systems. And reliability is what MINIX 3 is all about.
Being a microkernel (handling only interrupts, processes, scheduling and IPC) means you can have user mode drivers (actually that means you can have user mode everything else).
What the MINIX gang has asserted is that most of a monolithic kernel's codebase is made out of drivers (around 70%) and that driver code is of inferior quality (about twice as many bugs/KLOC). That means you're getting a lot of powerful bugs (running in kernel mode). What user mode drivers mean is that you're still going to have bugs (most drivers written are the first driver written by the author, and a lot of the times using poorly documented hardware) but they're not going to be able to bring the system down (by corrupting kernel memory, for instance), since they are running in protected mode.
They went as far as creating the infra-structure to automatically replace drivers without affecting running processes. This means MINIX is meant to be self-healing and aims for extremely high reliability.
Be sure to check out the MINIX 3 project website: http://www.minix3.org/ as well as these great notes on reliability.
Also check out this presentation by the man Tanenbaum himself:
According to Wikipedia, Silicon Valley accounts for 1/3 of all of the venture capital investment in the United States. It has the most high-tech jobs in the US (386,000). It also has the highest concentration of high-tech workers of any metropolitan area, and the highest average high-tech salary.
Its the kind of place you would want to replicate. And a LOT has been thought, written and said about it.
Well I was fortunate enough to stumble onto this very awesome, very detailed and very old article about the growth of Silicon Valley from the San Francisco Chronicle.
The article is about 12 years old, right before the dot-com bust. But its all about the History of the Valley, so still a valuable read. It really describes the evolution of the area throughout the years and adds a lot to the typical "Military Technology + Stanford Industrial Park + Shockley's silicon transistor" story.
I highly recommend you read through the whole article, but still let me just lay down three things that really caught my attention:
- Things weren't always this easy
- Openness + Enthusiasm + Tolerance for Failure
- Good neighbors ("regional network-based industrial system")
Things weren't always this easy
The article demonstrates very clearly the overwhelming importance of Stanford, especially in the early days. The school was commited to generating marketable innovations, and not only encouraged its students and faculty to go into business but also facilitated equipament, infra-structure and funds to directly support the emerging electronics industry.
Back in 1909 David Starr Jordan, Stanford's founding president, invested $500 himself to help a recent graduate named Cyril Elwell launch a wireless communications company, Federal Telegraph Co., in Palo Alto. Other Examples apparently abound, such as in the 1930s, when three alumni of the school Sigurd Varian and Russell Varian together with William Hansen, part of the faculty at Stanford, started working on the klystron, a microwave transmitter and receiver which later became important for radar-detection.
Back then the university gave them free use of a lab plus $100 worth of materials per year, in exchange for a 50 percent interest in their patents.
Another important example was the University's push for William R. Hewlett and David Packard, to build a business around the audio oscillator they had developed. The dean even invested some of his own money in the operation, which started in a Palo Alto garage and incorporated on Jan. 1, 1939, as the Hewlett-Packard Co. HP grew to become the Bay Area's largest company and turn out to serve as forge for future generations of engineers and managers to staff the Valley.
Nonetheless growing pains were significant in a place so far from the developed East.
(...) Federal Telegraph had relocated most of its operations to New Jersey in 1931, on the grounds that Palo Alto was too remote from its key markets and sources of supplies and skilled labor -- precisely the obverse of the arguments that were to bring so many technology companies to this area a few decades later."
Even after World War II, the size of the industry was still quite small relative to the developed electronics industry of the East Coast. The influent dean of engineering at Stanford, Frederick Terman, went east himself, to contribute to the war effort, assuming the position of director of the Radio Research Laboratory at Harvard. The article offers an interesting quote from Terman:
"Stanford emerged from World War II as an underprivileged institution. It had not been significantly involved in any of the exciting engineering and scientific activities associated with the war."
Openness + Enthusiasm + Tolerance for Failure
The article refers the importance of Fairchild Semiconductor around the 1960's, from which spawned numerous offshoots most importantly Intel. It references its role in creating part of the culture that would later defined what is the Valley.
Fairchild Semiconductor went on to become an important defense contractor, but it never lived up to the hopes of its founders -- in part, it is said, because its remote parent company never really understood the business. But in several ways, the company played a critical role in the creation of Silicon Valley.
First, its unofficial corporate culture was reportedly marked by the freewheeling exchange of ideas, enthusiasm about technology's potential and a tolerance for failure that was unheard of in most of corporate America -- all hallmarks of the distinct culture that eventually became identified with the valley.
Second, restless Fairchild employees -- including the original eight, all of whom had left by 1968, and scores of talented young engineers they hired, trained and inspired -- went on to establish dozens of other companies. These pioneers -- sometimes known as ``the Fairchildren'' -- carried with them not only cutting-edge semiconductor technology, but also the company's distinct culture.
Good neighbors (or "regional network-based industrial system")
The article refers a study done in the 1980s which demonstrated how a "dense network of specialist suppliers" created a better environment to deal with fast-changing technologies (as opposed to the traditional goal of vertical integration). The author called the model a "regional network-based industrial system".
Saxenian undertook the study in the 1980s, she said in an interview, to try to understand why events had disproved the dire predictions she made about Silicon Valley in a late 1970s master's thesis. Saxenian had predicted that high costs and inadequate infrastructure would soon bring the area's growth to a halt.
The answer she came up with, now widely accepted among other authorities, is that the Bay Area had evolved a unique culture and economic framework, one that gave it an unmatched ability to adapt to and profit from fast-changing technologies.
In that model, which she called a "regional network-based industrial system," technology leaders forgo the traditional goal of vertical integration -- producing as many of their own components as possible -- in favor of relying on a dense network of specialist suppliers.
Read More
The article is right here: http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/1999/12/27/BU52171.DTL
I'd also like to share with you a very cool talk, which is about a year old right now, with a vision on how to build a Silicon "Balley" here in Northern Portugal. The speaker is António Murta, a well known Portuguese enterpreneur and angel investor. Video after the jump: http://blog.tedxedges.com/2010/01/tedxedges-antonio-murta-video-portugal.html
While you're at it, why not take the time to revisit the popular The Next Web's post back from May 2010 on how Berlin should be Europe's Silicon Valley: http://thenextweb.com/eu/2010/05/12/why-berlin-needs-to-become-europes-silicon-valley/
(Yours truly on geeky tourism in an undisclosed location in Mountain View)
... don't we all love them.
And I just got myself a brand new one:
- 1 post each week. that's 52 posts throughout 2011.
Doesn't sound too hard, does it?
That's the thing with NYRs, even if you keep them nice and possible, there's always some excuse.
Let me know what your top NYRs are, or maybe share a tip on how to keep them.
Profile
Experience
- Sept 2008 - PresentPartner / Hive SolutionsVast range of demanding activities, covering almost all the spectrum of the companies day-to-day operation. I accumulate a series of roles such as Developer, Business Developer, Pre-Sales Engineer, Sales Rep, Project Manager and Support Engineer. With tasks and responsibilities ranging from developing core technology to doing sales to performing administrative work, PR, community management or corporate networking, this experience has been one of the most exhilarating of my life.
- 2007 - PresentProject Promoter and Lead Developer / Roda.cc Project• Developed and launched a crowdfunding platform focused on Peer-To-Peer Loans. • Kicked-off a private beta with friends and family with real cash and loans. • Wrote a simple yet very effective loan matching system in Prolog to support the platform operation. • Wrote MSc thesis on “Combining Loan Requests and Investment Offers in Peer-To-Peer Lending”, and eventually presented the work in Intelligent Agents and Technologies for e- Business (IAT4EB) workshop in European Conference on Artificial Intelligence (ECAI) 2010. • Failed to obtain regulatory clearance in Portugal, assembled a team with the required skills and handed over the project. • Ruby On Rails, ActiveMerchant, Prolog
- Mar 2007 - PresentProject Engineer / Enabler Wipro
- Sept 2006 - PresentJunior Research Scholar / INESC PortoPerformed software development inside a Research Team. Portal maintenance: Apache Jetspeed. Integrating research work done in Univ. Évora: GNU Prolog, ISCO.
- Mar 2006 - PresentUndergraduate Programming Course Tutor / FEUPGuide and support freshman students' course assignments. C++ and some OO Design.
- Nov 2005 - PresentCollaborator / European Space AgencyProducing the feasibility study for the On Board Data Handler subsystem for the European Student Moon Orbiter. CAN related information analysis, report writing.
- 2004 - PresentFreshman Students’ Project Tutor / FEUPGuide inexperienced project teams through simple projects, geared toward freshman student integration.
Education
-
2002 - 2007FEUPMsC in Software Engineering
Additional Information
I make software that can improve people's lives
