The Zen of Debugging

There was this senior engineer who once used to sit in the cubicle next to mine. He had an anecdote pasted on the wall of his cubicle. His purpose in putting it there was to educate newbies like me. Thought I’d share the wisdom. Story goes like this.

General Motors once received a complaint from a customer who claimed that his new Pontiac was allergic to vanilla ice-cream. The customer wrote, “This is the second time I have written you, and I don't blame you for not answering me, because I kind of sounded crazy, but it is a fact that we have a tradition in our family of ice cream for dessert after dinner each night. But the kind of ice cream varies so, every night, after we've eaten the whole family votes on which kind of ice cream we should have and I drive down to the store to get it. It's also a fact that I recently purchased a new Pontiac and since then my trips to the store have created a problem. You see, every time I buy vanilla ice cream, when I start back from the store my car won't start. If I get any other kind of ice cream, the car starts just fine.” So GM sent an engineer from the product team to check it out.

The engineer was surprised to be greeted by a successful, obviously well-educated man in a fine neighbourhood. He had arranged to meet the man just after dinner time, so the two hopped into the car and drove to the ice cream store. It was vanilla ice cream that night and, sure enough, after they came back to the car, it wouldn't start. The engineer returned for three more nights. The first night, the man got chocolate. The car started. The second night, he got strawberry. The car started. The third night he ordered vanilla. The car again failed to start.

All the while the engineer was taking down all sorts of data - time of day, type of gas used, time spent at the store before restarting etc. He noticed a correlation - on the days when the customer bought vanilla ice-cream and the car failed to start, the time spent at the store was much less. So the engineer rephrased the problem – why is it that the car will not restarted within a short period of time after the engine is turned off?

Once the problem was defined in terms of restart interval, and not ice cream flavour, the experienced engineer immediately realized vapour lock was causing the issue. (The engineer later learned that Vanilla, being the most popular flavour, was in a separate case at the front of the store for quick pick up. All the other flavours were kept in the back of the store at a different counter where it took considerably longer to find the flavour and get checked out.)

So how did this engineer analyse the root cause of a seemingly impossible bug? To start with, he did not dismiss the issue as being impossible or the customer as being a jerk. He diligently took data without jumping to conclusions or offering a quick fix work around. He knew what parameters to look for. He saw a pattern in the data. Finally he defined the problem – restart interval. The rest was trivial.

Defining the problem, asking the right question, that is the key to problem solving, the Zen of Debugging.


PS: In my first year at work, I remember being in the office at about 3am trying to debug a particularly tricky issue that had been around for a week and was threatening to kill my maiden feature. That’s when I first noticed this anecdote pasted in my neighbours cubicle. It helped me to step back, relax, see the facts and rephrase the problem. The issue was fixed before sunrise. Hope others find this useful too.

Five years on

Five years ago, on a rainy Friday afternoon, I was just coming out of an internet café in Thiruvella when the mobile rang. There had been no new mail in my inbox, so I wasn’t really expecting a phone call from Bangalore. The interesting thing about a CUSAT M.Tech is that you get to spend one full year doing research, either in academia or in industry. The dot-com gloom had lifted and after five years of classrooms, I couldn’t wait to get into a cubicle. So I answer the phone, wishing that it would be an interview invite.

The lady on the other end introduced herself as a HR person from Motorola. Asked if I could be in Bangalore by Monday. Motorola office, C.V. Raman Nagar. At nine in the morning. To join.

Wait…What! Join! No interview? I was confused, not sure if this was for real. Even after I got an email a few hours later with joining instructions, I wasn’t really convinced.

Sunday evening, I packed my trusty old travel bag and took a bus to Bangalore. Monday morning, Motorola office. Few of my classmates were already in the lobby. Sometime later, we got our joining letters. There was a collective sigh of relief. We were now officially in the corporate world. By lunch time, we were introduced to the hiring manager, who bought us lunch. And then we spent the rest of the day exploring the coffee machine.

That was my first day at work. It was a strange feeling. The thought of earning a salary. Of free coffee and urinals that flushed automatically. A feeling of having arrived and yet not having reached.

As I look back at the last five years, I can’t help but be grateful, for the opportunity to touch a million lives, for the excitement of a product release, for friendships that will last a lifetime, for hope and despair, for success and failure alike, for every single day of this exciting journey. Hallelujah!

The Brown Swan

An Indian film wins 4 awards at the Golden Globe and then goes on to bag 10 Oscar nominations. It is not really an Indian production, such a thing has never happened before and most people out here hadn’t even heard of the film. Now that is what I call a perfect The Black Swan. No one saw it coming.

But then, how could it not be a hit. A R Rahman, Danny Boyle, Simon Beaufoy, Fox Searchlight, a feel-good film and the worst economic crisis in most people’s living memory. With the world going through such difficult economic times, a feel good film about money, about how it’s only the means and never an end in itself, that is just the kind of movie everyone so desperately wants to watch right now. So seemingly obvious in retrospect.

Slumdog Millionaire is about fate, attitude and human nature. It’s not really about India or its slums. It’s a lot like my other favorite feel-good movie – The Pursuit of Happyness starring Will Smith and based on the true story of Chris Gardner.

Slumdog is about the startling success of the underdog, a kid who never went to school winning millions on a Quiz Show where doctors, engineers and professors have failed. Now consider this. An independent production, a director whose last movie didn’t exactly do too well, a cast that is virtually unknown to international audiences goes on to win so many awards where major Indian cinema houses and superstars have failed. Notice the parallel.

In the movie, instead of admiring Jamal’s success at the quiz, the game show host suspects him of cheating. In real life, instead of admiring the success of the film as one would expect, many Indians criticized the film for supposedly maligning the image of an otherwise shining India by showing the slums. As a wise man once said, “As is the microcosm so is the macrocosm.” The same pattern repeats infinitely at every level of observation and as you look deeper you see infinite complexity, the picture of a fractal and chaos in all its ethereal beauty. Finally, enlightenment emerges from the fluttering wings of a butterfly. Or maybe not.

History in the making

What would I do if a friend from school asked me to drop my career and join his startup? What would I do if I had a vision that I really believed in? Not just a great idea, but a vision of how the world would be 20 years from now. And the opportunity to be in the driver’s seat and make it happen. Do I leave everything and go after it? Or do I sigh and get back to my life and let somebody else grab the opportunity? I doubt if I’d have the courage to leave everything and follow a dream. But this man, and his friends, had the courage and conviction to act on a vision so far out that, at the time, even they could not imagine the enormity of it.

Yes, I’m talking about Bill Gates. Today is his last day as a full time employee of the company he founded. In an employee town hall a couple of hours back, he and Steve Ballmer talked about how they met and became friends. About the time the two of them went out to watch 'A Clockwork Orange'. How Bill convinced Steve to drop out of his MBA to join Microsoft. About how in spite of its size, Microsoft is a lean and agile company; and how they expect things to be down the road. And the foundation and the work it’s doing. But the most memorable moment would be the time when, towards the end of the brief talk, Steve became overwhelmed with emotion as he talked about his gratitude to Bill; and Bill who had managed to remain composed until then was seen wiping away tears.

The Monsoons

For all it was hyped up to be, the Y2K bug turned out to be a damp squib. The world didn’t come to an end, computer everywhere didn’t crash and IT was going full steam ahead. But the turn of the millennium was jinxed anyway. Within a year, the dotcom bubble burst washing away billions in tech stocks and thousands of jobs. It was my 3rd year in college. That year, freshers wouldn’t touch a Computer Science degree with a flagpole. Students opted out of Comp Sc departments for of civil engineering, something unthinkable only a year back. Seats in the best Comp Sc programs were available to almost anyone with a decent score. Such opportunities come very rarely, and often at great cost. A lot of tech companies had put job offers on hold. Some withdrew offers and cut headcount. Some gain, some loose, nobody gets credit. By the time I graduated, the industry was beginning to show signs of improvement but I decided to play it safe and stay in university. Oddly enough, the competition for Masters Programs in CS was at an all time high because of all the people leaving, or not entering, the workforce. A year later, by 2004, the clouds had passed and hiring was again in full swing.

Everybody didn’t live happily ever after though, because that happens only in fairy tales. Early 2006, Motorola stocks started sliding down as earnings failed expectations. This week it is down to a five year low and I’m left with a few costly lesson in finance. Lesson one, don’t put all your eggs in one basket. Try not to invest in the company you work for, infact avoid the sector entirely unless you know exactly what you are doing. Lesson two, don’t overlook brokerages and expenses because they can take away a good part of your investment. Three, learn from experience, particularly other peoples experience (Y2K anyone?). Four, low returns do not imply low risk. In 2005, MOT shares were expected to grow at 12% which is not much compared to the Sensex at that time. Instead, MOT share have since fallen by over 60%. Five, have a stop loss mechanism in place. Irrational optimism doesn’t pay any bills.

Meanwhile, we are somewhat back to where we started. It seems the US economic crisis might extend into 2009 dragging the rest of the world with it. Housing prices in the US have fallen 15% over the last one year. It is not entirely inconceivable that something similar might happen in India. On the employment front, it is the finance sector taking most of the cuts so far. Tech seems to be in fair weather. Except for MOT. And Symbian, because news has it that Nokia plans to buy the whole of Symbian and set it free. UIQ, which is now half owned by MOT, will also be joining Symbian at the Symbian Foundation. Not as romantic as it sounds. Free doesn’t pay any bills either.

The Monsoons are here. Its fun to watch the rain. Getting drenched may also be fun. Even catching a cold may not be so bad. But soon I'll miss the mangoes.

Church 2.0

One afternoon some 7 years ago, I was casually going over the notice boards in the main building at MA College of Engg in Kothamangalam. It must have been my third year at MACE. My attention was drawn to a small notice about a mailing list on yahoogroups. So out of curiosity and a desire to know what going on in the community, I subscribed to IndianOrthodox@yahoogroups.com from the college internet centre. Over the years ICON, as it later came to be known, has grown to over 6000 members from around the world (including a few bishops and many priests) and matured into a forum for serious discussion.

In grad school (yes, at CUSAT we call them schools), I started playing around with social networks. The following year I got an invite to Orkut from one of my fellow trainees at Motorola, where I was doing a rather long internship. Someone had just started an Indian Orthodox community in Orkut and I too joined in, though I rarely went through any of the posts. Recently, somewhat to my surprise, the community crossed the 10,000 member mark to be one of the largest communities of its kind on any social networking site. Although there is no way to know for sure, the vast majority of members must be in their teens or twenties.

A couple of months back I moved to Hyderabad and it was the week before Pasha. The move happened a few weeks earlier than I had planned and in the rush of things I didn’t get the chance to ask anyone for the address of an Orthodox church in Hyderabad. So now I was in a city that I barely knew, I need to find a church and I didn’t have the faintest clue whom to ask. So I turned to the internet, which had over the years gone from mailing lists to Web 2.0 - clouds, SNS, wikis et al. I started with a search on Live which gave me a rough idea of what I was looking for. I grabbed whatever info I could find and switched over to Orkut where I found three Orthodox churches with full addresses and alternate names of localities, but unfortunately there was no data on service timings. So now I had the address of 3 churches in Hyderabad, I had to figure out which was nearest and how to get there. So I go to Wikimapia, zoom in to Hyderabad, and search for the street or locality of each of the three churches. A few minutes of zooming and panning later, I locate the nearest church on a map of Hyderabad superimposed on a composite high resolution satellite image of the city. Someone I will probably never know had taken the time and effort to locate and mark the small church on the outskirts if the city. I then do my bit by adding some more details to the wiki tag. I then take a print out of the map, mark the location with an X and hand it over to my cab driver. Next morning, I reach St. Mary’s Orthodox Church at Ramachandrapuram just in time for the Hosanna service.

Dark Brew

The other day I was having breakfast with a colleague in the office cafeteria. The conversation went something like this:
Hey! What’s that you have there? (Pointing at my coffee)
Coffee. With chocolate. LOTS of chocolate.
Coffee and chocolate??? (Coffee and chocolate don't mix too well in the resident Indian psyche)
Really! It’s a great combination. (With that - you gotta try it - look)
Could you make me some of that? (Politically correct what!)
Yeah! Sure. Come on, I'll show you how it’s done.

So we walk over to the coffee machine and I go like:
3 sachets chocolate powder (150g)
2 sachets Bru instant 70-30 coffee-chicory mix (Pure instant coffee isn't as good)
2 sachets sugar substitute (Have to stay within the calorie quota - Sedentary Male, South Asian)
200ml hot milk
Mix well. There you go.

We return to breakfast. I half-jokingly tell him that I moved to Microsoft for the complimentary chocolate. But seriously, you can always judge an office by its cafeteria. I guess it was Napoleon who said that an army marches on its stomach. Things haven't changed all that much in the last 300 years.