THE COMPLETE TITLE of In Search of Stupidity includes the phrase “High- Tech Marketing Disasters,” and from these words you might conclude that it’s a firm’s marketers who usually bear the chief responsibility for major corporate catastrophes. This isn’t true. To be worthy of mention in this book, it took the combined efforts of personnel in upper management, development, sales, and marketing, all fiercely dedicated to ignoring common sense, the blatantly obvious, and the lessons of the past. Major failure doesn’t just happen: To achieve it, everyone must pull together as a team.
Chapter 4 of In Search of Stupidity helps drive this point home. For MicroPro to plummet from the software industry’s pinnacle to permanent oblivion took a) upper management’s mishandling of development and market timing, b) the marketing department’s idiotic decision to create a fatal product-positioning conflict, and c) the development team’s dimwitted decision to rewrite perfectly good code at a critical time because it wanted to write even better code that no one really needed. A magnificent example of different groups within a company all cooperating to ensure disaster.
In this spirit, I’ve decided to include selected portions of an interview with Joel Spolsky that ran on Softletter. (By the way, this interview was “picked up” by Slashdot [http://www.slashdot.org], a website dedicated to technology, coding, open source, and all things nerd. It generated a considerable amount of comment and controversy. You can search the Slashdot archives to read what other people thought and gain further insight into Joel’s opinions.)
I regard Joel Spolsky, president and one of the founders of Fog Creek Software (http://www.fogcreek.com), as one of the industry’s most fascinating personalities. He worked at Microsoft from 1991 to 1994 and has more than 10 years of experience managing the software development process. As a program manager on the Microsoft Excel team, Joel designed Excel Basic and drove Microsoft’s Visual Basic for Applications (VBA) strategy. His website, Joel on Software (http:// www.JoelonSoftware.com), is visited by thousands of developers worldwide every day. His first book, User Interface Design for Programmers (Apress, 2001), was reviewed by me and I regard it as a must-have for anyone involved in developing and marketing software.
Why this interview? If you’ve ever worked on the software side of high technology, you’ve probably experienced the following: After a careful analysis of your product’s capabilities, the competition, and the current state of the market, a development and marketing plan is created. Release time frames are discussed and agreed upon. Elaborate project management templates are built, and milestones are set. You post the ship date up on a wall where everyone in your group can see it, and your team begins to work like crazed beavers to meet your target.
Then, as the magic day looms nearer, ominous sounds emit from development. Whispers of “crufty code” and “bad architecture” are overheard. Talk of “hard decisions” that “need to be made” starts to wend its way through the company grapevine. People, especially the programmers, walk by the wall on which you’ve mounted the ship date, pause, shake their heads, and keep walking.
Finally, the grim truth is disgorged. At a solemn meeting, development tells everyone the bad news. The code base of the current product is a mess. Despite the best and heroic efforts of the programmers, they’ve been unable to fix the ancient, bug-ridden, fly-bespeckled piece of trash foisted on them by an unfeeling management. No other option remains. The bullet must be bitten. The gut must be sucked up. The Rubicon must be crossed. And as that sinking feeling gathers in your stomach and gains momentum as it plunges toward your bowels, you realize that you already know what you’re about to hear. And you already know that, after hearing it, you’ll be groping blindly back to your cubicle, your vision impeded by the flow of tears coursing down your face, your eyes reddened by the sharp sting of saline. And you’ve already accepted it’s time to get your resume out and polished, because the next few financial quarters are going to be very, very ugly.
And then they say it. The product requires a ground-up rewrite. No other option exists.
Oh, you haven’t been through this yet? Well, just wait. You will. However, as you’ll learn, what you’re going to be told may very well not be true. After reading this interview, you’ll be in a better position to protect your vision and your career in the wonderful world of high tech.
And now . . .
Rick Chapman: Joel, what, in your opinion, is the single greatest development sin a software company can commit?
Joel Spolsky: Deciding to completely rewrite your product from scratch, on the theory that all your code is messy and bug-prone and is bloated and needs to be completely rethought and rebuilt from ground zero.
Uh, what’s wrong with that?
Because it’s almost never true. It’s not like code rusts if it’s not used. The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they’ve been fixed. There’s nothing wrong with it.
Well, why do programmers constantly go charging into management’s offices claiming the existing code base is junk and has to be replaced?
My theory is that this happens because it’s harder to read code than to write it. A programmer will whine about a function that he thinks is messy. It’s supposed to be a simple function to display a window or something, but for some reason it takes up two pages and has all these ugly little hairs and stuff on it and nobody knows why. OK. I’ll tell you why. Those are bug fixes. One of them fixes that bug that Jill had when she tried to install the thing on a computer that didn’t have Internet Explorer. Another one fixes a bug that occurs in low-memory conditions. Another one fixes some bug that occurred when the file is on a floppy disk and the user yanks out the diskette in the middle. That LoadLibrary call is sure ugly, but it makes the code work on old versions of Windows 95.
When you throw that function away and start from scratch, you are throwing away all that knowledge. All those collected bug fixes. Years of programming work.
Well, let’s assume some of your top programmers walked in the door and said, “We absolutely have to rewrite this thing from scratch, top to bottom.” What’s the right response?
What I learned from Charles Ferguson’s great book (High [email protected], No Prisoners [Crown, 1999]) is that you need to hire programmers who can understand the business goals. People who can answer questions like “What does it really cost the company if we rewrite?” “How many months will it delay shipping the product?” “Will we sell enough marginal copies to justify the lost time and market share?” If your programmers insist on a rewrite, they probably don’t understand the financials of the company, or the competitive situation. Explain this to them. Then get an honest estimate for the rewrite effort and insist on a financial spreadsheet showing a detailed cost/benefit analysis for the rewrite.
Yeah, great, but, believe it or not, programmers have been known to, uh, “shave the truth” when it comes to such matters.
What you’re seeing is the famous programmer tactic: All features that I want take 1 hour, all features that I don’t want take 99 years. If you suspect you are being lied to, just drill down. Get a schedule with granularity measured in hours, not months. Insist that each task have an estimate that is 2 days or less. If it’s longer than that, you need to break it down into subtasks or the schedule can’t be realistic.
Are there any circumstances where a complete code rewrite is justified?
Probably not. The most extreme circumstance I can think of would be if you are simultaneously moving to a new platform and changing the architecture of the code dramatically. Even in this case you are probably better off looking at the old code as you develop the new code.
Hmm. Let’s take a look at your theory and compare it to some real-world software meltdowns. For instance, what happened at Netscape?
Way back in April 2000, I wrote on my website that Netscape made the single worst strategic mistake that any software company can make by deciding to rewrite their code from scratch. Lou Montulli, one of the five programming superstars who did the original version of Navigator, e-mailed me to say, “I agree completely; it’s one of the major reasons I resigned from Netscape.” This one decision cost Netscape 4 years. That’s 3 years they spent with their prize aircraft carrier in 200,000 pieces in dry dock. They couldn’t add new features, couldn’t respond to the competitive threats from IE, and had to sit on their hands while Microsoft completely ate their lunch.
OK, how about Borland? Another famous meltdown. Any ideas?
Borland also got into the habit of throwing away perfectly good code and starting from scratch. Even after the purchase of Ashton- Tate, Borland bought Arago and tried to make that into dBASE for Windows, a doomed project that took so long that Microsoft Access ate their lunch. With Paradox, they jumped into a huge rewrite effort with C++ and took forever to release the Windows version of the product. And it was buggy and slow where Paradox for DOS was solid and fast. Then they did it all over again with Quattro Pro, rewriting it from scratch and astonishing the world with how little new functionality it had.
Yeah, and their pricing strategy didn’t help.
While I was on the Excel team, Borland cut the MSRP on Quattro Pro from around $500.00 to around $100.00. Clueless newbie that I was, I thought this was the beginning of a bloody price war. Lewis Levin,1 Excel BUM (business unit manager) was ecstatic. “Don’t you see, Joel, once they have to cut prices, they’ve lost.” He had no plan to respond to the lower price. And he didn’t need to.
Having worked at Ashton-Tate, I have to tell you the dBASE IV code base was no thing of beauty. But, I take your point. Actually, I saw this syndrome at work in Ashton-Tate’s word-processing division. After they bought MultiMate, they spent about 2 years planning a complete rewrite of the product and wasted months evaluating new “engines” for the next version. Nothing ever happened. When a new version of the product was released, it was based on the same “clunky” engine everyone had been moaning about. Of course, in those 2 years WordPerfect and Microsoft ate Ashton-Tate’s wordprocessing lunch.
Ashton-Tate had a word processor?
Yes, but nothing as good as WordStar, mind you!
Hmm. That reminds me that Microsoft learned the “no rewrite” lesson the hard way. They tried to rewrite Word for Windows from scratch in a doomed project called “Pyramid,” which was shut down, thrown away, and swept under the rug. Fortunately for Microsoft, they did this with parallel teams and had never stopped working on the old code base, so they had something to ship, making it merely a financial disaster, not a strategic one.
OK, Lotus?
Too many MBAs at all levels and not enough people with a technical understanding of what could and needed to be built. SMS: And I suppose building a brand-new product called “Jazz”2 instead of getting 1-2-3 over to the Mac as quickly as possible, thus staking Microsoft to a 2-year lead with Excel, is an example of the same thing?
Actually, they made a worse mistake: They spent something like 18 months trying to squeeze 1-2-3/3.0 into 640KB. By the time the 18 months were up, they hadn’t succeeded, and in the meantime, everybody bought 386s with 4 megs of ram. Microsoft always figured that it’s better to let the hardware catch up with the software rather than spending time writing code for old computers owned by people who aren’t buying much software any more.
WordPerfect?
That’s an interesting case and leads to another development sin software companies often make: using the wrong level tools for the job. At WordPerfect, everything, including everything, had to be written in assembler. Company policy. If a programmer needed a little one-off utility, it had to be hand-coded and hand-optimized in assembler. They were the only people on Earth writing all-assembler apps for Windows. Insane. It’s like making your ballerinas wear balls and chains and taping their arms to their sides.
What should they have been coding in?
In those days? C. Or maybe Pascal. Programmers should only use lower-level tools for those parts of the product where they are adding the most value. For example, if you’re writing a game where the 3D effects are your major selling point, you can’t use an off-the shelf 3D engine; you have to roll your own. But if the major selling point of your game is the story, don’t waste time getting great 3D graphics—just use a library. But WordPerfect was writing UI code that operates in “user time” and doesn’t need to be particularly fast. Hand-coded assembler is insane and adds no value.
Yes, but isn’t such code tight and small? Don’t products built this way avoid the dreaded “bloatware” label?
Don’t get me started! If you’re a software company, there are lots of great business reasons to love bloatware. For one, if programmers don’t have to worry about how large their code is, they can ship it sooner. And that means you get more features, and features make users’ lives better (if they use them) and don’t usually hurt (if they don’t). As a user, if your software vendor stops, before shipping, and spends 2 months squeezing the code down to make it 50 percent smaller, the net benefit to you is going to be imperceptible, but you went for 2 months without new features that you needed, and that hurt.
Could this possibly account for the fact that no one uses WordStar version 3.3 anymore despite the fact it can fit on one 1.4 meg floppy?
That and Control-K. But seriously, Moore’s law makes much of the whining about bloatware ridiculous. In 1993, Microsoft Excel 5.0 took up about $36.00 worth of hard drive space. In 2000, Microsoft Excel 2000 takes up about $1.03 in hard drive space. All adjusted for inflation. So stop whining about how bloated it is.
Well, we’ve had much personal experience with the press slamming a product we were managing. For example, for years reviewers gave MicroPro hell over the fact it didn’t support columns and tables. Somehow the fact that the product would fit on a 360KB floppy just didn’t seem to mean as much as the idea that the reviewer couldn’t use our product to write his or her resume.
There’s a famous fallacy that people learn in business school called the 80/20 rule. It’s false, but it seduces a lot of dumb software start-ups. It seems to make sense. Eighty percent of the people use 20 percent of the features. So you convince yourself that you only need to implement 20 percent of the features, and you can still sell 80 percent as many copies. The trouble here, of course, is that it’s never the same 20 percent. Everybody uses a different set of features. When you start marketing your “lite” product and you tell people, “Hey, it’s lite, only 1MB,” they tend to be very happy, then they ask you if it has word counts, or spell checking, or little rubber feet, or whatever obscure thing they can’t live without, and it doesn’t, so they don’t buy your product.
Let’s talk about product marketing and development at Microsoft. How did these two groups work together?
Well, in theory, the marketing group (called “product management”) was supposed to give the development team feedback on what customers wanted. Features requests from the field. That kind of stuff. In reality, they never did.
Really?
Really. Yes, we listened to customers, but not through product management—they were never very good at channeling this information. So the program management (design) teams just went out and talked to customers ourselves. One thing I noticed pretty quickly is that you don’t actually learn all that much from asking customers what features they want. Sure, they’ll tell you, but it’s all stuff you knew anyway.
You paint a picture of the programmer almost as a semideity. But in my experience, I’ve seen powerful technical personalities take down major companies. For instance, in The Product Marketing Handbook for Software (Aegis Resources, 2006), I describe how the MicroPro development staff’s refusal to add the aforementioned columns and table features to WordStar badly hurt the product’s sales.3 How do you manage situations like these?
This is a hard problem. I’ve seen plenty of companies with prima donna programmers who literally drive their companies into the ground. If the management of the company is technical (think Bill Gates), management isn’t afraid to argue with them and win—or fire the programmer and get someone new in. If the management of the company is not technical enough (think John Sculley), they act like scared rabbits, strangely believing that this one person is the only person on the planet who can write code, and it’s not a long way from there to the failure of the company.
If you’re a nontechnical CEO with programmers who aren’t getting with the program, you have to bite the bullet and fire them. This is your only hope. And it means you’re going to have to find new technical talent, so your chances aren’t great. That’s why I don’t think technology companies that don’t have engineers at the very top have much of a chance.
Joel, thank you very much.