CNET had an interesting article last week called Tech Lessons Learned from the Wisdom of Crowds that caused me to chuckle cynically. The article was about using market dynamics to do predictions. At one point in the article the author alluded to attempts by companies to use internal company prediction systems – the Wisdom of Crowds, essentially – to predict the date software products will ship. Here is a quote from Microsoft’s experience:
When Todd Proebsting, director of Microsoft’s Center for Software Excellence, tested a prediction market internally, managers quickly gave it their blessing. The goal: to have 25 members of a development team predict when a Microsoft product would ship (this was an internal product, not one sold externally). The prediction market was set up in August 2004, and the product that “had been in the works for a long time” was scheduled to ship in November 2004…
“The price of ‘before November’ dropped to zero right away,” Proebsting said. “The price of ‘on time’ in about two to three minutes dropped to 2.3 cents on the dollar.” Translated, that’s more than 30-to-1 odds against on-time delivery…
“The next day the director comes into my office and said, ‘What have you done?’” Proebsting said. But further investigation showed that the product actually was behind schedule, even though nobody was telling management, and it eventually shipped in February.
SHOCKING! A PRODUCT AT MICROSOFT WILL BE LATE!
Ok, so it isn’t fair to pick on Microsoft. Software always ships late and building software is hard to predict. What I find amazing is that people feel you need a sophisticated market system to predict this. Let me suggest an alternative system.
Kevin Dewalt’s (patent pending) prediction system for determining when software will ship:
Ask the development team.
You see, software doesn’t build itself. People build it. People who usually know what they are doing and are very close to the problems. Unfortunately, the leadership at big companies isn’t close to the problems. Leadership “knows the big picture” – like dates, budget, and sometimes the name. Maybe even the scope. But more than anything else, leadership knows the date. “The next release will happen on June 1st!”
Why do they know the date? The date for delivery in big companies is usually decided in management meetings. Sometimes management gets input from the toiling masses, sometimes not. At these sort of meetings the project manager usually gets persuaded into committing to a date. The circumstances around these types of decisions and the unfortunate impact on software development teams is eloquently discussed in Death March by Edward Yourdon.
In any case, here’s the reality: the developers, testers, and others on the team know whether a company is going to hit the date. Unfortunately, nobody asks them because management doesn’t want to hear the truth. It happens every day in every large company. Leaders don’t want bad news, they want to believe that everything will work out according to the agreement at the meeting.
Ok, so you’re in management at a large company. Hopefully you’ve had some experience as a developer or software project manager. You’ve felt the pain of tradeoffs between scope and time and you know that technology has a funny way of not working very well. In any case, before you suggest building some innovative market prediction system, try strolling down the hall and talking to the development team. They may be very reluctant to talk to you at first, but if you ask them questions about the challenges and problems on the project I guarantee they will open up. They probably won’t tell you explicitly if the project will be late, but you will be able to figure it out for yourself with some patience and careful listening.
I guarantee you that you’ll quickly realize that the problem in most corporate development efforts is not the absence of a sophisticated prediction system: it is the lack of trust and frank dialog between developers and management.
