This is a repost of a facebook note I wrote last november, but I still like it, so reposting here

The waterfall (& big design upfront generally) method has been debunked as a good for a long time now. However, what's not generally known is that it was originally a "grandiose" strawman method created to explain the worst possible way of creating a system, and then to explain how it goes wrong, and possible solutions. In essence rather than the paper advocating waterfall method, it advocated (essentially) iterative cycles leading to better systems based around working closely with the customer.

The paper was misread by US military procurement as advocating the approach, and became defined as the way of managing computing projects when bidding for US military funded projects, and hence viewed as a stamp of approval..

The only way someone could misread the paper that way is simple - by only looking at the diagrams and their captions, and in particular just figures 1,2 and 3.

This URL describes quite well how the mistake and misreading came about: http://pascal.gugenberger.net/thoughts/waterfall-accident.html

Original paper is here: http://www.valucon.de/documents/managing_softwareprojects.pdf MANAGING THE DEVELOPMENT OF LARGE SOFTWARE SYSTEMS. Dr. Winston W. Royce

The irony of the original paper is this: it posits waterfall as being a bad idea, and a bad approach. Someone took that as a recommendation to do it - because there was a pretty picture of how it "worked" - and the next 1/4 century (or more) of software development proved the original author correct. Not only that, the inherent assumption in that conclusion - that you absolutely will not understand the implications of your large software system and assuming you can define it "right" first time is madness, and it's better to assume that you will get it wrong, and the sooner you find out how you get it wrong (so you can correct the mistake to build it righter) matters more.

A lot of money could probably have been saved worldwide if the original paper had actually been read rather than skimmed.

What's interesting to me here is this though: any software project that says "design a spec", "gain agreement", "declare a standard" and then build, is doing waterfall. Why do I mention that? Because I still see people doing that today. (Despite many alternatives existing)

I'm not saying every project is "agile ready" - I'd argue actually many projects aren't, but some balance between the two extremes is probably worth finding. (Boehm's spiral is reminiscient, but incomplete for example, VDM ends up being closer to waterfall (where actually applicable it's useful, but ...), and various aspects of Scrum/XP don't really seem to apply well where the project has by definition vagueness, and TDD only makes sense if you have absolute clarity about what you should be testing before you start coding. (When you do, TDD is great IMO))

What is clear though is that if someone has a long time horizon for a software project the assumption that you can spec, design, agree, standardise and then build is fundamentally flawed. Does tend to explain why so many large public service projects fail abysmally, unless people ignore the "rules" and try to do things more iteratively.