I found the reference to Dijkstra interesting because he was a huge advocate for up-front thinking. The difference between Dijkstra and Waterfall is Dijkstra wanted the programmer to think before working on the feature. The order in which features were developed was at a higher level. Waterfall is about listing all the features beforehand and parceling them out to be implemented or scheduled on a monster Gantt chart.
A formal methods based process can definitely be agile, so long as the planning has a feedback loop that includes iteratively-developed finished products.
It is clear to me that formal methods are avoided because they are difficult. It is very hard (I've tried) to learn to verify programs without formal education in the methods. Yet, we can write software without formal methods and we can make it pretty darn correct without proving it correct.
Formal methods will probably not catch on unless a concerted effort is made by CS baccalaureate programs to teach it from the beginning. I fear that most professors are not proficient enough to teach it. So, it will probably be one of those things that may be a good idea but will be unlikely to ever gain traction.
There's a chicken and egg problem with formal methods, which is that they're really hard, so nobody does them. But part of the reason they're really hard is that nobody does them, so not much time has been spent optimizing the process of generating them, and when you sit down to write a proof you don't have large libraries of already-proved code to draw from. Does your XML parser carry any proofs? Heck no.
A few people seem to be doing research on this problem, like the authors of Coq, but until the chicken/egg problem is solved I'm not optimistic about it even penetrating the obscure language community, let alone the mainstream. Plus nobody is really sure just how far we can go in making it easier, if we were able to share libraries and common practices; it will always be a challenge, certainly, just as writing good code is today anyhow, but can it ever exceed breakeven in terms of costs/benefits for a significant fraction of the code in the world? I'm cautiously pessimistic, but would love to be proved wrong.
The manifesto and principles should be required reading for all software engineers. It's really great to see how widespread adoption has been.
In the early 2000s I helped National Academies/CSTB workshop on creating dependable software, and put Kent Beck and a formal methods advocate from Praxis on the same panel. We expected fireworks but they were actually in agreement on most things :-)
I disagree. This article is full of wisdom like "They had learned how to Do Agile, but they didn't know how to Be Agile.
These are the best and brightest Agile evangelists? Why so many failures then? The article bemoans how Agile hasn't taken off as it should.
I humbly submit: because its not as good as it sounds. Everything will get done on a Friday! Really? how likely is that? Interrupting intensely concentrating programmers on a schedule - that will improve productivity?
I'm a little bitter. Been at two shops now that torpedoed productivity by "going Agile".
I tend to agree with you. This sounds awful, and cynical.. but in my experience formal Agile methods are a great way to get acceptable results at an acceptable pace with a team of mediocre developers... It's sad that it's constantly only compared to waterfall (the other way of building disappointing software).
There was a Kettering quote on HN yesterday that immediately reminded me of Agile.. “If you want to kill any idea in the world, get a committee working on it.”
with a team of mediocre developers, you're doing well to get acceptable results at an acceptable pace.
with a team of good developers, agile can be great if you do it well: the fastest way of getting high-quality stuff in users' hands.
if you've got at least one great developer (or a mix of experienced/inexperienced), agile's even better: the emphasis on shared ownership brings up the level of the team.
and of course it can be done really badly, and often is. if all your experiences are bad, i can see why you're cynical.
"with a team of mediocre developers, you're doing well to get acceptable results at an acceptable pace."
I totally agree. It's just not any fun. I also agree that shared ownership can bring up the overall level of the team, but I think it brings down the quality of work getting built by the "one great developer".
My experiences weren't bad, just boring. It seems like a good process for getting reasonably predictable outputs out of a variety of inputs (where the inputs are programmers).
I prefer to work in environments where there's a bit more specialization, more individual ownership (of components, certainly shared ownership of the 'whole'), and where developers are crafts(wo)men.
To me, the worst part about the growth of Agile is that I think fewer mediocre developers are becoming great ones.. because Agile ensures that no one learns to build something on their own. A quality that's in disappointingly short supply in certain areas (like my hometown of San Diego) where software development is rarely treated as a craft, and instead treated as an engineering effort.
A lot of shops "go agile" really badly. There are many ways to fail, for example the emphasis on avoiding unnecessary documentation can turn to an excuse for not doing enough up-front thinking. And if you embark on a TDD path and then don't write your unit tests, you've got a quality disaster.
That said there are a lot of agile successes: projects and companies, techniques (team ownership, SCRUM), and philosophies (the minimum viable product). So don't generalize too much from your experience.
Yeah I agree - I have worked in a few agile shops too and I found it to be highly disruptive and counter-intuitive to how I work. I had to get 'Agile' or leave I was told. Eventually I did just that.
I've always had the impression Agile was just another crutch bad programmers lean on. Or maybe it's just bad managers?
A formal methods based process can definitely be agile, so long as the planning has a feedback loop that includes iteratively-developed finished products.
It is clear to me that formal methods are avoided because they are difficult. It is very hard (I've tried) to learn to verify programs without formal education in the methods. Yet, we can write software without formal methods and we can make it pretty darn correct without proving it correct.
Formal methods will probably not catch on unless a concerted effort is made by CS baccalaureate programs to teach it from the beginning. I fear that most professors are not proficient enough to teach it. So, it will probably be one of those things that may be a good idea but will be unlikely to ever gain traction.