Would-be agile teams sometimes get stuck in half-waterfall, a combination of waterfall and agile.
Half-waterfall has the costs of both agile and waterfall approaches, but may not get enough benefit to offset the extra cost. Or, it may be a temporary penalty for a transitional stage, letting a team practice some aspects of incremental development.
Waterfall
A traditional waterfall process has phases:
Analyze → Design → Code → Test → Release
Each phase completes its work and then feeds into the next. Waterfall teams with any experience recognize that this is only an ideal, and that work sometimes needs to flow backwards. For example, a test might reveal a deep error in the design, requiring code to be unwound to to support a better approach.
Agile
An agile approach is more “all-at-once”, blending waterfall’s phases into near-simultaneous activities that take place in a short span commonly known as a sprint or iteration. Each of these is followed by a release, on a short cycle. (Many teams use two weeks, some a month, some release every checkin.)
Analyze/Design/Code/Test/Release → ADCTR → ADCTR…
Agile teams work cooperatively. They attempt to address the most valuable and/or risky stories first. They actively seek user involvement and feedback.
Benefits of an Agile Approach
Ongoing releases have several benefits:
- Faster feedback and learning – It’s hard for users (and everybody else) to get a good understanding of what software will be liike. Early releases allow for faster feedback and faster learning.
- Allow changes in direction – An agile team expects to respond to feedback – driving the system toward the latest understanding of what it needs to be, rather than locking in the understanding we started with.
- Lower investment – Delivering sooner means that the key benefits arrive sooner, and often means that payment is sooner too. Financially, an agile team can be closer to “pay as you go” rather than “hope the big investment pays off”.
- Smaller changes typically have less risk – Less of the system changes on any release, and agile teams test intensely, so each release is less risky.
- Whole-team alignment – Since the whole team works together throughout the project, and gets feedback as they go, it’s easier for them to align on direction.
Half-Waterfall, Half-Agile
When you start with big work up-front, then “throw it over the wall” to a team using a more iterative style, you end up in a half-waterfall.
The most common version of this is:
Analyze → DCT → DCT → DCT → DCT … → Release
(Occasionally you see a team doing:
Analyze → Design → CT → CT… → Release
But we’re not going to discuss that case further.)
Comparison to Agile
How does half-waterfall, half-agile compare to an agile approach?
- Feedback is slower – We get some feedback as we iterate, but we’re just as cut off from user and customer feedback as we would be in a waterfall approach. We’ll learn some things, but not as quickly or as well as if we had more feedback.
- Changes in direction are treated as problematic – With analysis and planning up-front, this process tends to be the domain of “plan the work, work the plan” and “burndown charts”.
- Higher investment – Since we don’t release early, we probably don’t have the option of charging early.
- Big risk – All the accumulated risk throughout the project is riding on the big release at the end.
- Trickier alignment – Even if the development side gets aligned, the planning, product management, user interface, and other up-front players tend not to be involved.
The Lean Startup people (see References) tend to say “Get Out of the Building”: we can’t be effective by thinking hard, without feedback from the target audience.
“Prediction is very difficult, especially about the future.”
—Niels Bohr
“Prediction is very difficult, especially about the future.” – Niels Bohr
Change is hard to judge – not only does deploying software change the context, but technology changes, people change, expectations change – the world is not stable.
Variation: Release Often
What if the process looks like this?
Analyze → Design/Code/Test/Release → DCTR → DCTR…
That is, an up-front analysis, followed by a series of iterations that each end in release.
It’s better, but still has challenges compared to a fully agile approach:
- Partial feedback and learning – The “up front” people (product managers, analysts, UI folks, etc.) aren’t there to receive the feedback. So even if the development side learns, it’s a struggle to communicate what they learned to the next release. The learning is often too late to do any good.
- The team doesn’t learn to work as a whole team, and with a full project community.
- “Steering” is often still missing. A long series of planned sprints is built around the idea “we don’t expect to learn much, so we don’t expect to change our plans”. This is a self-fulfilling prophecy.
There is one key upside to this approach: The development team has a chance to build skills. They learn to work in a more incremental way. For example, if there are issues with releasing, they’re forced to confront and address those early.
Would You?
If a team can use iteration on the development work as a way to build the skill of incremental development and delivery, and knows the compromise they’re making, they can still get some benefits. But they’ll have challenges, e.g., being held to unrealistic deadlines rather than actively adjusting the project.
Circumstances do vary; you’ll have to experiment where you can and use your judgment about the best way to work.
On a couple occasions only, I have seen teams where the requirements really were fully understood up front, and the team developed incrementally. One case was a rewrite, though about 40 or 50% through the project, they worked out a way they could have refactored and refreshed incrementally. The other was an embedded standards-based product in which the product management and management teams felt that no subset would be accepted by customers. (I believe they still could have gotten feedback on “prototype” deliveries, but now we’re in hypotheticals.)
Conclusion
Entrenched organizations often try to keep their old processes in place while adopting a partially plan-driven, partially agile approach. This risks raising costs without raising benefits proportionally. If you cut out whole-team collaboration, feedback from the project community, risk minimization, and the idea of steering a project, you’ve cut yourself off from some of Agile’s best benefits.
References
The Startup Owner’s Manual, by Steve Blank and Bob Dorf.