At the start I work out with you how best your change can be delivered:
Can you begin and get value from any little parts of the change individually?
versus
What efficiencies can be had by researching and planning the whole change up-front in more detail?
Neither approach is necessarily and always best, nor are they completely exclusive, rather it depends on the circumstances.
Agree what the problem is
Work out where you are now (what happens, who does it, how, where, when, how often, how well) and measure the performance
Decide if performance is now good enough (in which case STOP) or agree a goal (or at least a direction) for that measured performance
Agree a small step towards your goal that will give value
Execute that step - just that little bit of detailed requirements & design, construction & testing, deployment & snagging
REPEAT from step 1
Agree what the problem is
Work out where you are now (what happens, who does it, how, where, when, how often, how well) and measure the performance
Agree a goal for that measured performance
Plan out all the work that needs to be done in a sequence of stages; requirements & design, construction & testing, deployment & snagging
Execute the plan, confirming quality and agreement at each stage before moving on
Any approach can get into trouble through misunderstanding and weak decisions, which I will make sure you avoid. Of the two, while often sold on a promise of easier engagement, Agile is actually much harder to practise.
'Agile' is often suggested as a panacea, modern, a default choice. Sometimes it is embraced by leaders because they think it will require less from them, specifically:
they won't need to sign specific contractual documents (like designs)
they won't need to be involved so much
they'll be at liberty to change things later, without penalty
In fact, while Agile advocates doing only the work necessary for a little piece of the goal at a time, clients and leaders have to spend more time on the project almost every day:
specifying pieces of change for the developers and giving feedback
ensuring each little step of progress or design is widely shared
to make up for the lack of more complete specifications up-front.
'Waterfall' appears simple and appealing but requires good judgement of whether there's sufficient stability. Other common pitfalls include:
Over-confidence: Planning and estimating too much at the start, when least is known, sets dangerous false expectations.
Analysis paralysis: A good choice in good time is better than finding a better one too late.
Over-adherence: Determination to deliver the "agreed" plan is not more important than facing and delivering what is now needed.
Whale: Insisting on one single enormous waterfall delivery instead of breaking it up into a few manageable iterations is one of the highest causes of project failure.
Zombie: Even when the circumstances clearly require it, fear of the career impact of changing or stopping a project leads to further waste.
For Agile to work, the answers need to be a resounding "Yes!"
1. Valuable parts: Are any little bits useful alone?
Can parts of the solution be imagined that could be used on their own and still give some extra value?
Will the value gained outweigh any additional costs (e.g. extra work-around labour) involved in running only the partial solution?
Will the value gained outweigh any costs of delay to the ultimate better solution, or disposal and waste of the partial one?
If not, if in fact
only the whole, finished solution is useful and valuable
or the benefits of any half-way house are outweighed by the additional costs of tearing it down and rebuilding properly later
then Agile will cost more that Waterfall, with higher risk of failure.
2. Capacity & will: Can the end-users deal with it in little bits?
Can the receiving organisation cope with the changes in little pieces, and do they want to? One reason they might not be able to is if more line work would be required. For example, an early, partial solution might require, for a while, more than one system to be maintained in parallel and information to be joined across them. This might not be feasible, or the budget for the additional staff necessary might not be available.
Staff who are subject matter experts will be needed, often close to full-time, throughout the change. Do suitable expert people exist, and can they be freed up, to become part of the change team to define each step, help specify it in detail, inspect what is delivered and test it?
Staff leaders with authority will be needed, often up to half of their time, throughout the change. Do suitable people exist and can they be freed up, to prioritise and approve work, judge progress, make compromises, and decide when to change the goals or stop the project?
If not, if in fact
the staff cannot absorb the impact of any partial solutions
or critical people cannot or won't be part of the change team
then the stages of Waterfall will expose issues earlier.
3. Uncertainty: Is trial and error really necessary?
Is there significant uncertainty about what is wanted? Could this not be solved by some hard work from the end-users, before even engaging engineers?
Is there little vision of what is wanted and how it must work, and is it proving hard to achieve this vision on paper?
Is there serious disagreement about what is wanted or how it must work, and difficulty breaking this deadlock?
If not, if in fact
a lot of requirements are known, certain and pretty stable
and better planning decisions could then be made from their analysis
then it makes sense to do at least that known work up-front.
Where there's uncertainty but the client has the time and wants to engage fully, and if the team is managed strongly, an Agile approach can:
Deal with change: Should the goals change as understanding grows, or funding be cut short, then we'll regret any effort already spent beginning to design and build items that are no longer wanted or can't be afforded. By doing the least possible design and underlying construction necessary to support just the value needed in a small iteration at a time, Agile can maximise the chance of realising at least some value.
Realise value earlier: Agile can help deliver earlier some features that are truly useful, so some benefits can start to be enjoyed. Beyond the actual value generated by earlier delivery, the additional positive impact to morale and wider continued support for the project due to these successes should not be underestimated.
Break paralysis: Where there is uncertainty or disagreement, delivering actual working elements whose performance impact can be measured really helps demonstrate a solution in ways that sometimes an on-paper design or description would struggle.
When requirements and the target environment (technology, processes, people) are clear and stable, a Waterfall or iterative approach can:
Be more predictable: With the whole scope agreed and planned up-front, the timeline, scope and budget can be much more confidently and easily predicted early on. (Any changes later have to be carefully assessed for impact on the work already done, so a major change and impact will break those expectations, which is why this is only a good approach if requirements are clear and stable.)
Expose road-blocks clearly: By requiring the team to complete each stage and gain formal agreement that specification and quality is right before moving on, the people receiving the end product are motivated to work diligently to check the proposals and understand the long term implications, before giving approval for the next stage. Refusal to sign off brings issues to light very clearly.
Start easily: The simple, sequential steps make it easy to explain to a team and get everyone quickly clear what they should be doing.
For example, if in the first year of a project all the effort is spent building a large brick house for the whole family, but only the foundations are complete before winter so most of them don't survive the cold or lack of food, the work will be abandoned, leaving nothing but costly concrete foundations as its legacy.
A better approach might have been to
first build a house of sticks and move the family in
then add a fireplace
finally replace the stick walls with brick
Although at the end more money/effort would have been spent (as the stick walls will have been thrown away), this approach copes much better with changing circumstances or goals, for example
if money runs out before the end, or
if the family discover that a house of sticks is warm enough, and decide growing crops is now a higher priority than a warmer house.