It’s pretty usual to see people from Agile world keep on bashing the Waterfall methodology because of its inherent problems. To be honest, a big majority of the organizations still work in Waterfall mode. So sometimes I used to think, why can’t people in Waterfall world do improvement in the way they do things. Though organizations are switching to Agile, you can’t deny that people have delivered a lot of successful projects in Waterfall too.
In this blog, I am not into bashing mode but would like to ensure that people in Waterfall world benefit with some of Agile practices without changing the inherent nature of methodology.
Standup
From the outset, Standup is just 10 minutes ceremony but it has a huge impact. With answering 3 simple questions “What I did yesterday?”, “What I am going to do today?” and “Any impediments?”, it’s farely simple to bring transparency and problems on table. Only when you acknowledge and know the problem, you can solve it. That’s the basic principle, isn’t it?
In one of very big Waterfall project (200 people) I worked, only when people were in neck deep trouble, they used to sit together in order to identify issues. That exercise of jotting down issues itself used to take the entire day and even more. Resultant used to be a big excel sheet and to be honest I never knew what happened to that sheet after that.
I consider Continuous Integration as a philosophy and not just a XP practice. It simply says.
“Do things more frequently which cause you a lot of trouble if you do them only once in a while.”
In my example, instead of identify project issues once in 3 months, why don’t identify and solve them every day. You won’t have those intra-team or inter-team interactions issues anymore. That’s the beauty of this 10 minutes ceremony.
Apart from resolving impediments, team shares information on what they are doing and that does the self-cleaning within the team. It may sound very funny but it’s very difficult to talk about the same small task 3-4 days continuously. Waterfall hides a lot of non-performing people. Standup makes sure that the process itself put some cleansing within the team.
Regular retrospective
Irrespective of being in Waterfall or Agile, everybody has a tendency to improve oneself. Nobody can deny that. Waterfall is infamous in a way that inner-project problems which push you back in terms of improvement have absolutely no vent (forum) to come out.
Problems can be from any side. People may have very good solutions but they remain in their silos and don’t speak. That makes a person passive and continuous problems and no change pushes people to go into shell, in the sense of complete surrender.
Retrospective changes just that. What if team improvises itself on fortnight basis without needing any help from anybody. Definitely, at some points management has to step in to help the team. But having retrospective (talk about good points, points to improve, action points and action owner) put some FORMAL mechanism to improve things in the project.
Continuous Integration and Automated Build
In big projects, integration at the time of delivery is a mess. Though I am talking about long back (2008) but it used to take more than a day just to remove compilation problems and make build run in the project I mentioned. That’s a waste. Why not have automated build from day 1, have CI in place and people must write tests . Otherwise there is absolutely no guarantee why last build worked and why this one doesn’t.
Writing tests regularly to reduce the maintenance backlog
Waterfall is also infamous for maintenance nightmares. On the other hand for some companies it’s continued business too. Without bugs they won’t get the maintenance contract. The software which only they can maintain as only their developers know where all it will fail. It’s an irony but truth :P.
I am sure customers are becoming smart day by day and many companies offer free-maintenance for some months just to make sure that they are proud on their continued focus on quality and have delivered bug free software.
Irrespective of points mentioned above, it’s important to have tests both for customers and for maintainers. Application can never become extensible without tests as otherwise it will just become yet another monolithic structure which is waiting to be re-written again 😛 .
People have been talking about TDD or writing tests for a long time for Waterfall, I am not sure how much practically it has been possible for these guys to really do it as actual development phase is just one of the 4 phases (very less time) and also at that time of developers have pile of huge non-prioritized requirements to implement.
Jayadeep Purushothaman says
When your schedule for requirements, design, code etc. is in months, it doesn’t make sense to stand up every day and the methodology is to separate out things and each one work on it separately and integrate at a later stage – so there is no incentive to stand up.
Regular retrospective of what ? requirements gathering ? design ? coding ? these are not regular activities in a waterfall project and retrospectives at earlier phases are just speculative ones – how do you know if your design meets the requirements till it is coded ?
Integration and build is possible only as late as possible – there cannot be continuous integration when people are given an integration point in the plan. Nothing will be ready at earlier stages.
Testing is the responsibility of the test team – why should the developer write unit tests which will delay his delivery to the test team.
What I am trying to get at is it doesn’t make any sense to apply piece meal stuff in a waterfall project, the problems are fundamental in nature that cannot be fixed by band-aids. Again this thinking emanates from the SEI CMM framework where you use some of the KPIs in it. Agile is a totally different paradigm and these crutches would fail miserably IMO.
Same is true for applying the lean principles in your life – if you are a plan based guy who wants to lay out your life and work towards it, you are bound to miss the opportunities that comes out on the way because that wasn’t in your plan. Instead you need to have your principles laid out and grab the opportunities that are in line with your principles whenever it happens. Life and Projects cannot be planned on day-1!
Vin D'Amico says
Good points. Converting from waterfall to an agile approach like Scrum can be challenging. There are many differences and a lot of work to do. Adding a few agile techniques to waterfall is a good way to get a feel for agile practices without abandoning all the artifacts that waterfall teams know and like. Once they begin to feel comfortable with a few agile concepts, they are in a much better position to make a complete change.
Patrickb says
It’s absolutely the case that most firms that I’ve worked for prefer the waterfall approach when it comes to requirements. The main reason for this is budget, noone wants to hand over a blank cheque and the only to give estimates is to have a full set of requirements to hand. However in my experience the build phase usually does take on some sort of iterative nature. Even though components may not always be released to test, CI is normal and informal releases into a stable dev environment are the norm.
Zoya says
Well, these are great points… but in my case of high education industry (uniersity) not very useful and doable.
However, I am still trying to bring some “Agile into Waterfall” by using formats of the user stories when functional specifications are done from the high level business requirements.
So far people involved like that as it gives them a better perspective on functionality then the normal “system does…” points
Chakravarthy says
Hey,
Thanks for the required & meaninful stages of Agile Practices.
I have a suggestion / request on the same article. That is, if you can write about the tools that are used for implementing the scripts that are written on everyday. In other words, what kind of test tools that you recommend for the growing tests.
Finally, good work. Hope to see more from you
ShriKant Vashishtha says
@Chakravarthy Looking at your profile, it looks like you work in MS technologies. Though I work in other part of technology world, lemme Google for you similar tools in .Net.
* For Continuous Integration (CI), use TeamCity or JenKins (http://stackoverflow.com/questions/1427444/anything-better-than-cruisecontrol-for-net-ci)
* For build tool MSBuild, PowerShell (http://programmers.stackexchange.com/questions/33606/automated-build-platform-for-net-portfolio-best-choice)
* Test Coverage for MSBuild (http://megakemp.wordpress.com/2008/10/09/visual-studio-code-coverage-reports-with-msbuild/)
* Test framework (NUnit for unit tests and Selenium for .net for functional tests – http://seleniumtoolkit.codeplex.com/)
-ShriKant
http://twitter.com/vashishthask
ShriKant Vashishtha says
@Zoya I would be interested to know why you think it’s difficult to implement these ideas in High Education Industry. In your case, I believe it’s getting difficult to come out with real user-stories as people get used to think in terms of Waterfall requirements.
I completely understand that it’s difficult to really move towards a new paradigm called user-story without any preparation. For that I would recommend reading Mike Cohn’s book “User Story Applied” (http://www.amazon.com/User-Stories-Applied-Software-Development/dp/0321205685).
However as you are already in Waterfall, moving towards user-story will be far difficult for you compared to 4 practices I enumerated in blog. For these practices, you require absolutely no practice, mindset change or any paradigm shift.
Good luck.
ShriKant Vashishtha says
@Jaydeep Here is my reply:
When your schedule for requirements, design, code etc. is in months, it doesn’t make sense to stand up every day and the methodology is to separate out things and each one work on it separately and integrate at a later stage – so there is no incentive to stand up.
Standup is all about bringing focus, knowledge sharing and solving problems every day (CI for project problems). I have also worked in Waterfall projects which spanned for years. It’s humane to have lack of focus when deadline is 3-6 months away. At the same time I have seen people working day and night at the later part of those 3 months. Waterfall projects and those companies are infamous in making people working late hours. At Xebia I have never seen anybody working more than 8 hours just because everybody REALLY WORK 8 hours a day. At the end if you have complete focus on project every day which can be achieved with Standup, nobody needs to work late hours.
Regular retrospective of what ? requirements gathering ? design ? coding ? these are not regular activities in a waterfall project and retrospectives at earlier phases are just speculative ones – how do you know if your design meets the requirements till it is coded ?
Retrospective can be of anything. Even we do project retrospective. You don’t need to prepare for it or really need to have a phase for it. Team just need to answer “Good points” so that they celebrate their achievements of work they did, “Points to improve” (things they are bothering them) and then have action points and action owners. Just do it once and I am sure you will definitely find its value based on the principle “Inspect and Adapt”.
Integration and build is possible only as late as possible – there cannot be continuous integration when people are given an integration point in the plan. Nothing will be ready at earlier stages.
We commit code every hour or half an hour and with each commit we run build. We do it from day 1 and not after 3 months. I am not sure if you already have automated build in place and build is single click and takes a min to complete, why do you need to integrate system 3 months later when potentially you face a lot of issues and time to fix them. Simple plan is, write classes and tests, update source from SVN, run automated build script to make build successful and commit. That’s all I suggest. CI will tell if you still committed any mistake which people in real dev environment do a lot.
Testing is the responsibility of the test team – why should the developer write unit tests which will delay his delivery to the test team.
Functional testing is the responsibility of test team and not unit tests. Without unit tests people do JDD (Jesus driven development) where they are not sure if the change they are going to make will break the existing functionality or not.
What I am trying to get at is it doesn’t make any sense to apply piece meal stuff in a waterfall project, the problems are fundamental in nature that cannot be fixed by band-aids. Again this thinking emanates from the SEI CMM framework where you use some of the KPIs in it. Agile is a totally different paradigm and these crutches would fail miserably IMO.
Common sense cannot be replaced by any framework IMHO. Things that work and don’t work can be seen only by experiment. So I would suggest you to try at least once.
PM Hut says
Hi ShriKant,
The concept of a daily standup will be simply overhead in the Watefall world and it can be distracting and annoying to both the team members and the project manager.
In any case, I think that PM Hut readers will appreciate your above post and that’s why I would like to republish it on PM Hut. Please either email me or contact me through the “Contact Us” form on the PM Hut website in case you’re OK with this.
By the way, you might find this post, Agile to Waterfall Dictionary interesting. Hope you’ll get the chance to read it.
ShriKant Vashishtha says
@PMHut I am surprised to even think that how come a 10 min standup is overhead in a day when it brings so much value. In my 10 years of experience with Waterfall, I have seen people having absolutely no idea what they are going to do in a day. I have seen too many occasions when somebody is doing JUST nothing as in a crowd of 50 people it’s difficult to even know what that guy is doing. I can still show you many people getting into stock market as intra day traders as they have lots of time without work. Customers absolutely have no visibility sitting in distant shore in the work done on daily basis.
So I am really surprised. I have experimented Standup in Waterfall project (200 people offshore project). That’s the reason why I wrote “Agility meets the Waterfall” (http://www.javaworld.com/javaworld/jw-03-2008/jw-03-agile-practice.html) way back in 2008. At that time I was a beginner in Agile. Now based on my experience with Agile all these years I feel I can return something back to the community.
Ajay Danait says
Common sense refers to beliefs or propositions that seem to many people to be prudent and sound in judgment without dependence upon esoteric knowledge.
The context of being agile in software is a different context than being agile in other industries. Just common sense is not enough.
Common sense is not natural to humans and yes, very contextual.
Agile practices can be taught and cannot just be learned by categorizing them as common sense, and how people grasp them makes the same people realize the common sense that exists in them.
If you tell someone about daily standup ceremony, XP engg practices .. for a person with Waterfall context .. I would say “It doesn’t make any sense”.
Only when the practices are really practiced, experimented and then innovation kicks in, they start making sense.
Again all contextual to the environment and not just cult following …
ShriKant Vashishtha says
I think the focus here is more about what common sense is all about. And why I say Agile practices are common sense.
For me, I see things in two ways, common sense is something which is based on rationality and logic. If you explain it to any individual he/she should be able to make some sense out of it.
Rocket science based thinking or application are not and cannot be interpreted or invented in straight forward way. Based on my internet research I found few good resources on common sense. Take a look on the following before moving ahead:
* http://perspicuity.net/common/comsen-2.html
* http://en.wikipedia.org/wiki/Common_sense
Now, instead of explaining what common sense these Agile practices contain, I would rather ask a reverse questions and that is – which Agile practice look like rocket science and which one will take some time or applicability to explain people?
Sometimes we say that we should not implement partial Agile if we want to see expected results from the implementation.
Why?
It’s just because there are very good reasons for each and every ceremony. Unless team has even better reasons not to do it, all these practices should be implemented.
I just know the fundamental principle of Agile philosophy, i.e. “Inspect and Adapt”. You try with certain practices, see if they work for you, if not you identify the reasons and again make certain changes. Just because of this reasons we have been able to evolve so many Agile practices which made sense to the context such as Distributed Agile. Now I call these practices Agile practices but I can also call them based on just pure common sense. The idea is, based on the problems we faced we identified root cause, identified options to solve it and then took a step forward. If it worked fine, we keep on doing it. Otherwise we inspect and again make changes until we are able to resolve problems. As I mentioned, just because of these reasons, here are some evolved Agile practices.
* Distributed pair programming
* Local retrospective
* Nut Bolt pattern
* One team multiple projects
* Collocation and its best practices
* ATDD in context of distributed Agile
* Distributed Agile: Virtual One room
* Chain-link pattern
* The need of Agile architect
* Incremental functional test approach
* Knowledge sharing between distributed teams
* Agile way of documentation
All these patterns or practices are available at http://sampreshan.svashishtha.com/patterns/
Every pattern or practice has logical reasons and straightforward thinking behind them. I don’t put Agile in certain boundaries. Our (software craftsmen’s) motto is to create business solutions with our technical boundaries. How best we can do that cannot be or should not be confined in something which is written in books or made as framework. Based on the situations, new scenario, context and changing time, you need to keep on innovate new ways of doing effective software development.
Now if these certain practices also make sense to Waterfall world, what’s the issue? Why can’t we experiment with them and see how they work. In my blog, I have specifically mentioned reasons why I see their applicability in Waterfall world. At the same time, I have tried implementing them in real projects and have been successful. In case we want to debate over them, lets try to come out with valid scenarios in which any of the practices mentioned will not work. I sincerely welcome those suggestions, comments because my purpose is not to devise something just for the heck of it or for so called branding. It’s important that we should be able to ease-out the way current software implementation or execution works in Waterfall world. Let’s evolve the discussion based on certain facts and experiences which will be much more helpful in my humble opinion.
Bala says
Thanks for the post. good One…We can implement most of the practices in an (incremental) iterative model.
Nitin Reddy Katkam says
In my years of experience, I have seen many firms that refuse to move away from the waterfall model of software development because to them it feels right and seems to be more efficient. There are other firms that believe they are performing agile development but, in reality, it is simply a modified waterfall model often with cycles of bug-fixes, feature-requests, and feedback at the end.
A complete process change is perhaps too drastic a change to bring about within most organizations, and I do believe that introducing some agile practices to an otherwise waterfall-following culture is the way to bring about the transition.