Friday 23 October 2009

The Perfect Software Process - And It Really Works!

Sarah A. Sheard has written a really insightful essay on software processes, or silver bullets. Having read several blog posts on Agile methods it looks like we are fast approaching Phase 10, Blaming the Method, especially regarding Scrum (well, okay, some of those are already quite old and Scrum is still alive 'n kicking). So, it's Kanban and Lean methods next? How many years will they survive? The sorry part is that while methods like XP now deemed a failure by many evolve, people who tried them and failed (usually because of the reasons Sheard describes) won't look back.

Sheard's article explains in a indirect manner, what the perfect Software Process is. I would summarize it like this:
  1. Pick a process. Any will do, but the closer it is to your needs, the better.
  2. Use the process until you understand why things are done the way they are (shouldn't take too long for people familiar with processes).
  3. Start a continuous improvement process where you gradually improve the process to best meet your needs. This should be embedded to the software process so that this is done all the time and by the people who really are involved.
And that's it! Eventually, if you do this right you will have a perfect process (or at least until things change - but at least you will close to the ideal most of the time). Of course if you make a poor choice in the first phase it will take a long time to get the right one.

What Scrum seems to want you to do is to keep short iterations and keep this improvement cycle going around those iterations (Scrum reviews). So if it is done correctly, constant improvement is built into the process, which can't be said of every process. CMMI for instance includes this continuous improvement only at the higher levels.

Of course this cycle could lead to a Scrum team eventually not using Scrum if it doesn't fit their way of working. I don't know if Scrum ideology endorses this idea, but I'm confident in saying that if some organization manages to do this they are definitely working well.

In any case, any organization trying to endorse a new software process (or any other process, for that matter) that fail to implement phases 2 and 3 are almost certainly going to be disappointed with their choice. I'm constantly amazed at how so many people fail to see this obvious thing. No two projects are the same so how do you think a process developed by people you don't know in environments you are probably unfamiliar with is going to be a perfect fit in you projects?

I find Agile methods appealing for one reason: they not only accept change, but endorse it. I'm yet to see a software project that didn't have a backlog of change requests (bug reports are just a subset of these). Most processes I know handle change requests by having a change request process that includes a CCB (Change Control Board), requires extensive impact analysis, lots of communication, and so on just to be able to decide whether to go ahead with the change or not and how to go ahead with it. That's an awful lot of extra work. And why? Because no matter what the process developers say, they do not endorse change. Change is inevitable, but also a negative thing. So handling it is unnecessary complicated because even though change is the norm, these processes handle change as an exception. "We have this neat process that is constantly being interrupted by these change requests."

Contrast with agile methods which take changes granted. They are not exceptions but the rule around which the process is largely developed.

Having said that agile methods do not work for every project. You can't use Scrum as-is developing a cellular base station because you have thousands of pages of specification you must follow. You have to understand them and carefully plan your architecture in such a way it doesn't stop you from creating a base station that works according to the specs. So you need a rigorous specification and design phase which means that be default changes will be exceptions. You could still use some parts of agile methods but trying to use, say, Scrum without any modifications would be just stupid.

Having said this I realised one major problem with this approach: how do you handle this in a company with multiple projects? If every project modifies the process to fit them best you will end up with different processes for each project. This increases cost of training when people switch projects and also assumes that people fine-tuning the processes are doing a good job.

Oh, one more thing: I find Alistair Cockburn's Ph.D. dissertation on the web. I have to find time to read it because it's exactly about these issues.

No comments:

Post a Comment