Sheard's article explains in a indirect manner, what the perfect Software Process is. I would summarize it like this:
- Pick a process. Any will do, but the closer it is to your needs, the better.
- Use the process until you understand why things are done the way they are (shouldn't take too long for people familiar with processes).
- Start a continuous improvement process where you gradually improve the process to best meet your needs. This should be embedded to the software process so that this is done all the time and by the people who really are involved.
What Scrum seems to want you to do is to keep short iterations and keep this improvement cycle going around those iterations (Scrum reviews). So if it is done correctly, constant improvement is built into the process, which can't be said of every process. CMMI for instance includes this continuous improvement only at the higher levels.
Of course this cycle could lead to a Scrum team eventually not using Scrum if it doesn't fit their way of working. I don't know if Scrum ideology endorses this idea, but I'm confident in saying that if some organization manages to do this they are definitely working well.
In any case, any organization trying to endorse a new software process (or any other process, for that matter) that fail to implement phases 2 and 3 are almost certainly going to be disappointed with their choice. I'm constantly amazed at how so many people fail to see this obvious thing. No two projects are the same so how do you think a process developed by people you don't know in environments you are probably unfamiliar with is going to be a perfect fit in you projects?
I find Agile methods appealing for one reason: they not only accept change, but endorse it. I'm yet to see a software project that didn't have a backlog of change requests (bug reports are just a subset of these). Most processes I know handle change requests by having a change request process that includes a CCB (Change Control Board), requires extensive impact analysis, lots of communication, and so on just to be able to decide whether to go ahead with the change or not and how to go ahead with it. That's an awful lot of extra work. And why? Because no matter what the process developers say, they do not endorse change. Change is inevitable, but also a negative thing. So handling it is unnecessary complicated because even though change is the norm, these processes handle change as an exception. "We have this neat process that is constantly being interrupted by these change requests."
Contrast with agile methods which take changes granted. They are not exceptions but the rule around which the process is largely developed.
Having said that agile methods do not work for every project. You can't use Scrum as-is developing a cellular base station because you have thousands of pages of specification you must follow. You have to understand them and carefully plan your architecture in such a way it doesn't stop you from creating a base station that works according to the specs. So you need a rigorous specification and design phase which means that be default changes will be exceptions. You could still use some parts of agile methods but trying to use, say, Scrum without any modifications would be just stupid.
Having said this I realised one major problem with this approach: how do you handle this in a company with multiple projects? If every project modifies the process to fit them best you will end up with different processes for each project. This increases cost of training when people switch projects and also assumes that people fine-tuning the processes are doing a good job.
Oh, one more thing: I find Alistair Cockburn's Ph.D. dissertation on the web. I have to find time to read it because it's exactly about these issues.