The Perfect Software Process - And It Really Works!

Sarah A. Sheard has written a really insightful essay on software processes, or silver bullets. Having read several blog posts on Agile methods it looks like we are fast approaching Phase 10, Blaming the Method, especially regarding Scrum (well, okay, some of those are already quite old and Scrum is still alive 'n kicking). So, it's Kanban and Lean methods next? How many years will they survive? The sorry part is that while methods like XP now deemed a failure by many evolve, people who tried them and failed (usually because of the reasons Sheard describes) won't look back.

Sheard's article explains in a indirect manner, what the perfect Software Process is. I would summarize it like this:

1. Pick a process. Any will do, but the closer it is to your needs, the better.
2. Use the process until you understand why things are done the way they are (shouldn't take too long for people familiar with processes).
3. Start a continuous improvement process where you gradually improve the process to best meet your needs. This should be embedded to the software process so that this is done all the time and by the people who really are involved.
   

And that's it! Eventually, if you do this right you will have a perfect process (or at least until things change - but at least you will close to the ideal most of the time). Of course if you make a poor choice in the first phase it will take a long time to get the right one.

What Scrum seems to want you to do is to keep short iterations and keep this improvement cycle going around those iterations (Scrum reviews). So if it is done correctly, constant improvement is built into the process, which can't be said of every process. CMMI for instance includes this continuous improvement only at the higher levels.

Of course this cycle could lead to a Scrum team eventually not using Scrum if it doesn't fit their way of working. I don't know if Scrum ideology endorses this idea, but I'm confident in saying that if some organization manages to do this they are definitely working well.

In any case, any organization trying to endorse a new software process (or any other process, for that matter) that fail to implement phases 2 and 3 are almost certainly going to be disappointed with their choice. I'm constantly amazed at how so many people fail to see this obvious thing. No two projects are the same so how do you think a process developed by people you don't know in environments you are probably unfamiliar with is going to be a perfect fit in you projects?

I find Agile methods appealing for one reason: they not only accept change, but endorse it. I'm yet to see a software project that didn't have a backlog of change requests (bug reports are just a subset of these). Most processes I know handle change requests by having a change request process that includes a CCB (Change Control Board), requires extensive impact analysis, lots of communication, and so on just to be able to decide whether to go ahead with the change or not and how to go ahead with it. That's an awful lot of extra work. And why? Because no matter what the process developers say, they do not endorse change. Change is inevitable, but also a negative thing. So handling it is unnecessary complicated because even though change is the norm, these processes handle change as an exception. "We have this neat process that is constantly being interrupted by these change requests."

Contrast with agile methods which take changes granted. They are not exceptions but the rule around which the process is largely developed.

Having said that agile methods do not work for every project. You can't use Scrum as-is developing a cellular base station because you have thousands of pages of specification you must follow. You have to understand them and carefully plan your architecture in such a way it doesn't stop you from creating a base station that works according to the specs. So you need a rigorous specification and design phase which means that be default changes will be exceptions. You could still use some parts of agile methods but trying to use, say, Scrum without any modifications would be just stupid.

Having said this I realised one major problem with this approach: how do you handle this in a company with multiple projects? If every project modifies the process to fit them best you will end up with different processes for each project. This increases cost of training when people switch projects and also assumes that people fine-tuning the processes are doing a good job.

Oh, one more thing: I find Alistair Cockburn's Ph.D. dissertation on the web. I have to find time to read it because it's exactly about these issues.

Mounting a Windows network drive to Linux

Easy enough after I figured the first part, that Ubuntu did not come with the necessary package by default. So this is how I did it:

• sudo aptitude install smbfs
• sudo mkdir /mnt/mntpoint
  
• sudo mount -t smbfs //server/directory /mnt/mntpoint -o credentials=/home/my/.smbpasswd

The .smbpasswd file looks like

username=<AD username>
password=<AD password>

I did a chmod 600 on that file so no-one else can see my AD password.

To automount this on boot, I added the following line to /etc/fstab:

//server/directory /mnt/mntpoint smbfs
credentials=/home/my/.smbpasswd,uid=1000,gid=1000 0 0

Where uid and gid are my uid and gid on the box.

Scrum resources

In my quest to get my B.Sc. thesis done I will list some Scrum and Continuous Integration resources in this post. I will add to this if and when I find better ones.

• Kniberg, Henrik: Scrum and XP from the Trenches
• The Scrum Guide.
• The Scrum Software Development Process for Small Teams.
• Sutherland, Jeff: Agile Development: Lessons learned from the first Scrum.
• Fowler, Martin: Continuous Integration.
• Richardson, Jared: Continuous Integration: The Cornerstone of a Great Shop.
• Lots of resources: http://www.gaminghorror.net/scrum-resources/
• Steve Yegge: Good Agile, Bad Agile. This is an intereseting article and especially interesting are the comments from other people. Steve is understandably quite fond of the way Google work, the comments show that it's a very exceptional place to work in and how there are very few places where it can be implemented (although there is also a comment from someone else working at Google who paints a different picture).
  
• Cockburn, Alistair: People and Methodologies in Software Development. Cockburn's Ph.D. dissertation.
  

Parsing files with Bourne shell

My useful piece of Bourne shell script for the day is a piece of script that goes through .change files in the current directory and copies all the files listed to a different directory. It skips the _sources.change files as well.



The obvious use is to copy the files so we can use "reprepro processincoming" to save packages to a repository.

There are probably smarter ways to do this, but this is pretty portable as it only requires grep and cut, and very minimal functionality from them as well. And even grep is not needed if you want to copy everything.

Changing user names in Gitorious

It seems there is no mechanism for changing a user name in Gitorious. I was using the wrong one in one server so I decided to change it straight from the database. After that I kept getting "You don't have write access to this repository" but no matter how hard I looked everything still looked fine (git configuration, ssh key's fingerprint etc). Looking at pre-receive-guard.log, I noticed that for some reason I was trying to access Gitorious with the wrong user name which happened to be the old one. I even deleted and re-added the ssh key just in case.

A bit of digging revealed that the problem was simple: the old ssh key was still in ~git/.ssh/authorized_keys and being the first one it was used instead of the one connected to the new user name. Deleting this key from the file was all that was required.

Adding More Disk Space to a Virtual Machine

This was something I didn't know and as the main function of this blog is to save this sort of stuff for myself for later reference, here's how it can be done when using kvm virtualisation.

There are at least two ways:

1. Start off by creating a an extended partition of any free space you have (with fdisk) OR create a a file to act as the disk with something like "dd if=/dev/zero of=/path/to/file bs=1024 count=$((10*1024*1024))" (this would create a 10 gigabyte file).
2. Open virt-manager and go to the virtual machine's Details page. Click the "Add Hardware" button.
3. Select "Storage" as the hardware type.
4. If you created the extended partition, enter the device name in the block device text box, or if you created the file, select the file in the disk image box. In both cases, use "IDE disk" as the target.
   
5. Start up the virtual machine and create the partition(s) using fdisk.
   

Redmine installation issues

There are some more straightforward ways to install Redmine than what I have discussed earlier, as this wiki article describes. But my method also seems to work - I have now deployed Redmine in production.

Also, it seems installing plugins to Redmine is really easy if those plugins are in GitHub. Couldn't get any easier, all you do is run script/plugin install git://github.com/plugin_dir/plugin.git and hey presto!

Getting LDAP to work with Redmine was really easy as well. The Redmine guys have written short instructions that tell almost all you need. And that's all you need, LDAP support is written directly into Redmine meaning you don't even have to configure your webserver.

Linking Gitorious repositories to Redmine is a bit annoying thanks to the fact that Gitorious hashes the repository names. The solution I came up with is a crontab task that creates links in the form of project/repository.git (i.e. in plain text) that link to the Gitorious repositories. I can then link from these links to the repositories in Redmine, meaning I don't have to dig up the hashed directories from Gitorious by hand.