Why Not To Git

The header of my previous post was a lame joke that nobody else got, I admit...

Anyways, there are also reasons why git is not the best choice and these come down to one (or two, depending on how you look at it) reason: complexity. Distributed version control systems are a different scheme to start with and git doesn't make an attempt to make it any easier to understand.

There's also the problem of security in the more common way, rather than way where git excels. I'll discuss this point in this post and maybe delve into usability and ease of learning leater.

In the modern time companies are out-sourcing development to off-shore companies and in these cases it's usually desirable that not everyone gets access to everything. There are no in-built mechanisms to control access in git, by default everyone can access everything in a repository. The idea as far as I can tell is to create small repositories so that we can give access to entire repositories to people. This is quite awkward in practice. And since git doesn't deal with single files but commits which can include several files at a time fixing this is not simply a case of forbidding access to some files for some people. Imagine if we have two files and two developers. One of them has access to both, the other only to one file. Developer one changes both files and commits. What happens when developer two tries to pull? We would have an incomplete repository (very much against the whole idea of distributed vcs's) and furthermore the SHA calculation would fail (?).

I can only think of one way this can be solved in git's model. That is to use encryption on files where the user doesn't have access. But this would also require that the entire access set is delivered with the repository. Possibly this would also require the inclusion of public keys into the repository and it would definitely need access to the server where the company's key is. Crypting and decrypting these files would make repository cloning a lot slower especially if there are lots of files with access sets (which often is the case).

So it might well be that this is something where git will never be good at forcing companies to either use other vcs's or building complicated work-arounds. But it is very common practice that access is restricted. For instance in a large system you might not want to give every design document to everyone, only what is needed for them to do their work. But if every part is in a different repository, building the system would be difficult and people who needed access to everything (like architects) would find this awkward. The practical solution would be built a middle-layer that automatically updates some repository which holds all this info and where only select people have access, but this would require a lot more work to build and maintain.

I believe this is not something Linus and the guys had in mind since withholding information goes against FOSS, but in reality not every company can afford to give out everything. Otherwise some Chinese company will push them out of business with their version.

Why git

There are compelling reasons why git should be choice of the next generation (of developers, and why not us old farts as well). Rather than listing them here, here's some links:

• Why Git is better than X
• Git vs Subversion comparison
• 3 Reasons to Switch to Git from Subversion
• Git vs Mercurial
• Git vs Mercurial
• Git vs Mercurial ("CVS is dead and Subversion is not good enough")

And so on. There are clearly some problems with git:

1. GUI support is not great. There are some tools like git extensions and qgit but apparently they are still work in progress.
2. Git does not work as well in Windows as in Linux, a lot of this has to do with NTFS/ext filesystems and the fact git was designed for Linux. This means you need to install cygwin to even run git on Windows.
3. Plugin support is still lacking.
4. Documentation, while quickly improving, is still behind some of the competition.

I'm sure these will improve rapidly because git is gaining popularity fast. Hopefully the development community realises that git really needs to work well on other platforms as well. There are many cross-platform projects and very few companies run on Linux alone so this could be the sticking point for many to move to git from what ever vcs they are currently using.

In any case, git is still a huge improvement over such vcs's like cvs and svn and free compared to such extremely expensive ones like ClearCase and Dimensions.

I have used ClearCase, PVCS/Dimensions, Continuus, StarTeam, SourceSafe and some others extensively and I have come to the conclusion these do not offer enough bang for the buck. Some pros and cons I have noticed:

ClearCase:

• Pros: Merge works well.
• Cons: Complicated, requires people just to support the repository, expensive, slow.

PVCS/Dimensions:

• Pros: Reasonable simple for developers.
  
• Cons: Expensive, very slow, poor user interface, some things do not comform to any normal conventions.
  

Continuus:

• Pros: Change tracking works well (if you use it correctly).
• Cons: Very complicated to use (requires extensive training), expensive.

StarTeam:

• Pros: Easy to use, quick, intuitive.
  
• Cons: Even though basic version is a lot cheaper than the alternatives, full version costs about the same.

SourceSafe:

• Complete piece of crap. Stay clear at all costs.

This list was just some things I could think of straight away. There are lots of other issues, for instance cross-platform usage that I didn't consider. Someone should look at StarTeam's UI and write a similar one for git and it would be a winner.

The whole point of a vcs should be that it's quick and doesn't hinder development. The developers should be able to do what they need quickly, easily, hassle-free and without making any mistakes (like committing to wrong branches etc). I would say the first two points are the most important ones. If the version control is quick and easy to use, developers use it more (they merge more often, they check-in their stuff more often etc) which is basically what you want them to do. A good vcs is more than just a safe place to store your code. It's a tool used to collaborate with others and the easier this is to do, the more your developers collaborate (share code). Collaboration is the key for git and something where it stands heads and shoulders above all these expensive commercial products (and cvs, svn etc). Distributed vcs has such possibilities in improving productivity that very few techniques at the moment can boast anything similar.

Mach and mock

Mach and mock are used to build rpm based packages in chroot's. Nice, if, for instance, you need to do that in Ubuntu.

Today's lesson is: use mock if you can. Mach doesn't really seem to work and is a hell to configure, mock is a re-write and I actually managed to get it to work after I updated sqlite. This is how you create an rpm to the environment you desire:

1. Install mock if you don't have it already.
2. Add root to the mock group.
   
3. Run mock -r rebuild where is one of the files in /etc/mock (like fedore-devel-i386) and package is the srpm you want rebuilt.
   

And that's it. It should be possible to run this as non-root, but so far I have only gotten "permission denied"s. Adding myself to the group didn't help.

The downside is mock only supports epel and fedora, not Red Hat, centOS, SuSE, Yellow Dog etc. like mach. Hopefully either mock people add support or at least it's easy to add it yourself. (should be if adding a config file is adequate, they seem simple enough). At least it looks like adding Red Hat to the list is simple enough.

There are still a number of issues that should be checked:

1. How easy would it be to use LVM snapshots with these chroots?
2. If I have a deb package and run alien to convert it to rpm and then rebuild it to a different target, is it really a functional package (i.e. dependencies are correct)? The other alternative is to build the rpm directly from source which requires a spec file (alien doesn't).
   

All in all, another day which showed that there are still lots of things in Linux that are poorly implemented, complicated to use and lack proper documentation. Basic stuff is there, but if you run into problems and google doesn't help you are out of luck. The bug in Ubuntu certainly didn't help either.

Things you learn..

It's only 10.30 and I have already had to learn a few new things, so all in all, not a bad day at the office. Except some of this stuff is a bit annoying.

Let's start with Gnome. Which managed to completely lock up. Quick googling on an other computer reveals that Gnome is really dead, because ctrl+alt+backspace did nothing. But I was able to connect to the Linux machine from the other comp, so more googling and

sudo /etc/init.d/gdm restart

Did the trick, Gnome restarted. Of course everything I had open in desktop closed but at least I didn't have to reboot the machine.

I did remember I can somehow open other tty's but couldn't remember the shortcut before after that. Then I remembered: ctrl+alt+f[1-9] opens it, so I could have fixed the problem locally as well. Oh well, I'll remember the next time.

What else?

Oh yes, Thunderbird doesnt' support OpenGPG keys, you have to install the Enigmail plugin for that. After that it's a snip, just run a wizard and select whether you want to use an existing key or create a new one. I had to switch over to Thunderbird from Evolution, because Evolution just doesn't work. Twice now it has decided there is no connection to the mail server when there clearly is. Restarting doesn't help, last time I just kept trying all sorts and finally it started working again. I don't know what I did and didn't want to figure it out either so I switched to Thunderbird which I use at home as well and has worked well enough for years.

I need digital signatures so I can report the bug mentioned earlier to Launchpad. But for some reason it's not accepting my key fingerprint, at least not so far. I tried to to add my key to Ubuntu's key server with gpg --keyserver keyserver.ubuntu.com --send-keys and Ubuntu's key server says it's there. Another annoying thing: that command requires a parameter, the key id. Now, the command is "send-keys", i.e. in plural which suggests it sends all the keys. Nope, if you omit the key id, gpg says nothing. Very helpful, because you don't know that you needed to supply it and don't get any hint that nothing was actually done. This begs the question: what sort of propeller head writes these programs?

Ah, now it worked. There was a delay before Launchpad realised I had sent the key.

EDIT: Yipee! I managed to report this bug after a few tries (like, you need a space before the "affects" line in the email -> not good). Direct link to bug.

Bug in Ubuntu jaunty

There seems to be a problem in the jaunty jackalope release. Namely the sqlite3 version you can get with the package manager has a serious bug that prevents some programs from running. I found this out when I tried to run mach.

Apparently that version (and a few older ones) don't allow the creation of a column named release without quoting it. And guess what mach tries (or, rather yum, but I don't know which of these names the column). The bug is described here.

Ubuntu's version is 3.6.10, installing the latest one (3.6.14-1) seems to have fixed the problem.

Installing the latest version of sqlite is done like this:

1. Download the amalgamation package from the download page.
2. Extract the package and cd to that directory.
3. Run ./configure --prefix=/usr && make && make install.

Creating rpm's in Ubuntu

Converting a deb package to an rpm package is simple, just use alien (alien also supports other package types, tgz, pkg, slp and lsb). But what if you need to start from scratch? Here's how to create an rpm package from scratch in Ubuntu:

1. Install rpm ("sudo aptitude install rpm").
2. Create the spec file (e.g. pkg-1.0.spec).
3. Create the tarball for your sources and place it in /usr/src/rpm/SOURCES.
4. Run rpmbuild -ba pkg-1.0.spec.

This will create two packages: /usr/src/rpm/SRPMS/pkg-1.0-1.src.rpm and /usr/src/rpm/RPMS/i386/pkg-1.0-1.i386.rpm (if your target was i386).

The spec file looks like:

Summary: My SW
Name: MySw
Version: 1.0
Release: 1
Source0: %{name}-%{version}.tar.gz
License: GPL
Group: Development/Tools
%description
This is my SW.
%prep
%setup -q
%build
./configure
make
%install
make install
%files
%defattr(-,root,root)
/usr/local/bin/mysw
%doc /usr/local/info/mysw.info
%doc %attr(0444,root,root) /usr/local/man/man1/mysw.1
%doc COPYING AUTHORS README NEWS

This file should be self-explanatory. In this example the file in /usr/src/rpm/SOURCES should be called MySw-1.0.tar.gz. The spec file can be named anything.

Perl formats

Reasons to love Perl, part whatever. The following script takes in input that looks like

ID  WorkId  Date
1   2       12-10-2009
2   16      01-01-2008

(Fields are tab formatted)

and prints out:

+-----+----------+---------------+
|ID   |WorkId    |Date           |
+-----+----------+---------------+
|1    |2         |12-10-2009     |
+-----+----------+---------------+
|2    |16        |01-01-2008     |
+-----+----------+---------------+

How many lines would it take in some other languages? Perl needs this many:

It's that easy to transform input to almost any output, it's obviously trivial to do calculations on fields, change their orders etc as well. And you can also define headers, print page numbers, define page widths and so on easily as well.

Frank Sinatra - Sinatra at the Sands

One more and then I'm off for while:

I picked this album up because I got it so cheap. One of the best buys I have ever made regarding CD's. Live albums don't get much better than this and this is a great testament to why Sinatra was one of the great entertainers of the 20th century. Great singer, excellent comedian with an immaculate timing and an ability to improvise. Not only are the songs great to listen, the jokes crack me up every time I hear them.

"People ask me if Dean Martin is an alcoholic. I can tell you he is an unqualified, one hundred percent drunk. If drinking was an Olympic sport he would be the coach of the US team!"

On top of Sinatra's great performance the band is in great swing as well. This album has one of the best moods in any album. You can almost feel being at the Sands. People who were there were very lucky to witness this.

I'm not really even into this sort of music in general but if you don't like this album, there's something wrong with you, not the album.

Steely Dan - Aja

More reviews from rateyourmusic:

What can be said of one of the greatest albums ever that hasn't been said already? How the musicians managed to stay sane while doing this album, I don't know. Usually I'm not too interested in over-produced music, but this is so over-the-top produced that it takes on a different level.

It's obvious some get this album, some don't. From the out it could remind you of elevator music but I wish elevator music could be this good. I have to admit it took a few tries before I got this album, but just like Velvet Underground & Nico for instance music doesn't always have to be easy to understand. You have to grow to some albums.

Steely Dan has produced a number of great albums like Pretzel Logic and Can't Buy a Thrill but without Aja they would be just another good band.

The Velvet Underground - 1969 Velvet Underground Live With Lou Reed

For some reason I hadn't blogged some of the albums I have reviewed at the time I did them, so I'll blog them now, starting with a brilliant live album.

Velvet Underground, arguably one of the greatest bands of all time, at least one of the most influential, playing live at the top of their career sounds like a perfect recipe.

Well, it is. And isn't. Sort of. Depends on how fussy your are because the sound quality is, frankly, quite bad at some points. The underlying "whiz" sound is quite loud, the reproduction of different sounds is not very accurate and even though this is a stereo album (I think) most tracks sound like mono. I would have thought this *is* mono but for a few weak glimpses of stereo sound on some tracks. Overall, some tracks sound like they were recorded on a poor C-cassette, copied a few times and the final copy was used for the album. Note that this review is based on the vinyl version so it is possible that later CD versions (which I haven't heard) have better sound quality.

But once you start to listen to music, most of this is forgotten. The music is simply so brilliant especially because the tracks are performed live giving the artists the freedom the good ones benefit from. The album is full of superb tracks like "Waiting for My Man", "What Goes On", "Femme Fatale", "Beginning to See the Light", "Pale Blue Eyes" and "White Light/White Heat" to pick the best one. I'm sure this album can be listened to over and over again and every time you find something new. Somehow Lou Reed also sounds a bit more.. enthusiastic, if that's a good word, than he did later in his career. There is a bit more...spring (another interesting choice of words if I may say so) in his singing. It doesn't make the songs any more chirpy or happy than they are but somehow I like it. Lou Reed can sound really miserable but when he also sounds like he doesn't really care about the singing it can get a bit too depressing.

Funnily enough I finally completely realised why it is said that anyone who went to a VU concert started their own band. I've been practising with an electric guitar for a few days (which means I can barely get out a few notes) so I've been listening to the guitars in particular. They mostly sound quite simply yet the music is brilliant. There really isn't that much stuff that sounds difficult to play (at least with my limited understanding) so it's easy to start thinking that if it really is possible to make such great music with such simple ingredients why couldn't I? But the geniousnes of VU is not about skillful use of instruments but the way they merge together. Each instrument has a very distinct identity yet they form a fantastic whole.

Velvet Underground shows what great music is. Anyone should get one from The Velvet Underground & Nico, The Velvet Underground or this album and listen to them a few times. Say five. If you still don't like it, fair enough. But just in the off-chance that you do "get" the album you will forever be happy you gave it a try.

I "only" gave the album four and a half stars because of the sound quality. I'm fussy like that. Don't let that put you off, though.

Hidden Secrets of Git

This caused a bit of a headache. This is the scenario:

In repo 1:
- Edit some stuff,
- git commit,
- git tag -a,
- git push --tags.

In repo 2:
- git pull --tags,

-> The changes from the edit are visible.

This definitely isn't clear from the documentation:

--tags

All refs under $GIT_DIR/refs/tags are pushed, in addition to refspecs explicitly listed on the command line.

But very cool, nevertheless. This means it's not possible to accidentally only push the tags and not the changes associated with them.

On the Edge

I wrote a review of Brian Bagnall's brilliant On the Edge: The Spectacular Rise and Fall of Commodore to Librarything.

A brilliant book that pretty much makes other books about Commodore redundant. This book is so comprehensive that it's hard to think of what might be missing. This is a book about a very peculiar company and some brilliant people doing brilliant stuff. Commodore did some brilliant stuff, like the sound and video chips on the Commodore 64. I hadn't even realised that the CPU of C64 was actually the same as in VIC-20, C64 just had those chips and more memory, but it was on a totally different level performance and price wise to anything on the market at the time. The sound chip, SID, was way better than anything else in anything affordable to normal consumers even though it was a rush-job (like most of the stuff done at Commodore during their peak). The result was a chip that has produced some of the most memorable game music ever and is used to this day.

I also didn't know Amiga wasn't actually a Commodore product at first but the way it was built was reminescant of the way Commodore built some of it's products (PET, VIC-20, C64): by going far beyond what anyone else had ever done. Amiga, at the time of it's release, was an amazing home computer offering unrivalled price and performance. It's too bad Commodore marketing messed the North American market completely and effectively killed the product there. European sales were not enough to keep Commodore afloat in the end. What made Commodore a success at first was the combined price cutting by Tramiel who always wanted to sell at far lower prices than the competition and the engineers who didn't like compromises and were brilliant at what they did. To make such important products required both.

But first and foremost this book is about people. The odd, the brilliant, the obsessed people that made Commodore what it was. From founder and CEO Jack Tramiel who was not a very nice person and ruined a lot of lives but after whose dismissal by majority shareholder Irving Gould Commodore went downhill fast and would have folded a lot earlier without Amiga to the brilliant engineers like Chuck Peddle who created the computer section of Commodore and was then destroyed by Tramiel and the engineers who created C64, Robert Russell and Bob Yannes and many others. These guys were motivated to the point of obsession, brilliant at what they did and some very eccentric, none more so than Bil Herd, a brilliant hardware guy and alcoholic. The stories of some of the antics they made are just brilliant, like how and why Herd punched a hole in the wall of his office.

For someone who spent way too much of his youth playing with the C64 this book is a goldmine of information. For instance, the reason why the 1541 floppy disk was so ridiculously slow is revealed. Commodore was a major factor in bringing computers to home. Much more so than IBM and Commodore easily out-sold companies like Apple during the critical years when the home-market was created (the first three true home computers were TRS-80, Commodore PET and Apple II and the Apple sold way less than the other two). So this book is also a valuable tale of how the computer arrived to our homes. Everyone might have an IBM PC clone now but in the 80's it looked for a while like the future might be in Commodore machines but for some shocking marketing decisions. The whole computer business seems to have changed from brilliant engineers doing brilliant, ground-breaking work to the business-driven model of today where products are changed just enough to make people want to buy it.

The only little thing I was left hoping for was some sort of timeline of the events listed in the book, including the things competition did that was mentioned in the book. Sometimes it was hard to follow the order of the events because things happened in parallel so you might be tossed back several years with a new chapter. Another thing is, of course, that since this book is largely the voice of the engineers that made this happen, the opinions might be a bit lop-sided but this really only bothered me on a few occasions.

For me some of the best gaming experiences ever were with C64 and it's the reason I now work in the computer industry even though I never did do much programming on it. But I, like countless others, were drawn into the computing industry by Commodore. So their legacy is still strong even though the company folded in 1994. This book brings back many memories and also makes you wonder have we lost something while gaining computers with computing powers completely unthinkable 25 years ago.

A company that created PET, VIC-20, C64 and Amiga (they made it a success by creating a cheaper version which made it affordable to many more than the original Amiga 1000) deserves it's history to be told. This book does it and brilliantly. A must-read for anyone who has ever owned one of Commodore's computers.

Using signed tags with git

Another thing I tried out with git was signing tags with gpg keys. Things would have been a lot easier if I knew more of git or gpg - I'm not yet too familiar with either. Obviously I have used encryption with emails before, but since this has been with Outlook, the whole thing works a little bit differently (the interface for key ring handling is in Outlook).

So here's how you can do it:

1. The user creates a gpg key with gpg --gen-key.
2. Then he exports the public key with gpg --armor --export user@email.com > mypk
3. He sends the public key file to you for you to save it in the git user's keyring (assuming you are using gitosis). You then import it with gpg --import mypk. It might also be a good idea to sign this key with gpg --edit-key user@email.com.
4. Next you need to add the verification somewhere in the git hooks, pre-receive might be the best bet. Checking the validity is done with git tag -v "tag_id". The code could be something like (I haven't done this yet myself):


And hey presto! You have just made sure that your system only accepts tags signed by people you have accepted.

Improved security with git and gitosis

So, I started working on Linux and one of my first tasks has been to learn to use git, the open source version control system used by, among others, Linus Torvalds. I'm familiar with a number of SCM systems but git has quite a lot of new stuff for me. I also installed gitosis for added security. Gitosis removes the need to create user accounts for everyone who needs to read or write to the repository, which improves security a lot. Setting it up had it's small quirks which meant I couldn't use the otherwise excellent guides to the point. But I did get there eventually.

Anyways, what I wanted to do was to check that the user is who he says. By simply saying "git config user.email=whoever@whatever.com && git config user.name=Mr. Fake" a user can hide his identity - in practice allowing him to add what he wants to the repository as long as he has write access. Also, there is no extra security for anything else. For instance if you have conditional hooks in your git repository, you can't just the user id for access rights.

The solution is to use gitosis and check that the user really is who he says he is. I already asked this question in stackoverflow and then ended up solving the problem myself. The solution requires fixing gitosis, reinstalling it and the adding a pre-receive hook to the git repository. Not overly complicated, but hopefully someone could add that fix to the "official" gitosis code as well.

This solution verifies that the user email address used to create the ssh key for gitosis matches the address the user is using which should be pretty secure. This way the repository history is correct and the culprits can always be tracked down.