Using btrfs to speed up Android development

Building Android from the scratch is slow. In my work machine building Oreo takes about three hours. This is ok, if you don't need to do it too often but sometimes you have to run a full build for one reason or other. Often because something got corrupted and partial build won't finish.

A nice way to improve this would be to create a golden image which has a full build and make a copy of that when you want to do some work. If this new area gets corrupted, just make a new copy. The problem with this approach is it takes a lot of disk space. Just one workspace is almost 200 gigabytes and since SSD is the preferred disk type since it's much quicker to build using one of them it gets quite expensive if you need to support several variants. I have a terabyte of space for this on my work machine but I have potentially four different products to support.

Btrfs is an almost perfect solution: by using snapshots space requirements are much smaller. Btrfs only makes a new copy if a file is changed and since most files are unaltered the savings are huge.

Following is for my work machine which has two 500 gigabyte SSDs for this. Data is striped while metadata is mirrored.

sudo apt install btrfs-tools
sudo mkfs.btrfs -d raid0 -m raid1 /dev/sdc /dev/sddsudo

mkfs.btrfs /dev/sdc -L Work
mkdir /work
sudo mount -t btrfs /dev/sdc /work
You might need to change ownership with chown.
To add the file system to /etc/fstab you can use something like this:

UUID= /work btrfs defaults 0 0

UUID can be the UUID of any disk in the btrfs file system.

After this it's time to create the base volume. Base volume only contains the source code and is the only volume that is synced from main repository.

mkdir /work/product-BASE_VOL

btrfs subvolume create /work/product-BASE_VOL

Android source is fetched to this directory.

Next we create a snapshot of the base volume for building:

btrfs subvolume snapshot /work/product-BASE_VOL /work/product-BUILD-variant

A full build is done in this snapshot.

Now when we want to code something, we create a snapshot of /work/product-BUILD-variant and work in that directory. No need to do a full build. If we somehow mess up, just delete this directory (you might want to save changes first, or just use mv to rename the directory) and create a new snapshot.

The actual magic happens when you use a script to update the base volume and build directories every night. It just syncs the base volume and replaces the build directory with a fresh build. Now you have a fresh build every morning with the latest changes! Again, you might want to keep a copy of the old build directory for a while just in case.

You add this to your crontab by running crontab -e and adding the following:

MAILTO=""

PATH=/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin:/usr/local/sbin

01 00 * * * /path/to/script/update-golden.sh btrfspath product variant

Note that you might need to alter the PATH variable. This will run the script every day one minute after midnight.

This script will create a backup of the previous build directory. Old build directories are best removed with 

btrfs subvolume delete

References:

https://btrfs.wiki.kernel.org/index.php/Getting_started

https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices