Crypted Android device startup sequence

This blog deals with Android Marshmallow, things might have changed since then, especially as newer version support FBE (File-Based Encryption). M and older only supported FDE (Full-Disk Encryption) and the way it works is far from obvious but can be explained relatively concisely. I'll try to explain the flow below without getting stuck into too much detail.

Partition mounting starts in system/core/init/builtins.cpp, function do_mount_all(). This calls function fs_mgr_mount_all() and takes note of the return value. fs_mgr_mount_all() tries to mount each partition. If it fails, then the function checks if "encrypted" option is set for that partition in the fstab file. In this case it returns FS_MGR_MNTALL_DEV_MIGHT_BE_ENCRYPTED.

If do_mount_all() receives this return value, it will set three properties:

ro.crypto.state = encrypted
ro.crypto.type = block
vold.decrypt = trigger_default_encryption

So at this point the system has seen there exist an encrypted device but does not know whether it has already been encrypted or not because this could be the very first boot.

trigger_default_encryption causes cryptfs_mount_default_encrypted() to be run. I.e. it starts the process of running the default encryption which is run the first time (with the default password). This function will check the type of password and if it has a valid value which is not CRYPT_TYPE_DEFAULT (i.e. user has changed it) it sets an important property:

vold.decrypt = trigger_restart_min_framework.

This starts the minimum framework which mounts the encrypted partition (/data in practice) to a temporary RAM disk and starts Zygote which then prompts the encryption password from the user. After user supplies the correct password, vold.decrypt is set to trigger_restart_framework, partition is decrypted and mounted (to /data) and Zygote is restarted. If the device wasn't encrypted (/data did not have the encrypted option) then init skips straight to this part.

This is, IMO, a pretty ugly solution and probably the easiest one for Google to handle encryption. It involves starting the almost the entire framework just to open a very simple view for the user to enter the password.

Using btrfs to speed up Android development

Building Android from the scratch is slow. In my work machine building Oreo takes about three hours. This is ok, if you don't need to do it too often but sometimes you have to run a full build for one reason or other. Often because something got corrupted and partial build won't finish.

A nice way to improve this would be to create a golden image which has a full build and make a copy of that when you want to do some work. If this new area gets corrupted, just make a new copy. The problem with this approach is it takes a lot of disk space. Just one workspace is almost 200 gigabytes and since SSD is the preferred disk type since it's much quicker to build using one of them it gets quite expensive if you need to support several variants. I have a terabyte of space for this on my work machine but I have potentially four different products to support.

Btrfs is an almost perfect solution: by using snapshots space requirements are much smaller. Btrfs only makes a new copy if a file is changed and since most files are unaltered the savings are huge.

Following is for my work machine which has two 500 gigabyte SSDs for this. Data is striped while metadata is mirrored.

sudo apt install btrfs-tools
sudo mkfs.btrfs -d raid0 -m raid1 /dev/sdc /dev/sddsudo

mkfs.btrfs /dev/sdc -L Work
mkdir /work
sudo mount -t btrfs /dev/sdc /work
You might need to change ownership with chown.
To add the file system to /etc/fstab you can use something like this:

UUID= /work btrfs defaults 0 0

UUID can be the UUID of any disk in the btrfs file system.

After this it's time to create the base volume. Base volume only contains the source code and is the only volume that is synced from main repository.

mkdir /work/product-BASE_VOL

btrfs subvolume create /work/product-BASE_VOL

Android source is fetched to this directory.

Next we create a snapshot of the base volume for building:

btrfs subvolume snapshot /work/product-BASE_VOL /work/product-BUILD-variant

A full build is done in this snapshot.

Now when we want to code something, we create a snapshot of /work/product-BUILD-variant and work in that directory. No need to do a full build. If we somehow mess up, just delete this directory (you might want to save changes first, or just use mv to rename the directory) and create a new snapshot.

The actual magic happens when you use a script to update the base volume and build directories every night. It just syncs the base volume and replaces the build directory with a fresh build. Now you have a fresh build every morning with the latest changes! Again, you might want to keep a copy of the old build directory for a while just in case.

You add this to your crontab by running crontab -e and adding the following:

MAILTO=""

PATH=/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin:/usr/local/sbin

01 00 * * * /path/to/script/update-golden.sh btrfspath product variant

Note that you might need to alter the PATH variable. This will run the script every day one minute after midnight.

This script will create a backup of the previous build directory. Old build directories are best removed with 

btrfs subvolume delete

References:

https://btrfs.wiki.kernel.org/index.php/Getting_started

https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices