Instant Android Systems Development How-to [Instant] — Save 50%
Get your hands dirty with Android Systems Development through carefully thought-out source code examples with this book and ebook
This article by Earlence Fernandes, the author of the book Instant Android Systems Development How-to, sets up your build computer and instructs you on how to download and build the Android operating system from scratch.
(For more resources related to this topic, see here.)
You need Ubuntu 10.04 LTS or later (Mac OS X is also supported by the build system, but we will be using Ubuntu for this article). This is the supported build operating system, and the one for which you will get the most help from the online community. In my examples, I use Ubuntu 11.04, which is also reasonably well supported. You need approximately 6 GB of free space for the Android code files. For a complete build, you need 25 GB of free space. If you are using Linux in a virtual machine, make sure the RAM or the swap size is at least 16 GB, and you have 30 GB of disk space to complete the build.
As of Android Versions 2.3 (Gingerbread) and later, building the system is only possible on 64-bit computers. Using 32-bit machines is still possible if you work with Froyo (Android 2.2). However, you can still build later versions on a 32-bit computer using a few "hacks" on the build scripts that I will describe later.
The following steps outline the process needed to set up a build environment and compile the Android framework and kernel:
- Setting up a build environment
- Downloading the Android framework sources
- Building the Android framework
- Building a custom kernel
In general, your (Ubuntu Linux) build computer needs the following:
- Git 1.7 or newer (GIT is a source code management tool), JDK 6 to build Gingerbread and later versions, or JDK 5 to build Froyo and older versions
- Python 2.5 – 2.7
- GNU Make 3.81 – 3.82
How to do it...
We will first set up the build environment with the help of the following steps:
All of the following steps are targeted towards 64-bit Ubuntu.
- Install the required JDK by executing the following command:
sudo add-apt-repository "deb http: //archive.canonical.com/ lucid
sudo apt-get update
sudo apt-get install sun-java6-jdk
sudo add-apt-repository "deb http: //archive.ubuntu.com/ubuntu
hardy main multiverse"
sudo add-apt-repository "deb http: //archive.ubuntu.com/ubuntu
hardy-updates main multiverse"
sudo apt-get update
sudo apt-get install sun-java5-jdk
- Install the required library dependencies:
sudo apt-get install git-core gnupg flex bison gperf build-
zip curl zlib1g-dev libc6-dev lib32ncurses5-dev ia32-libs \
x11proto-core-dev libx11-dev lib32readline5-dev lib32z-dev \
libgl1-mesa-dev g++-multilib mingw32 tofrodos python-markdown \
- [OPTIONAL]. On Ubuntu 10.10, a symlink is not created between libGL.so.1 and libGL.so, which sometimes causes the build process to fail:
sudo ln -s /usr/lib32/mesa/libGL.so.1 /usr/lib32/mesa/libGL.so
- [OPTIONAL] On Ubuntu 11.10, an extra dependency is
sudo apt-get install libx11-dev:i386
- Now, we will download the Android sources from Google's repository.
- Install repo. Make sure you have a /bin directory and that it exists in your PATH variable:
curl https: //dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/
chmod a+x ~/bin/repo
Repo is a python script used to download the Android sources, among other tasks. It is designed to work on top of GIT.
- Initialize repo.
- In this step, you need to decide the branch of the Android source you wish to download. If you wish to make use of Gerrit, which is the source code reviewing tool used, make sure you have a live Google mail address. You will be prompted to use this e-mail address when repo initializes.
- Create a working directory on your local machine. We will call this
- The following command will initialize repo to download the "master" branch:
repo init -u https://android.googlesource.com/platform/manifest
- The following command will initialize repo to download the Gingerbread 2.3.4 branch:
repo init -u https: //android.googlesource.com/platform/manifest
The -b switch is used to specify the branch you wish to download.
- Once repo is configured, we are ready to obtain the source files. The format of the command is as follows:
repo sync -jX
-jX is optional, and is used for parallel fetch.
- The following command will sync all the necessary source files for the Android framework. Note that these steps are only to download the Android framework files.Kernel download is a separate process.
repo sync -j16
The source code access is anonymous, that is, you do not need to be registered with Google to be able to download the source code. The servers allocate a fixed quota to each IP address that accesses the source code. This is to protect the servers against excessive download traffic. If you happen to be behind a NAT and share an IP address with others, who also wish to download the code, you may encounter error messages from the source code servers warning about excessive usage. In this case, you can solve the problem with authenticated access. In this method, you get a separate quota based on your user ID, generated by the password generator system. The password generator and associated instructions are available at https://android.googlesource.com/new-password.
- Once you have obtained a user ID/password and set up your system appropriately, you can force authentication by using the following command:
repo init -u https://android.googlesource.com/a/platform/manifest
Notice the /a/ in the URI. This indicates authenticated access.
If you are downloading from behind a proxy, set the following environment variables:
Next, we describe the steps needed to build the Android framework sources:
- Initialize the terminal environment.
- Certain build-time tools need to be included in your current terminal environment. So, navigate to your source directory:
The sources can be built for various targets. Each target descriptor has the BUILD-BUILDTYPE format:
- BUILD: Refers to a specific combination of the source code for a certain device. For example, full_maguro targets Galaxy Nexus or generic targets the emulator.
- BUILDTYPE: This can be one of the following three values:
- user: Suitable for production builds
- userdebug: Similar to user, with with root access in ADB for easier debugging
- eng: Development build only
- We will be building for the emulator in our current example. Issue the following command to do so:
To actually build the code, we will use make. The format is as follows:
Where X indicates the number of parallel builds. The usual rule is: X is the number of CPU cores + 2.
This is an experimental formula, and the reader should feel free to test it with different values.
- To build the code:
Now, we must wait till the build is complete. Depending on your system's specifications, this can take anywhere between 20 minutes and 1 hour. At the end of a successful build, the output looks similar to the following (note that this may vary depending on your target):
target Dex: SystemUI
target Package: SystemUI (out/target/product/generic/obj/APPS/
Finding NOTICE files: out/target/product/generic/obj/NOTICE_FILES/
Combining NOTICE files: out/target/product/generic/obj/NOTICE.html
Target system fs image: out/target/product/generic/obj/PACKAGING/
Install system fs image: out/target/product/generic/system.img
Installed file list: out/target/product/generic/installed-files.
DroidDoc took 440 sec. to write docs to out/target/common/docs/
A better check for a successful build is to examine the newly created files inside the following directory.
The build produces a few main files inside android_src/out/target/product/<DEVICE>/, which are as follows:
- system.img: The system image file
- boot.img: Contains the kernel
- recovery.img: Contains code for recovery partition of the device
In the case of an emulator build, the preceding files will appear at android_src/out/target/product/generic/.
- Now, we can test our build simply by issuing the emulator command:
This launches an Android emulator, as shown in the following screenshot, running the code we've just built:
The code we've downloaded contains prebuilt Linux kernels for each supported target. If you only wish to change the framework files, you can use the prebuilt kernels, which are automatically included in the build images. If you are making specific changes to the kernel, you will have to obtain a specific kernel and build it separately (shown here), which is explained later:
Faster Builds – CCACHE
The framework code contains C language and Java code. The majority of the C language code exists as shared objects that are built during the build process. If you issue the make clean command, which deletes all the built code (simply deleting the build output directory has the same effect as well) and then rebuild, it will take a significant amount of time. If no changes were made to these shared libraries, the build time can be sped up with CCACHE, which is a compiler cache.
- In the root of the source directory android_src/, use the following command:
To set a cache size:
prebuilt/linux-x86/ccache/ccache -M 50G
This reserves a cache size of 50 GB.
To watch how the cache is used during the build process, use the following command (navigate to your source directory in another terminal):
watch -n1 -d prebuilt/linux-x86/ccache/ccache -s
In this part, we will obtain the sources and build the goldfish emulator kernel. Building kernels for devices is done in a similar way.
goldfish is the name of the kernel modified for the Android QEMU-based emulator.
- Get the kernel sources:
Create a subdirectory of android_src:
git clone https: //android.googlesource.com/kernel/goldfish.git
git branch -r
This will clone goldfish.git into a folder named goldfish (created automatically) and then list the remote branches available. The output should look like the following (this is seen after the execution of the git branch):
origin/HEAD -> origin/master
Here, in the following command, we notice origin/android-goldfish-2.6.29, which is the kernel we wish to obtain:
git checkout --track -b android-goldfish-2.6.29 origin/android-
This will obtain the kernel code:
- Set up the build environment.
- We need to initialize the terminal environment by updating the system PATH variable to point to a cross compiler which will be used to compile the Linux kernel. This cross compiler is already available as a prebuilt binary distributed with the Android framework sources:
- Run an emulator (you may choose to run the emulator with the system image that we just built earlier. We need this to obtain the kernel configuration file. Instead of manually configuring it, we choose to pull the config file of a running kernel.
Make sure ADB is still in your path. It will be in your PATH variable if you haven't closed the terminal window since building the framework code, otherwise execute the following steps sequentially.
(Note that you have to change directory to ANDROID_SRC to execute the following command).
adb pull /proc/config.gz
cp config .config
The preceding command will copy the config file of the running kernel into our kernel build tree.
- Start the compilation process:
If the following comes up:
Misc devices (MISC_DEVICES) [Y/n/?] y
Android pmem allocator (ANDROID_PMEM) [Y/n] y
Enclosure Services (ENCLOSURE_SERVICES) [N/y/?] n
Kernel Debugger Core (KERNEL_DEBUGGER_CORE) [N/y/?] n
UID based statistics tracking exported to /proc/uid_stat
(UID_STAT) [N/y] n
Virtual Device for QEMU tracing (QEMU_TRACE) [Y/n/?] y
Virtual Device for QEMU pipes (QEMU_PIPE) [N/y/?] (NEW)
Enter y as the answer. This is some additional Android-specific configuration needed for the build.
- Now we have to wait till the build is complete. The final lines of the build output should look like the following (note that this can change depending on your target):
Kernel: arch/arm/boot/Image is ready
Kernel: arch/arm/boot/zImage is ready
As the last line states, the new zImage is available inside arch/arm/ boot/.
- To test it, we boot the emulator with this newly built image.
- Copy zImage to an appropriate directory. I just copied it to android_src/:
emulator -kernel zImage
- To verify that the emulator is indeed running our kernel, use the following command:
# cat /proc/version
The output will look like:
Linux version 2.6.29-gef9c64a (earlence@earlence-
Satellite-L650) (gcc version 4.4.3 (GCC) ) #1 Mon Jun 4
16:35:00 CEST 2012
This is our custom kernel, since we observe the custom build string (earlence@earlence-Satellite-L650) present as well as the time of the compilation. The build string will be the name of your computer.
- Once the emulator has booted up, you will see a window similar to the following:
Following are the steps required to build the framework on a 32-bit system:
- Make the following simple changes to build Gingerbread on 32-bit Ubuntu. Note that these steps assume that you have set up the system for a Froyo build. Assuming a Froyo build computer setup, the following steps guide you on incrementally making changes such that Gingerbread and later builds are possible. To set up for Froyo, please follow the steps explained at http://source.android.com/source/initializing.html. In build/core/main.mk, change ifneq (64,$(findstring 64,$(build_arch))) to ifneq (i686,$(findstring i686,$(build_arch))).
Note that there are two changes on that line.
- In the following files:
LOCAL_CFLAGS += -m64
LOCAL_LDFLAGS += -m64
LOCAL_CFLAGS += -m32
LOCAL_LDFLAGS += -m32
- Install the following packages (in addition to the packages you must have installed for the Froyo build):
sudo apt-get install lib64z1-dev libc6-dev-amd64 g++-multilib
- Install Java 1.6 using the following command:
sudo apt-get install sun-java6-jdk
The Android build system is a combination of several standard tools and custom wrappers. Repo is one such wrapper script that takes care of GIT operations and makes it easier for us to work with the Android sources.
The kernel trees are maintained separately from the framework source trees. Hence, if you need to make customizations to a particular kernel, you will have to download and build it separately. The keen reader may be wondering how we are able to run the emulator if we never built a kernel in when we just compiled the framework. Android framework sources include prebuilt binaries for certain targets. These binaries are located in the /prebuilt directory under the framework source root directory.
The kernel build process is more or less the same as building kernels for desktop systems.
There are only a few Android-specific compilation switches, which we have shown to be easily configurable given an existing configuration file for the intended target.
The sources consist of C/C++ and Java code. The framework does not include the kernel sources, as these are maintained in a separate GIT tree. In the next recipe, we will explain the framework code organization. It is important to understand how and where to make changes while developing custom builds.
Resources for Article:
- Android Native Application API [Article]
- Animating Properties and Tweening Pages in Android 3-0 [Article]
- So, what is Spring for Android? [Article]
About the Author :
Earlence Fernandes is pursuing a Ph.D. degree in Systems Security from the University of Michigan, Ann Arbor, USA. Previously, he was a Scientific Programmer at Vrije Universiteit, Amsterdam, where he built several Android research prototypes for various cutting-edge security mechanisms. He also co-authored several scientific papers on the topic. He has been playing with Android since 2008 and is an active member of the Android Google Groups for platform internals.