Download code from GitHub
In this chapter, the user will learn about Android hardware and software architecture. We will provide an overview on the Android Compatibility Definition Document to properly understand what we need in order to create a fully compliant and certified device.
The user will learn about the Android Application Framework (AAF), the two different Android runtime systems—Dalvik, and ART, and a collection on Google-provided system libraries.
The user will have a first hands-on moment, setting up and running Android Compatibility Test Suite. We will test together an existing certified device and we will take the first step on the path towards the creation of a certified device.
Android, as with every other operating system, has a layer-based structure. The next image shows a properly abstracted overview of the whole system architecture:
We can divide the system into the following main layers:
Linux kernel
Hardware abstraction layer
Core libraries and runtime environment
Application framework
Binder IPC
Applications
The software layer closest to the hardware architecture is the Linux kernel. This layer is in charge of communicating with the hardware components and provides an easy-to-use interface for the layer above.
Moving up on the architecture path, we have Android runtime and core libraries. This layer provides the basics tools for the application framework. The application framework is a collection of ready-to-use components that the system provides to the Applications layer via the Android SDK. The top layer contains all those applications we use everyday—games, productivity apps, multimedia, and so on.
Android is based on the Linux kernel, but it's not a classic Linux-based desktop system: it's not Ubuntu. However, Android architecture designers and developers rely on the Linux kernel, because it's open source, it's extensively tested worldwide, and it can be easily tailored to fit Android-specific hardware needs, on any kind of device.
From a very pragmatic point of view, choosing to base the system on an open source heart reinforced the Android philosophy of being an open system, supported by its community and trusted by enterprise companies, thanks to its transparency. Besides, this approach saved a lot of development time—they didn't have to start from scratch and they could focus on the rest of the architecture, taking advantage of a popular and well-documented core.
The vanilla Linux kernel needed some love to properly fit all the Android requirements. Most of the contributions by Google were focused on:
Fixing bugs
Enabling new hardware
Improving power management
Improving error reporting
Improving performance
Improving security
From a hardware point of view, the Android team made a great effort to add new goodies to the Linux kernel. Lots of fixes and hacks were released to improve Bluetooth support and management, lots of General Purpose Input/Output (GPIO) drivers were added, ARM compatibility was enhanced, as ARM was the primary Android-supported architecture and also MMC management received lots of contributions. The new ADB gadget driver was added to help developers to communicate via USB with external devices.
From a memory point of view, the Android team introduced PMEM, the process memory allocator. This gave the ability to manage large physically contiguous memory regions between user space and kernel space. Working in a specific low-resource hardware domain, the Android team released Ashmem, Android Shared Memory, which targeted low-memory devices and provided an easy-to-use file-based API to manage shared memory, especially under memory pressure.
From a power management point of view, the Android team introduced an improved suspend system, wakelocks, and Android Alarm Timers, the kernel implementation to support Android Alarm Manager.
The other interesting contributions were the kernel support for Android logcat command, that provides logs of system messages, application debug messages, and exceptions, and Android Binder, an Android-specific interprocess communication system, used for remote method invocation too.
To overcome the increasing hardware fragmentation, Android engineers created an abstraction layer that allows the system to interact with the hardware just being aware of a specific intercommunication interface. The system completely ignores the low-level implementation of hardware and drivers. This approach enforces the idea of developing software against an interface instead of against an implementation. With this approach, the Android system does not know and does not need to know how hardware is accessed or managed.
As a mid-level layer between the hardware and the system, Android HAL is commonly developed using native technology—C/C++ and shared libraries. There is no constraint from Google about how we need to implement our HAL and our device drivers: it's up to us to design it as we think best for our scenario. There is only one simple rule:
Our implementation must provide the same interface that the system is expecting.
Going up on the architecture ladder, we find the two most important software layers. The Android application framework and Android system libraries are the middleware between the bare hardware, managed by the Linux kernel, and all those fancy, shiny apps we have on our smartphones.
Android system libraries are a set of libraries, specifically created to work on Android, to allow and help with system components and app development. The most important are:
SQLite: SQLite is the entry point to the SQL world. It's a tiny SQL implementation for embedded systems and it provides a standard way to access data published by content providers or SQL DB created by the user.
SSL: SSL provides the standard security environment for network communication.
OpenGL: OpenGL libraries are the link between the Java (and C/C++ JNI) world and the OpenGL/ES 3D graphics rendering API.
Media framework: Media framework provides codecs for rendering, recording, and playback for the most common media formats.
libc: The libc library is a BSD-derived implementation of the standard C library, specifically tuned to best perform on embedded Linux-based devices.
Surface manager: Surface manager manages access to the display subsystem.
This is the core of the Android software ecosystem. It provides a plethora of managers that facilitate the most common tasks of Android developers and the Android system itself. The most important components of the Application Framework are:
Activity manager: This provides the navigation backstack and manages the Android activity lifecycle
Resource manager: This provides access to noncode resources contained in the apps: graphics, localized string, styles, and colors
Location manager: This is in charge of providing the most accurate position information, using data collected by the GPS sensor, from cell towers and Wi-Fi networks nearby
Notification manager: This enables apps to display notification alerts in the status bar, according to Google Design Guidelines, to provide a common and familiar user experience
Content providers: This provides a common approach to share data between different apps, for instance, accessing contacts data or sharing a common data set between two apps
Views and widgets: These comprise the UI core of the Android experience. Buttons, text fields, and layouts are the building blocks of every Android system component and user app
Everything on Android is achieved using the official Android SDK that provides a consistent and documented way to use all these system managers, views, and logic components to let you create the next big hit of the Google Play Store.
From an Application Framework point of view, the Binder Inter-Process Communication (IPC) is a hidden layer. It takes care of creating a transparent communication channel between the high-level Android API, accessible via the Android SDK, and the actual Android system.
All the applications created by third-party entities, such as smartphone manufacturers or Android programmers will be installed on the application layer.
Usually, this relies on a read/write area of the handset solid memory, but for software provided by manufacturers, typically, it uses a read-only memory area to be sure that these applications will always be installed no matter what. Apps such as Google Maps, YouTube, Samsung TouchWiz Nature, and HTC Sense are examples of apps in this very group: they are shipped with the device's operating system, they are installed on a read-only memory area of the device, and they are meant to be uninstallable as a core component of the system.
As we will see, this is not 100% true—once you have the proper skill set, you will be able to manipulate the whole system. In the following chapters, you will acquire these skills and you will learn how to heavily modify an already existing Android version and get rid of those apps, if necessary.
Every successful Android device on the market, before being launched, has been certified. Manufacturers have designed, developed, and tested their device according to precise guidelines, rules, and constraints.
To make the task as easy as possible, Google has created the Android Compatibility Program that defines details and tools that help OEMs to create a device that will properly support the OS, the SDK, and the developers' expectations:
"To run Android apps on a variety of Android devices."
As a manufacturer, creating and distributing a certified device has critical importance. Our goal is to create a device with a unique, but at the same time familiar, user experience: we have to be cool, but not weird! Users want to customize their Android device and they want to be sure that their favorite apps will run smoothly, without problems of any sort. Developers want to be sure that they won't waste time fixing bugs on every different smartphone, tablet, or TV—they want a common ecosystem on which they can rely.
A well-defined and well-supported ecosystem brings more certified devices that bring more and more developers that bring more and more happy users. The following diagram shows exactly how the Android ecosystem lives thanks to the constant creation of well-designed, well-produced, certified devices:
The Android Compatibility Definition Document (CDD) is Google's way to specify guidelines, rules, and constraints to be considered for an Android-compatible device. Every device designer and manufacturer has to refer to the CDD to be able to easily port Android onto its own hardware platform.
For each release of the Android platform, Google provides a detailed CDD. The CDD represents the policy aspect of Android compatibility and its role is to codify and clarify all the requirements and eliminate any ambiguity. The main goal is to provide rules for manufacturers to let them create complex hardware devices, compatible with Android SDK and Android apps.
Designing and developing a new device is no easy task. Even the smallest detail matters. Think about OpenGL support. There is no possible way to be sure that the graphical experience will be great for the user. The only thing that's possible is working according to the guidelines and then "test, test, and test". That's why providing as many details and guidelines as possible is the only way to help the manufactures to achieve their goal.
However, the CDD does not attempt to be comprehensive—it couldn't be. It just serves as guidance to approach as easily as possible the final goal—a compatible device. Further help comes from the source code itself and from the Android SDK API that can be considered a compatibility-proof test bench. Think about CDD as an overview of the minimum set of constraints to be compliant with: it's the very first step of the journey.
In the beginning, Android was born to run on digital cameras. Luckily for us, a lot has happened since then: smartphones invaded our world! Then we had tablets and mp3 players. Nowadays, we have TVs, watches, media centers, glasses, and even cars, running Android and Android apps. Every device on the market will probably land in one specific category, according to its features. CDD gives a few pointers about which category your new device would be placed in:
Every device with an embedded touchscreen, a power source that allows mobility, and that can be held in hand can be considered an Android Handset.
An Android Television device is a device designed for media content: video, music, TV, games, with users sitting about three meters or ten feet away. This kind of device must have an embedded screen or an output video interface—HDMI, VGA, DVI, or a wireless display port.
A device designed to be worn on a wrist, with a touchscreen display with a diagonal between 2.79 cm and 6.35 cm is considered an Android Watch.
Having a car with an infotainment system, based on Android, gives us an Android Automotive implementation.
From a software execution point of view, the basic requirement is being capable of executing the Android Dalvik bytecode. Our device must support the Android Application Programming Interface and must provide complete implementations of any documented behaviors of any documented API exposed by the Android SDK or annotated with the @SystemAp annotation.
Hardware compatibility is a tricky task, because even if our device is lacking some specific hardware, for instance GPS or accelerometers, our implementation must contain GPS-related code and should be capable of handling inappropriate requests in a reasonable way to avoid crashes or misbehaviors.
One of the main players of software compatibility is the ability of our device to support intents. Every device properly implementing Android API must support Android loose-coupling intent system. Intents allow Android apps to easily request functionality from other Android components and avoid the effort to implement everything from scratch. The Android system has a set of core applications that implement the intent pattern:
Desk clock
Browser
Calendar
Contacts
Gallery
Global Search
Launcher
Music
Settings
As a vendor, we could integrate the default Android components or implement our own component, according to the public API. Those components will have special system permissions to act as system apps and they will be the first proposed choice for the matching intent filter.
For instance, when a developer ask to open a web page, the system will suggest "our browser component" as the first chosen app to perform the task. Of course, being a good citizen means that we must provide a proper settings menu to give the user the possibility to override our default choice and let the final user pick a different app for the task.
Android applications development is mostly based on Java programming. The SDK is based on Java, the runtime system is fully compliant with Java6, partially with Java7, and Google is already experimenting with Java8. Most developers will easily approach the platform if they already know Java programming language. However, Android offers a lot more to those developers that are dealing with heavy-duty, performance-oriented scenarios: Android Native API.
Native API gives the developers the opportunity to call native C, and partially C++, code from an Android Java application. Native code is compiled as standard ELF .so files and stored in the app APK file. Being native code, it has to be compiled for every architecture we are going to support, because, contrary to the bytecode, it can't be built once and run on every architecture.
As integrators, we must embrace one or more Android Application Binary Interfaces (ABIs) and aim for having full compatibility with the Android NDK. Of course, Google provides guidelines and constraints to easily reach this goal. These are the basic rules for proper compatibility:
Our implementation must include support for code running in the managed environment, that is Java code, to call into native code, using the standard Java Native Interface (JNI) semantics
If our implementation supports the 64-bit ABI, we must support its relative 32-bit version, too, because we must provide compatibility to non-64 bit potential devices
Google suggests that we build our implementation using the source code and header files available in the Android Open Source Project—just don't reinvent the wheel
From a libraries point of view, our implementation must be source-compatible (that is, header compatible) and binary-compatible (for the ABI) with all the following libraries:
libc (C library)
libm (math library)
liblog (Android logging)
libz (Zlib compression)
libdl (dynamic linker)
libGLESv1_CM.so (OpenGL ES 1.x)
libGLESv2.so (OpenGL ES 2.0)
libGLESv3.so (OpenGL ES 3.x)
libEGL.so (native OpenGL surface management)
libjnigraphics.so, libOpenSLES.so (OpenSL ES 1.0.1 audio support)
libOpenMAXAL.so (OpenMAX AL 1.0.1 support)
libandroid.so (native Android activity support)
libmediandk.so (native media APIs support)
These libraries also provide minimal support for the C++ JNI interface as well as support for OpenGL.
An implementation of each one of these libraries must be present in our system to be compatible with Android NDK. This is a dynamic list and we cannot treat it as a definitive set of libraries: future versions of Android could add new libraries and increase development possibilities and scenarios. That's why native code compatibility is challenging. For this reason, Google strongly suggests to use the implementations of the libraries listed earlier from the Android Open Source Project, taking advantage of the Open Source philosophy of Android and to enjoy well-supported and well-tested source code.
Nowadays, all major manufactures are switching to 64-bit architecture and new ARMv8 architecture deprecates lots of old CPU operations. Unfortunately, the market is still full of 32-bit compatible software and even on 64-bit architecture we must still support these deprecated operations, to avoid scaring developers and losing precious market share. Fortunately, we can choose to make them available via real hardware support or software emulation, at the expense of performance.
Supporting 32-bit architecture can be very tricky. We can just think about one simple scenario, for example, accessing the /proc/cpuinfo file. Legacy versions of the Android NDK used /proc/cpuinfo to discover CPU features. For compatibility with applications built using 32-bit NDK, we must specifically include the following things in /proc/cpuinfo when it is read by 32-bit ARM applications:
The tricky part is that these requirements only apply when /proc/cpuinfo is read by 32-bit ARM applications. The file must be not altered when read by 64-bit ARM or non-ARM applications.
The original Android runtime implementation was Dalvik. Dalvik was a virtual machine, specifically created for Android, due to the necessity to target low-memory devices. It was an integral part of the system until Android KitKat.
As we already said, Android applications are mostly written in Java. When Dalvik was the in-use runtime system, the Java code was compiled into bytecode. This bytecode was then translated to Dalvik bytecode and finally stored into a .dex (Dalvik Executable). After this procedure, Dalvik was able to run the Android app.
Although Dalvik had been designed for slow devices, with low memory, its performance has never been astonishing, not even when the Just-In-Time compilation was introduced, back with Android 2.2 Froyo. Dalvik JIT was supposed to bring a huge performance boost to Android apps and, from some points of view, it did, but with limitations, such as the infamous maximum methods number, and the pressure from alternative solutions forced Google to look forward to a new runtime:
When Android 4.4 KitKat was released, users could select a new experimental runtime environment in the Settings menu: ART. Android RunTime or, shortened, ART, is the current default runtime solution that replaced Dalvik from Android 5 Lollipop. The previous diagram shows a comparison between Dalvik and ART architecture.
The idea behind Dalvik's JIT (just-in-time) execution was to profile the applications while they were being executed and dynamically compile the most-used segments of the bytecode into native machine code. Native execution of these most-used segments called traces would then greatly speed-up the execution of the application even though most of the code would still be interpreted.
Art re-introduces the concept of AOT (ahead-of-time) compilation. It works as most compilers do, that is, it compiles the whole application code into the native machine code, without interpreting bytecode at all. This takes some time, but it is done only once when the user downloads the app, so considering the time and amount of resources needed for JIT profiling and optimization that are needed on every application start, it is an acceptable trade-off. Also, since the whole application is now compiled, it is quicker overall and the power consumption is reduced, which improves the device autonomy.
ART is the default runtime since Android 5, but Android needs to ensure compatibility with all those apps that are already on the market and all those devices that are running a previous version of Android and won't receive any operating system updates.
For backward compatibility reasons, the input bytecode is the same for ART and Dalvik. The application APK file still contains standard .dex files, but replaces the .odex files (Optimized Dalvik Executables) with the standard Unix ELF files (Executable and Linkable Format). During the installation, ART uses dex2oat utility to compile the bytecode into native code stored in the ELF file. As already mentioned, this step is performed only once and requires fewer resources and less overhead than Dalvik's JIT compilation. The downside is that the APK files are larger because they effectively contain double the code (uncompiled bytecode and compiled executable). After this compilation, the system will run just the ELF executable.
The bottom line is faster apps, but a bit less free space on your smartphone memory.
AOT compilation is not the only improvement that ART brought in. One of the most important features is the improved garbage collection. Garbage Collection (GC) is a form of automatic memory management, completely different from the old idea where the developer was the one in charge of allocating memory when needed and freeing it when it was not needed anymore.
The whole philosophy is based on the concept of Garbage Collector, an entity that tries to reclaim memory occupied by objects that are not used anymore in the program. It's a well-known tool in the Java world and Android has always suffered from its downside—GC is very slow and blocking.
Android 2.3 introduced the concurrent garbage collector—GC is not blocking the app anymore when it occurs, but there will always be an overall slowdown when it occurs. Finally, ART introduced a few more performance improvements:
Just one pause for garbage collection instead of Dalvik's two pauses
GC processing is now parallelized, reducing the duration of the GC pause
New Rosalloc memory allocator that uses thread-local region allocations for smaller objects and separate locks for bigger objects, instead of a single global lock
Full garbage collection is run only when the phone is locked so that the user doesn't notice when the GC is run
There is a compacting GC that reduces memory fragmentation and so diminishes the need to kill other applications just because bigger contiguous memory chunks are needed
From a development and debugging point of view, ART brought in the support for sampling profiler, support for more debugging features, and improved diagnostic details in exceptions and crash reports.
The upcoming version of Android will bring some enhancement to the current ART runtime. Google will introduce a so-called Profile-guided JIT/AOT compilation. JIT stands for Just In Time and looks similar to the old Dalvik approach: a compiler with code profiling capabilities. This JIT compiler will work together with ART and will provide constant performance improvement as it will continuously be profiling code and resource usage.
To improve performance during the installation phase, ART won't pre-compile Ahead-Of-Time the whole app. Instead, thanks to the profiling approach, it will detect hot methods in the app and will only pre-compile them, leaving unused parts of the app uncompiled. This precompilation process is smartly performed when the device is idle and charging, to have the smallest negative impact on the user experience and allow the user to install in instants apps that in Android 6 would take several seconds to be installed.
This whole new approach aims to improve applications and system performance on low end devices, reducing RAM memory footprint, battery draining and increasing runtime performance, for a satisfying Android experience on a wide range of devices.
We are aware of the CDD and we did our best to create a compatible device. A lot of aspects could still have glitches and we surely want to get rid of them. To make sure that everything works as expected, our Android implementation must be tested with Android Compatibility Test Suite. Android CTS will accompany us throughout the journey to our certified device. We will constantly use it to keep an eye on what is working and what is not working yet.
Every new version of Android platform comes with a new Compatibility Test Suite (CTS). This automated testing suite has two main components:
There is also the CTS Verifier, a tool for manual testing that consists of the verifier app that is executed on the device and collects the test results; and other executables or scripts that are executed on the desktop machine in order to provide further data or control for some test cases in the Verifier app.
The following diagram shows the CTS workflow:
The test suite on your computer will install the test on the device and will launch it. The device will test that particular subset of features and will give the results back to the test suite on your computer. The test suite will store these results, install the next test, and will start the cycle again, until every test is executed.
Currently, the CTS provides two main types of test cases:
Unit tests
Functional tests
Unit tests test the smallest logical units of code within the Android platform, for example, a single class, such as java.util.HashMap.
Functional tests are used to test a specific function that can consist of numerous API method calls.
Google is planning to provide more tests in the future versions of the test case. A couple of ideas are:
The following table shows the areas covered by the Compatibility Test Suite:
|
Area |
Description |
|---|---|
|
Signature tests |
For each Android release, there are XML files describing the signatures of all public APIs contained in the release. The CTS contains a utility to check those API signatures against the APIs available on the device. The results from signature checking are recorded in the test result XML file. |
|
Platform API Tests |
These tests test the platform (core libraries and Android Application Framework) APIs as documented in the SDK Class Index to ensure API correctness, including correct class, attribute and method signatures, correct method behavior, and negative tests to ensure expected behavior for incorrect parameter handling. |
|
Dalvik Tests |
These tests focus on testing the Dalvik Executable Format. |
|
Platform Data Model |
The CTS tests the core platform data model as exposed to application developers through Content Providers, as documented in the SDK |
|
Platform Intents |
The CTS tests the core platform intents as documented in the SDK Available Intents. |
|
Platform Permissions |
The CTS tests the core platform permissions as documented in the SDK Available Permissions. |
|
Platform Resources |
The CTS tests for correct handling of the core platform resource types, as documented in the SDK Available Resource Types. This includes tests for: simple values, drawables, nine-patch, animations, layouts, styles and themes, and loading alternate resources. |
Our journey will be very practical and hands-on, that's why in this section we are going to set up Android Compatibility Test Suite to test an existing device. We can't start working on our own Android implementation without knowing what we are going to support and test. To be able to run Android CTS, we will need:
A computer running Linux or OS X
Android SDK: http://developer.android.com/sdk/installing/index.html
Java SDK 6 or 7: http://www.oracle.com/technetwork/java/javase/downloads/index.html
Android CTS: http://source.android.com/compatibility/downloads.html
Android CTS Media: https://dl.google.com/dl/android/cts/android-cts-media-1.1.zip
There are a lot of files to download. In the meantime, we will set up our device.
We are testing an existing device, a smartphone, so we are already satisfying needs such as having a screen and we can move to device software configuration.
Tests should be executed on a clean device, so we should run a Factory Restore to erase all the data on the smartphone. Be sure of having a backup of your data if you are not using a development device. On Android 4.4 KitKat, you can reach the specific menu by navigating to Settings | Backup & reset | Factory data reset.
This will take a while—the device will shut down and the erasing process will start. The procedure will remove every single byte that is not part of the original Android system provided with your device, restoring all the settings and bringing the device to its original setup.
When the device restarts, we need to select English US language by navigating to Settings | Language & input | Language.
Now we need to turn on the Location: We need Wi-Fi and GPS and we need to provide some Internet connectivity. We need to disable any Screen Lock by navigating to Settings | Security | Screen Lock = 'None'.
We need a few settings from the Developer options menu. On a brand new installation of a vanilla Android system this menu is hidden. We can enable using the following steps:
Navigate to Settings | About phone.
Scroll to the bottom.
Tap continuously on build number item.
You are now a developer!
Note
If you are working with an HTC, Samsung, or Sony device and its custom version of Android, the previous steps could be a bit different. We leave it as an exercise to find the right navigation path for your non-vanilla Android version.
Once the Developer options menu has been enabled, navigate back to the Settings screen. In the Developer options menu, we need to enable the following:
USB debugging
Stay awake
Allow mock locations
Before running any tests, it's important that the device is on a steady support to avoid triggering accelerometers and the gyroscope. The camera should be pointing to a focusable object. Don't press any buttons or keys during the tests—this could invalidate test results.
To properly run all the tests, we will need a few multimedia files on the device—Android CTS media files. First of all, let's connect the device to the USB. If this is the first time that you connect this device to this host PC, the device will display a dialog to authorize the connection—allow the connection:
Any Android device can communicate with a host PC using Android ADB. This key tool is covered in great detail in the next chapters, so, for now, we can start downloading the latest Android SDK from https://developer.android.com/studio/index.html#downloads, according to your platform. Once the download is completed, decompress the file and you will be provided with an android-sdk folder, containing a platform-tools folder, containing adb executable.
Back to our media files setup now:
Open a terminal.
Navigate to the downloaded file, for instance:
$ cd ~/DownloadsUnzip the file:
$ unzip android-cts-media-1.1.zipEnter the brand new
android-cts-mediafolder with:$ cd android-cts-mediaThis folder contains a file that we must make executable:
$ chmod u+x copy_media.shNow we are ready to copy all the media files we need onto the device:
$ ./copy_media.sh all
The next screenshot shows the output of the whole procedure:
Everything is in place now and we can use cts-tradefed to run some test plans. Move to the Android CTS folder and run the following command to enter the cts console:
$ ./tools/cts-tradefed
The previous screenshot shows how cts-tradefed automatically identifies our connected device and gets ready to test.
CTS console provides a few useful commands:
list plans: This will list all the available test plans in the repositorylist packages: This will list all the available test packages in the repositoryrun: This will allow us to run all the tests we want
Typically, the following test plans are available:
All CTS tests required for compatibility
Signature tests the signature verification of all public APIs
Android tests for the Android APIs
Java tests for the Java core library
VM tests for ART or Dalvik
Performance tests for your implementation
As our first approach to CTS, we are going to run CTS plan:
cts-tf > run cts --plan CTS --disable-reboot
The testing will start immediately and the console will be full of log messages in the blink of an eye, as shown in the following screenshot:
Now, grab some coffee or make some good tea: this will take a while. cts-tradefed will test everything that is possible to test with an automatic test. Luckily for us, there is a lot that can be tested in this way.
Time has passed, the tea has gone, and the tests are over. On a quad-core smartphone, such as a Motorola Moto G or Nexus 4, this could take up to 10 hours. Eventually, we have got some nice results to check out. According to the folder's path we are working in, we will have results in a .zip file in the cts folder:
$ unzip ~/bin/android-cts/repository/results/START_TIME.zip
Unzipping the file, we will find a testResult.xml file. Opening this file with a recent web browser (Firefox is working fine here) will show plenty of meaningful tables, with all kind of test and results. The next screenshot shows the initial Test Summary. We have information about the test duration, how many tests were executed, how many tests passed, and how many tests failed:
As you can see, even testing a certified smartphone, currently on the market, will produce some failed tests. This gives you an idea about the complexity of producing the perfect Android device.
The next screenshot shows Test Summary by Package, specifying the test results one test after the other. For brevity, we are showing just a subset of the results:
The previous Test Summary screenshot shows that 29 tests have failed. If we dig into the test result file, we see that detailed reports are also available. This further information is hugely useful to precisely spot the failed test, like the one in the following screenshot, and investigate the issue:
The test result file tries to play polite and, for brevity, does not show the full stacktrace of the failed test. To reach the stack trace of the failure, we must inspect the source code of testResult.xml. For every executed test, there is a corresponding <Test> tag. For those tests that failed, we will have a <StackTrace> tag too. That's what we are looking for!
As a final note, testResult.xml contains a huge section with all the information about the device it has been able to retrieve. It's a very large amount of data that, for brevity, we are not reporting here, not even as an example.
We already know that there are lots of APIs and functions that we can automatically test with cts-tradefed, but what about all the other APIs and functions that cannot be tested within an automated environment?
CTS Verifier comes in every time an API or a function cannot be tested on a device without manual input. These are scenarios involving audio quality, touchscreen effectiveness, accelerometer precision and reactivity, camera quality, and features that are meant so specifically for human interaction that they are impossible to test without human interaction.
All we need to run CTS Verifier is an Android certified device and the appropriate CTS Verifier APK file. As we are testing an Android 4.4 device, we need to pay attention to downloading the proper CTS Verifier version. You can download the APK for your Android version and device architecture here: http://source.android.com/compatibility/downloads.html.
You just need to unzip the downloaded file and you will find a folder hierarchy and two .apk files. You can install CtsVerifier.apk using ADB:
$ adb install –r CtsVerifier.apk
The following screenshot shows the properly installed CTS Verified app and the initial screen:
As we know, CTS Verifier contains tests that need manual input to execute, evaluate, pass, or fail. Every test has its own Info screen that helps the tester to perform the test. As an example, we will run the Accelerometer Test, in the Sensors section.
Launching the test, we are welcomed by the info screen, as shown in the following screenshot:
The Info button explains how to perform the test and what to evaluate. As we move into the testing, we can evaluate if the accelerometer is working as expected. The following screenshot shows three different moments of the test:
The smartphone lays on the desk
The smartphone is held in hand, in portrait mode
The smartphone is held in hand, in landscape mode
As specified in the Info button, the arrow is always pointing in the same direction as the gravity: the sensor is working properly. We can consider that we have passed the test and click on the Pass button.
We have passed our first test. CTS Verifier provides dozens of tests and, one by one, we are going to run, verify, and pass them, in the long journey towards our first Android Certified Device.
When every test has been executed, we can save the result using the Save icon in the top-right corner of the initial screen, as shown in previous screenshot. The results will be saved on the device and a dialog box will show the precise path, as shown in the following screenshot:
Now, let's open a terminal and copy all the results from the phone to our computer:
$ adb pull /mnt/sdcard/ctsVerifierReports/ . $ unzip *.zip
At this point, we have a ctsVerifierReport-[…].xml with all the info about our manually executed tests.
Congratulations! You have fully tested an Android device. Step 0 of our journey is complete.
In this chapter, we learned what we are going to need to create a certified Android device. We saw the Android Compatibility Definition Document and we learned how to design a system to match the Android architecture. We had an overview of the two different runtime systems: Dalvik and ART and their main differences.
We had a full immersion into Android device testing, we learned how to run CTS automated tests and CTS manual tests on a already certified device.
The next chapter will be very hands-on. We will learn how to retrieve Android source code and we will understand the code structure and organization.
