Android explained

I will try to make it short and simple.

The Launcher

Most users will only see the Launcher of Android. This Launcher or Home Screen is the GUI between the user and Android.

There are many launchers beside your stock launcher. In general more or less the same. They present App icons and Widgets on multiple screens, an App Dock and and App drawer.

Some interesting launchers:

Apex Launcher

Microsoft Arrow Launcher

Asus ZenUI Launcher

Total Launcher (the best Launcher ever made)

Under the hood

Basically Android is an linux-ish OS with a Java VM Engine. The OS takes care of the booting, loading drivers for the hardware and managing the memory. Java is used for the GUI and the packages called APK's or apps.

When I use the word binary in this story this is the same as an linux program.

An overview of all Android versions and API/SDK levels can be found here.

Your Android device is basically build with the following components:

• CPU/GPU

• Flash memory to store ROM (Android)

• RAM memory for running programs

• LCD screen with LED backlight and Touch input layer

• Cellular Telephone module (GSM/CMDA/LTE)

• Wifi/Bluetooth

• Battery

• Speaker(s)

• Buttons

• GPS module

• SDCard reader

Android ROM Buildup

Android and the firmware or rom is built up with separate blocks. To name a few of these blocks (emmc/mtd) partitions:

• recovery - special boot mode for special functions on the partitions or files (prefer ClockWorkMod)

• boot - starts your device, loads the linux kernel and drivers, shows battery level when charging in Off mode

• system - linux binaries, libraries, configuration files, Java Engine an system APK's

• data - the user data, settings, user installed apk's, temp files, wiped at factory reset

The filesystem used in each partition varies. Today most use ext4.

On your device check /dev/block/bootdevice/by-name to match block device to partition. (or find /dev/block -name "by-name" or use blkid )

Older devices show partitions in /proc/mtd

System partition

The system partition is mounted at /system in the file system of Android and contains the Android system part. This part is readonly for normal users unless you decide to root your device. (see Rooting later on). This system partition is static and never changes until you flash a new ROM.

Some interesting folders from the /system partition:

• /system/app - the system apk's, preinstalled apps, Android 5 also stores the apk's binary libs here instead of /system/lib

• /system/priv-app - system apk's, like above

• /system/etc - the OS configuration files, /etc is linked to this folder

• /system/lib - the libraries used by the OS

• /system/bin - binary executables

• /system/xbin - binary executables

• /system/framework - graphic resources and java apps (am, services)

• /system/vendor/ - can have all system subfolders from above

• /system/build.prop - Android properties file, this are settings for the OS and apps

Data partition

The data partition is mounted at /data and is basically blank when the device is turned on the first time or after a factory reset. Normal users cannot write to this partition. Only applications can. Each application gets a unique UID when it is installed on the device. With this UID the app can write to the designated data folder.

The /data partition holds these folders:

• /data/app - user installed apps

• /data/data - both user and system app have a folder here for their settings

• /data/dalvik-cache - the Java VM optimized bytecode parts of apps (Dalvik Executable or dex files)

• /data/dalvik-cache/arm - the newer ART optimized bytecode

• /data/local/tmp - users can write to this one since owner is shell (UID 2000)

• /data/misc - holds data for devices as wifi, bt, gps etc.

• /data/system - holds package and UID administration, usage and battery stats, user accounts

  • /data/system/throttle - bandwidth usage

We know now that apps store their info in the /data/data/<appname> folder. This folder has a mix of file types in which the app stores its info. You will find plain xml files, sqlite db files and even binary library files (.so) that the app needs but are not common in the /system/lib folder. These are like dll's in MS Windows analogy.

Variables and Settings

There are two "worlds" of variables that you can find in Android. Since it is based on linux you have the sysctl variables from a linux kernel.

Also the memory management of your Android device is taken care of by linux.

You can get them with the command: sysctl -a or sysctl vm.swappiness

You can set them with the command: sysctl -w vm.swappiness=100

Secondly we have the Android specific variables. You can find a few in the /system/build.prop or the /init.rc file.

You can list them with the getprop shell command: getprop ro.fling.duration.coef

You can set them with the setprop command: setprop ro.fling.duration.coef 2.0

In addition to variables Android has settings stored in a Sqlite3 db. These settings are stored as content in com.android.providers.settings system, secure and global tables.

content query --uri content://settings/system --sort "name ASC"

content query --uri content://settings/secure --sort "name ASC"

content query --uri content://settings/global --sort "name ASC"

From version 4.2+ you can use the settings command too.

settings get global wifi_idle_ms

The Android properties and settings vary for every Android version. See my Android Tweaking page for more details.

The memory management

Basically you need to study linux for this. First we only look at RAM memory. If your device has for example 1 GB RAM you can run a lot of apps simultaneously. When you turn on your device first the linux kernel will load with the drivers and modules and then Android.

Linux will divide your RAM into working memory for programs and a part for buffers for program data and file cache. Linux will allocate this dynamically.

If too little working memory is available at a given point the cache buffers will shrink and/or background apps will be stopped by OOM. (Out Of Memory)

The app running on your screen is in foreground. Together with the system services and apps that need to run in working memory it will stay in working memory. Apps that have run recently will be moved to active memory (if memory left) for quick startup later. The kernel can remove them from the cache at any time if more working memory is needed. So on Android both file cache and recently used apps will live in cache memory.

You can see this RAM process on your device if you goto Settings > Apps > RUNNING

At the bottom of the screen you see RAM with a blue and a grey part. You can click on either color. Cached RAM is not free RAM per definition! Linux will always try to use RAM for file caching.

You can also add SWAP memory (disk or file) into the process. This will cause memory pages (4KB) to be moved to swap (paging). Newer linux kernels support zram. This will create a compressed RAM drive that will act as swap memory. Basically what the kernel does is moving unneeded used working memory into the swap drive and move it back when needed. This process is called paging but will take milliseconds of time and is only useful with low memory devices.

Don't mix up cache RAM memory with swap memory (paging). They are different things. The fact that zram disks live in RAM is only for speed and RAM compression. Cache is only for file buffering and recent apps. But even cache memory can be swapped/paged out.

On top of the linux RAM management comes the Android Dalvik VM management.

What is adb?

Android Debug Bridge is an interface intended for Android developers. We are not developers but never the less it is very interesting and useful. Mainly it is use via the usb port. But it also can be used via Wifi (see "adb over wifi" on the Android tweaking page). On your device you need to turn on the USB debugging feature in the Development Settings.

Now you need a Windows (Mac) USB driver for your device to work in adb mode. Here you find the latest Google adb usb driver for Windows.

The main use of adb is getting things done on your device from a dos box in Windows. Rooting the device first is needed. Most used commands are:

• adb shell followed by the su command, after this it is linux commands

• adb shell <cmd> Example to list the data folder on the device: adb shell ls -l /data

• add shell netcfg Get your network card names and actual IP addresses

• adb pull <file> Copy a file from the device to your PC

• adb push <file> Copy a file from your PC to the device

• adb install <apk> Install an app (apk) from your PC to the device

• adb logcat get Android log file

• adb reboot recovery

If you use the adb shell frequently it is good to know the quit command for interrupting commands as top and ping. In the shell type stty and you will find that it is CTRL+\ and ENTER

Download the latest from below in platform-tools or via installing the complete SDK tools.

Rooting

Rooting is getting root user access to the device. You become root (UID 0) also known as superuser or su. The big question now might be why? The answer is simple. You want to change some this on the /system partition:

• uninstall some factory apps (remove bloatware)

• Note: best practice to rename the apk to apk.OFF to keep the original

• change /etc/hosts file to block certain domains

  • change some conf file for gps or add something to the build.prop file

  • Note: better use a new default.prop file there, which does the same

• make the /system partition writable. Example: mount -o remount,rw /system

For the process of rooting you need two files: su binary and SuperUser.apk. For safety reasons we don't want that any program uses root without us (the user) knowing it. su and SuperUser.apk work together. When a programs wants root access the su binary is called. This is when SuperUser.apk kicks in. It will ask on screen permission from the user if that is OK.

The su binary is usually copied to the /system/xbin folder. It has special permission on it. The owner.group of the file must be root.root (0.0) and the sticky bit is set for the owner (chmod 4755 su). That means if it is executed it is executed as root, whoever starts it.

To make more of rooting a binary called busybox and sqlite3 are installed too. Busybox has a lot of little programs in it. When you have root acces type busybox in a shell and you will see all the commands available. The busybox binary is also copied to the /system/xbin folder. It has 755 permissions and usually also root.root ownership.

To install busybox in the /system/xbin folder with all its apps use: /system/xbin/busybox --install -s /system/xbin

To remove all its symbolic links again use: find /system/xbin -type l -delete (assuming there are no other symbolic links in /system/xbin)

Note1: su should be the only file in /system/xbin with 4755 permissions. sqlite3 and busybox should have 755, else you compromise security big time.

Note2: you can use the shell of busybox too instead of the standard shell if you rename /system/bin/sh and create a net link to the busybox shell: ln -s /system/xbin/busybox /system/bin/sh

Rooting in Recovery mode with CWM or TWRP

Default recovery mode won't let your root. You need TWRP or CWM Recovery to be flashed over your standard Recovery partition on the device.

When in a custom recovery mode you can use ChainFire's SuperSU and install that ZIP from recovery.

To root you need to get the su binary into /system/xbin. But you cannot write there. Recovery mode it the solution. In starting your device in Recovery mode with adb reboot recovery (or hold special hardware keys during the boot process). In Recovery mode (best with a ClockWorkMod version) you can apply updates (zip format) to the system.

This zip file holds the su binary but also an updater-script. This script takes care of mounting the /system partition writable and put the su binary in the /system/xbin folder.

Example updater-script:

ui_print("Install root for Gingerbread..");

run_program("/sbin/busybox", "mount", "/system");

show_progress(0.100000, 0);

show_progress(0.500000, 0);

run_program("/sbin/mount", "/system");

package_extract_dir("system", "/system");

set_perm(0, 0, 04755, "/system/xbin/sqlite3");

set_perm(0, 0, 04755, "/system/xbin/su");

set_perm(0, 0, 04755, "/system/xbin/busybox");

run_program("/sbin/busybox", "umount", "/system");

ui_print("Installation complete!");

Some devices are "pre-rooted". adb is running in non-secure mode. That means you have direct root access when you adb into it. I don't like that situation. Better to use secure adb and use the su binary to get root access from an adb shell.

These devices have ro.secure=0 or ro.adb.secure=0 in the kernel /default.prop file. (perhaps also service.adb.root=1)

Note: rooting has nothing to do with the / folder, also called root. The / folder is the kernel from the boot partition. You cannot change that on the device. Then you must change a part of the boot.img flash file. That goes to deep for now.

Apps (apk)

If you want to install an app on your device you login on your device with your google account once and then install an app from the play store. This process sends the apk to your device and installs it in the /data/app folder. Lets take a closer look to what an apk is made of.

If you look into a apk file with zip your see this

assets (folder) - app extra files

lib (folder) - private lib files

META-INF (folder) - certificate

res (folder) - extra screen resources: images, text etc

AndroidManifest.xml

classes.dex - Java VM

resources.arsc - resources

When downloading from the Play Store an app is first stored in the /sdcard/Android/data/com.android.vending/files folder.

Large files then will be stored in /sdcard/Android/obb/ or /data/media/obb/as obb files (7zip to unpack). Then they are installed in the /data/app folder.

The Java VM

The classes.dex is the Java VM bytecode part, it will get prepared and optimized at boot time and stored into the /data/dalvik-cache.

A system apk sometimes lacks the classes.dex file in the apk. Then you will find a prepared or optimized dex file called .odex file in the folder of the apk (/system/app) with the same name. This has the advantage of not consuming too much flash memory from the user's /data partition. It is already optimized by the device manufacturer.

With "deodexing" you put the odex file back into the apk as dex file and remove the prepared odex files from the /system/app folder.

Although deodexing will consume more storage on the /data partition it makes it possible to prepare the classes.dex file with better dalvik runtime options before they are written into the dalvik-cache. You can set the optimization flags your build.prop file.

Example for multicore CPU:

dalvik.vm.dexopt-flags=v=n,o=y,u=n,m=y

dalvik.vm.execution-mode=int:jit

When an app is installed the following folders and files are important:

/data/app/ is where the apk is installed

/data/data/<app name> is where the settings db (sqlite), libs (.so binaries) and user preferences (xml) are stored

/data/dalvik-cache/ is where the java dex (precompiled) part of app's is stored

/data/system/packages.xml is where the package manager stores the app info

Two types of apps: user and system

The user and system apk are basically the same. Except for a few things:

• system app's are stored in /system/app

• system app's are considered safe and thus don't need to have a signed certificate (no META-INF folder)

• system app's classes.dex can be prepared (dex optimized) and so won't need space in the /data/dalvik-cache

  • system app's libraries cannot be in the apk lib folder, they need to be in the /system/lib folder

  • system app's are not meant to be removed (need root access for that and: mount -o remount,rw rfs /system)

  • system app's can be upgraded, but are then stored like an user app, so updates can be removed

• user app's are stored in /data/app

• user app's always are signed

• user app's come from Google play, installed with adb or with the pm command from the android shell

• user app's libraries are stored in the /data/data/<app name> folder

• user app's can be removed

• user app's can be upgraded but upgrade cannot be removed (apk is overwritten by update)

pm is the internal package manager (apk's). When you have access to adb you can you can use it like this.

pm set-install-location 2 (external/sdcard)

pm set-install-location 1 (internal/data)

pm set-install-location 0 (auto)

pm get-install-location

How Apps run on Android? (advanced) more info

Zygote is the first Android VM process to start. It will for example have all core libraries linked. All other Android processes will fork from this. The big advantage is that libraries only have to be loaded once which saves RAM memory.