Frugal Following: Power Thrifty Object Detection and Tracking for Mobile Augmented Reality

Kittipat Apicharttrisorn, Xukan Ran, Jiasi Chen, Srikanth V. Krishnamurthy, Amit K. Roy-Chowdhury

University of California, Riverside

MARLIN Applications for Android

This is a website for a best paper finalist at ACM SenSys 2019 MARLIN-AR. This site provides apk files for users to download and install on Android phones so that they can experience MARLIN's live experiments themselves. There are two packaged apk files to compare energy consumption - default algorithm and MARLIN, as explained in the paper.

Note

This application is intended to use as a comparison between MARLIN and Default-DNN methods, not as individual. For example, users should look at relative detection and tracking accuracy, relative power consumption, and relative phone tempurature.
The application is tested by Google Pixel 2 and LG G6 smartphones, so users with older phone models may expect CPU heating and possibly CPU throttling because of high-computation demands of deep neural networks.
The application achieves the frame rates (FPS) close to the camera frame rates and the tested phone is Google Pixel 2 which has a 30-FPS rear camera.

MARLIN Architecture

Requirements

Android phone
- Mininum Android version: Android 5.0 (Lollipop)
- Target Android version: Android 6.0 (Marshmallow)
- Tested phones: Google Pixel 2 (Android 8.0) and LG G6 (Android 7.0)
Android Debug Bridge (adb) installed on a local machine (https://developer.android.com/studio/command-line/adb)

Files

Download files here

marlin.apk (Android applicaion of the proposed system)
default.apk (Default DNN using Tiny YOLO)
cdmodel.dat (change detector trained model used in MARLIN)

Energy Measurement Tools

Install Qualcomm Trepn Power Profiler on the phone (https://developer.qualcomm.com/software/trepn-power-profiler/installation)
- Click Settings -> Data Points and choose Battery Power as a data point
- Go back to Main page and click Profile App then choose TF Detect
In case Trepn cannot read battery power (output all zeros), e.g. on Google Pixel2, you need to resort to rooting the phone and directly reading Android's virtual files /sys/class/power supply/battery/current_now and /sys/class/power supply/battery/voltage_now as mentioned in the paper.

Instructions

1.Go to Android's shell and create tensorflow folder

$ adb shell

$ cd ./sdcard/

$ mkdir tensorflow

2. Copy cdmodel.dat to the tensorflow folder from the download location of local machine

$ adb push cdmodel.dat /sdcard/tensorflow/

3. Install Android application (MARLIN or default) by

$ adb install -r xxxx.apk

After installation, make sure that you give "TensorFlow Demo" app both the Camera and Storage permissions.

4. Start energy measurement tool (e.g. TREPN)

5, Open "TF Detect" application on Android

6. Push "Volumn Down" to see available debug information

Change Detection latency in ms (MARLIN only)
Object Tracker latency in ms
DNN Inference latency in ms
DNN Inference count (accumulated counts of DNN invocations)
Track_Status (MARLIN only)
FPS (estimated frame per second)

Screenshot

Debug info

Google Sites

Report abuse