To evaluate the effectiveness of Mystique and the defense capabilities of AMTs, we generate multiple sets of malicious apps for different evaluation targets with Mystique. The malware is grouped based on its attack targets, and covers multiple attack and evasion features. On the other hand, we use the malware to test the defense capabilities of AMTs, especially, the state-of-the-art public AMTs introduced. The evaluation subjects are described in the following two aspects.
Offence: to evaluate the strength of the malware generated using Mystique. Each malware sample has at least one attack target, which is listed in Section 4.1. We give feature labels for malware to assess the attack capabilities. All the features used in Mystique feature model are manually summarized from the 1,260 malware samples in Genome. Totally, we have 266 attack features and 14 evasion features in our feature model. We sketch a diagram of the cumulative distribution for each kind of AF defined in Genome. Since EFs are difficult to be categorized from the code, we do not show the distribution of EFs.
Defense: to evaluate the four types of tools (machine learning, static analysis, dynamic analysis, anti-virus tools) to cover a complete protection from three aspects: untrusted app analysis, install-time checking, and continuous runtime monitoring. We need an initialization for machine learning and dynamic analysis tools. For machine learning tools, we select all 1,260 malware samples in Genome, and 1,260 benign apps from Google Play as their training set. For dynamic analysis tools, we implement a driver in Python to simulate all possible triggers in our scope, e.g., starting an app, receiving an SMS message, changing the geography location. And the script to simulate all triggers is as follows:
All Triggers in to be simulated
Startup. It contains two steps as follows: 1) install the app into the device; 2) start it with the main activity.
adb install [PACKAGE_NAME]
adb shell am start -n [PACKAGE_NAME]/[MAIN_ACTIVITY]
Broadcast messages. To issue a broadcast message, Mystique run the following command:
adb shell am broadcast -a [BROADCAST_MESSAGE]
And the [BROADCAST_MESSAGE] contains:
android.intent.action.BATTERY_CHANGED
android.intent.action.BATTERY_LOW
android.intent.action.BATTERY_OKAY
android.intent.action.DATA_SMS_RECEIVED
android.intent.action.DATE_CHANGED
android.intent.action.DEVICE_STORAGE_LOW
android.intent.action.DEVICE_STORAGE_OK
android.intent.action.INPUT_METHOD_CHANGED
android.intent.action.LOCALE_CHANGED
android.intent.action.MY_PACKAGE_REPLACED
android.intent.action.NEW_OUTGOING_CALL
android.intent.action.PACKAGE_ADDED
android.intent.action.PACKAGE_CHANGED
android.intent.action.PACKAGE_DATA_CLEARED
android.intent.action.PACKAGE_FIRST_LAUNCH
android.intent.action.PACKAGE_FULLY_REMOVED
android.intent.action.PACKAGE_INSTALL
android.intent.action.PACKAGE_NEEDS_VERIFICATION
android.intent.action.PACKAGE_REMOVED
android.intent.action.PACKAGE_REPLACED
android.intent.action.PACKAGE_RESTARTED
android.net.wifi.p2p.DISCOVERY_STATE_CHANGE
android.net.wifi.p2p.PEERS_CHANGED
android.net.wifi.p2p.STATE_CHANGED
android.net.wifi.p2p.THIS_DEVICE_CHANGED
android.net.wifi.supplicant.CONNECTION_CHANGE
android.net.wifi.supplicant.STATE_CHANGE
android.provider.Telephony.SIM_FULL
android.provider.Telephony.SMS_CB_RECEIVED
android.provider.Telephony.SMS_EMERGENCY_CB_RECEIVED
android.provider.Telephony.SMS_RECEIVED
android.provider.Telephony.SMS_REJECTED
android.provider.Telephony.SMS_SERVICE_CATEGORY_PROGRAM_DATA_RECEIVED
android.provider.Telephony.WAP_PUSH_RECEIVED
Listeners. We consider two types of listeners in our triggers: listeners for phone state and geography location. So we provide the following commands to trigger such kinds of listeners.
telnet [HOST] [PORT]
gsm call [PHONE_NUMBER]
geo fix [LATITUDE] [LONGITUDE]
quit
Content Provider. Some malicious behaviors are triggered by the change of a Content Provider. We use the following commands to simulate such kind of the change.
adb shell am start -a [ACTION_TYPE] -t [CONTENT_PROVIDER]
And there are basically three types of actions in Android: android.intent.action.INSERT, android.intent.action.EDIT, and android.intent.action.DELETE.
[CONTENT_PROVIDER] contains:
content://sms
content://mms
content://mms-sms
content://settings
content://downloads/all_downloads
content://com.android.contacts
content://com.android.contacts
content://downloads/download
content://com.android.calendar
content://contacts
content://call_log
content://browser