Free for All! Assessing User Data Exposure to Advertising Libraries on Android



Authors: Soteris Demetriou, Whitney Merrill, Wei Yang, Aston Zhang, Carl A. Gunter

Affiliation: University of Illinois at Urbana-Champaign

Description

(This work was presented at ISOC NDSS, February 16', San Diego, CA.)

Pluto is a framework that can be leveraged to analyze an app and discover whether it exposes targeted user data—such as contact information, interests, demographics, medical conditions and so on—-to an opportunistic ad library. In this work present a prototype implementation of Pluto, that embodies novel strategies for using natural language processing to illustrate what targeted data can potentially be learned from an ad network using files and user inputs. Pluto also leverages machine learning and data mining models to reveal what advertising networks can learn from the list of installed apps. We validate Pluto with a collection of apps for which we have determined ground truth about targeted data they may reveal, together with a data set derived from a survey we conducted that gives ground truth for targeted data and corresponding lists of installed apps for about 300 users. We use these to show that Pluto, and hence also opportunistic ad networks, can achieve 75% recall and 80% precision for selected targeted data coming from app files and inputs, and even better results for certain targeted data based on the list of installed apps. Pluto is the first tool that estimates the risk associated with integrating advertising in apps based on the four available channels and arbitrary sets of targeted data. 

  • Link to paper: pdf

Datasets

Name Number Description
Full Dataset (FD) 2535 Unique apps collected from the 27 Google Play categories.
Level One Dataset (L1) 262 Apps randomly selected from FD.
Level Two Dataset (L2) 35 Apps purposively selected from L1.
App Bundle Dataset (ABD) 243 App bundles collected through survey.

Inspection methodologies

  • L1-I: Level-1 Inspection. Looks into files the apps generate and store in their isolated storage space and the permissions the app requests.
  • L2-I: Level-2 Inspection. Looks into files the apps generate and store in their isolated storage space, the permissions the app requests and the User Interface layout.

Implementation

  • Pluto DAM module is implemented in bash.
  • The rest of Pluto's modules are implemented in Java.
  • Source-code: (github link)



Modeling the Libraries' Access Capabilities

Libraries can access user data through four major channels:
  • Host app runtime generated files
  • Protected APIs
  • User input
  • Unprotected APIs



    Manual Analysis: In-app Exposure

    • Apps manually Inspected
     Attack Strategy   Category   Application
    Name 
     #Installations
    (M= million)
    Exposed Data Points 
     

    L1-I 

     

    MEDICAL 

     

    Menstrual Calendar 

     

    1M - 5M 

     

    pregnancy, trimester, headache 

     

    L1-I

     

    EDUCATION 

     

    myHomework Student Planner 

     

    1M - 5M 

     

    gender, age, address 

     

    L2-I 

     

    HEALTH & FITNESS 

     

    Run with Map My Run 


    5M - 10M

     

    phone, email, first name, last name, age, gender, address, workout 

     

    L2-I 

     

    LIFESTYLE 

     

    BeNaughty - Online Dating App & Call 


    5M - 10M

     

    phone, email, age, gender, address, marital status, parent 


    • Number of data points exposed (determined through L1-I inspection on the L1 dataset):
    No. of data points exposed (L1 Manual Analysis)
    • Number of data points exposed (determined through L2-I inspection on the L2 dataset):
    No. of data points exposed (L2 Manual Analysis)
    • Data Points Exposed Per Category:
    Exposure per category (manual analysis)


    Manual Analysis: Out-app Exposure

    User Survey

    We have conducted a user survey launched on Microworkers. We used questionnaires to collect user attributes from 300 people. The users were also asked to download and run an app that anonymously collects the installed apps the participants had on their Android phones. We used those collected apps and their associated user attributes as ground truth to evaluate the performance of Pluto's out-app exposure discovery.

    Survey app: 

    • Screenshots:
     
     

    https://play.google.com/store/apps/details?id=edu.illinois.seclab.appsurvey



    Pluto In-app

    Design

    Design: Pluto In-app Discovery

    Evaluation

    A. CDF of number of data points found exposed per app % (L1, L2)

     
     

    B. Attribute Discovery

     
     
     

     
     


    C. Interest Discovery:
    •    droidLESK: Pluto's Context Disambiguation Layer uses droidLESK to decide whether a found data point should be accepted. droidLesk is designed to find the similarity between a data point (domain knowledge) and a Google Play category (disambiguation term). If these two are similar enough the match is accepted.
      • freq(word, category): It denotes how many times (absolute number) the word 'word' was found present in runtime generated files of apps of the 'category' Google Play Category.
    • DroidLESK similarity scores between the two most prevalent user interests and all Google Play Categories:
    C freq
    (workout)
    freq
    (vehicle)
    droidLESK
    (workout,C)
    droidLESK
    (vehicle,C)
    BOOK,REFERENCE 7 3 3.53E-03 1.70E-03
    BUSINESS 10 121 4.20E-03 5.35E-02
    COMICS 17 3 3.49E-03 7.05E-04
    COMMUNICATION 0 0 0.00E+00 0.00E+00
    EDUCATION 19 184 6.70E-03 7.34E-02
    ENTERTAINMENT 7 3 1.50E-03 6.55E-04
    FINANCE 103 1841 2.48E-02 4.32E-01
    GAME 1 0 3.03E-04 0.00E+00
    HEALTH AND FITNESS 6790 37 1.00E+00 9.30E-03
    LIBRARIES AND DEMO 0 0 0.00E+00 0.00E+00
    LIFESTYLE 4 782 5.53E-04 1.58E-01
    MEDIA AND VIDEO 4 0 6.43E-04 0.00E+00
    MEDICAL 432 27 7.33E-02 5.46E-03
    MUSIC AND AUDIO 3 5 1.29E-03 1.42E-03
    NEWS AND AMAGAZINES 441 152 1.34E-01 7.39E-02
    PERSONALIZATION 7 0 0.00E+00 0.00E+00
    PHOTOGRAPHY 0 0 0.00E+00 0.00E+00
    PRODUCTIVITY 3 0 3.21E-04 0.00E+00
    SHOPPING 98 234 5.25E-03 2.17E-02
    SOCIAL 102 4 2.32E-02 1.22E-03
    SPORTS 18 0 7.55E-03 0.00E+00
    TOOLS 13 0 4.93E-03 0.00E+00
    TRANSPORTATION 3 1387 1.03E-03 1.00E+00
    TRAVEL_AND_LOCAL 27 143 1.30E-02 8.04E-02
    WALLPAPER 9 0 6.02E-04 0.00E+00
    WEATHER 5 9 1.38E-03 2.85E-03
    WIDGET 863 16 1.23E-01 4.28E-03
    • Here we show with which Google Play categories the target words are found to be most similar with, using establish similarity metrics and our droidLESK. The results show that our metric is more suitable when applied to the context of Android.

    RANK WUP(w,c) JCN(w,c) LCH(w,c) LIN(w,c) RES(w,c) PATH(w,c)
    w=VEHICLE 1 TRAVEL AND LOCAL MEDIA AND VIDEO *TRAVEL AND LOCAL MEDIA AND VIDEO MEDIA AND VIDEO *APP WIDGET
    w=VEHICLE 2 *TOOLS NEWS AND MAGAZINES *WIDGET TRAVEL AND LOCAL NEWS AND MAGAZINES *TRAVEL AND LOCAL
    w=VEHICLE 3 *APP WIDGET TRAVEL AND LOCAL TOOL TOOL TRAVEL AND LOCAL TOOLS
    w=WORKOUT 1 *BUSINESS SPORTS *BUSINESS BUSINESS *MEDICAL *BUSINESS
    w=WORKOUT 2 *EDUCATION EDUCATION *EDUCATION MEDICAL *BUSINESS *EDUCATION
    w=WORKOUT 3 *GAME ENTERTAINMENT *GAME SPORTS *GAME *GAME

    RANK LESK(w,c) HSO(w,c) freq(w) droidLESK(w,c)
    w=VEHICLE 1 TRANSPORTATION *TRANSPORTATION FINANCE TRANSPORTATION
    w=VEHICLE 2 BOOKS AND REFERENCES *TRAVEL AND LOCAL TRANSPORTATION FINANCE
    w=VEHICLE 3 TRAVEL AND LOCAL *APP WIDGET LIFESTYLE LIFESTYLE
    w=WORKOUT 1 BOOKS AND REFERENCES *BUSINESS HEALTH AND FITNESS HEALTH AND FITNESS
    w=WORKOUT 2 TRAVEL AND LOCAL *EDUCATION APP WIDGET NEWS AND MAGAZINE
    w=WORKOUT 3 MUSIC AND AUDIO *GAME NEWS AND MAGAZINE APP WIDGET
    (*) Tie

    • Pluto's in-app interest discovery 
     
     


    Pluto Out-app

    Design

              Design: Pluto Out-app

    Evaluation

    • Evaluation (after dimensionality reduction and balancing):

    Age Marital_Status Sex
    Precision Recall Precision Recall Precision Recall
    RandomForest 0.886 0.866 0.95 0.938 0.938 0.929
    SVM 0.448 0.354 0.669 0.505 0.809 0.701
    KNN 0.857 0.836 0.925 0.912 0.916 0.899

    Allergies Backpain Children
    Precision Recall Precision Recall Precision Recall
    RandomForest 0.91 0.909 0.95 0.944 0.937 0.928
    SVM 0.602 0.602 0.633 0.623 0.576 0.576
    KNN 0.884 0.878 0.92 0.904 0.9 0.873

    Country Education Income
    Precision Recall Precision Recall Precision Recall
    RandomForest 0.979 0.976 0.834 0.815 0.944 0.927
    SVM 0.788 0.724 0.465 0.406 0.754 0.719
    KNN 0.975 0.972 0.81 0.804 0.925 0.907

    LoyatyCard Search_Auto Search_Financial
    Precision Recall Precision Recall Precision Recall
    RandomForest 0.868 0.866 0.822 0.814 0.905 0.902
    SVM 0.551 0.402 0.676 0.617 0.701 0.643
    KNN 0.806 0.804 0.802 0.788 0.875 0.865

    Search_Fitness
    Precision Recall
    RandomForest 0.953 0.948
    SVM 0.652 0.611
    KNN 0.908 0.885


    SAMPLE RANKING


    • Risk Score [ 0 - 1 ]: 
      • α: app under analysis
      • D: set of data points
      • X: set of data point values
      • n: the number of data points in the cost model (e.g. Financial Times calculator values)
      • |D|=|X|=n

    • Top 10 in the MEDICAL category:
    Package Name Exposed Data Points by In-app Pluto RISK SCORE
    com.excelatlife.depression phone;occupation;weight;diabetes;obesity;
    parent;disease;education;depression;email;
    address;headache;income;pregnancy;age;gender
    0.814166497
    com.medicaljoyworks.prognosis phone;occupation;weight;headache;address;email;
    pregnancy;age;diabetes;obesity;parent;disease
    0.6314636581
    com.medicaljoyworks.prognosis.emergency occupation;phone;weight;headache;address;email;
    pregnancy;parent;disease
    0.3792174309
    com.mysugr.android.companion phone;occupation;weight;headache;email;address;
    diabetes;parent;gender;education
    0.3409322206
    com.mydiabetes occupation;phone;weight;email;age;diabetes;obesity;
    parent;gender
    0.3406899092
    com.epocrates phone;occupation;weight;first name;last name;parent;disease;education;address;email;age;
    pregnancy;gender
    0.2539424074
    com.diogines.pregnancy occupation;weight;age;pregnancy;parent;gender;
    education;disease;trimester
    0.253700096
    com.smsrobot.period occupation;phone;weight;headache;address;email;
    pregnancy;parent
    0.253215473
    com.medscape.android occupation;phone;weight;address;email;parent;
    disease;education
    0.2146879513
    com.glassesoff.android occupation;phone;weight;address;email;diabetes;
    age;parent
    0.2146879513
    com.gexperts.ontrack occupation;phone;weight;address;email;diabetes;
    parent
    0.2144456398

    • Top 10 in the Health & Fitness category:
    Package Name Exposed Data Points by In-app Pluto RISK SCORE
    com.workoutroutines.greatbodylite phone;occupation;weight;diabetes;obesity;parent;disease;
    education;depression;headache;age;gender;workout
    0.7334767817
    com.cigna.mobile.mycigna phone;occupation;weight;diabetes;parent;disease;education;
    address;email;headache;income;pregnancy;age;gender
    0.5621625813
    com.noom.walk occupation;phone;weight;address;email;diabetes;parent;
    disease;depression;workout
    0.480988243
    com.glow.android occupation;phone;weight;address;email;diabetes;pregnancy;
    parent;gender;disease;workout
    0.3939984298
    com.sleekbit.ovuview phone;occupation;weight;headache;email;age;pregnancy;
    parent;gender;trimester
    0.2534577845
    com.jawbone.up occupation;phone;weight;email;address;parent;gender;
    education;disease;workout
    0.2294689502
    com.uhg.mobile.health4me occupation;phone;weight;email;address;diabetes;age;parent;
    gender;workout
    0.2294689502
    com.getsomeheadspace.android occupation;phone;weight;address;email;parent;depression;
    workout
    0.2289843273
    com.dailyburn.challenge occupation;phone;weight;address;email;parent;disease;workout 0.2289843273
    com.healthagen.iTriage occupation;phone;weight;email;address;age;parent;gender;
    education;disease
    0.2151725742