You'll need a host to run the agent. We create one agent per device. A host can run multiple agents.
Any OS that supports NodeJS and adb should work but Linux and Windows are the only ones that get regular testing. If your host has only USB 3, you will likely need to use Windows (at least as of February 2014, the USB 3 drivers on Linux were very flaky). If your host as USB 2.0 ports, then either Linux or Windows is a good option.
For Android, you will need a phone running KitKat (4.4) or newer that can be rooted (Nexus devices and Motorola G's work well).
We'll assume you're using Linux and a Nexus4.
Prepare the Device
- Root the device (do this first as it will wipe out all of your settings)
- Go into the SuperSU settings (if installed) and disable notifications and set default access to grant
- Install Chrome, Chrome Beta and Chrome Dev from the Play Store (make sure Chrome is updated to the latest release)
- Launch both and accept the terms of service and dismiss any "help make Chrome better" dialogs
- Make sure Google Play is configured to automatically install updates
- Disable swipe to unlock (optional, in security settings)
- Activate developer mode (tap on the OS Version 7 times)
- Enable USB debugging (in developer options)
- Disable screen sleeping (in developer options)
- Disable screen auto-rotate (unless setting device for landscape testing)
- Most distributions have a quick setting to lock screen orientation, otherwise it is likely under the "display" menu as a setting "when device is rotated", select to stay in portrait mode.
- Turn screen brightness down to the minimum (reduces heat and power use)
- Mute volume (optional)
- Put phone in airplane mode and enable WiFi (optional and only if not testing over a cellular connection)
- If on Android 5.0 or later (Lollipop+) disable tab integration with recent apps
- Open each instance of Chrome (stable, beta, dev)
- Open settings
- "Merge tabs and apps" - turn off
iOS (work in progress)
- Jailbreak the device (include Cydia)
- Install OpenSSH
- Create ssh key on tethered host and save as ~/.ssh/id_dsa_ios
- Add ssh public key to ~/.ssh/authorized_keys
- Install tcpdump
- Lock the device rotation (if needed)
- Enable Web Inspector (Settings->Safari->Advanced)
- Useful apps for device management (from cydia):
- "veency" - VNC server for remotely triggering swipe to unlock in case you need to reboot devices
- "Core Utilities" - includes du for inspecting storage use
- "Support Unsupported Accessories 8" - Prevents "this accessory may not be supported" message from intermittently coming up (make sure to enable it in settings)
Prepare the hostRaspberry Pi devices make great hosts for running android devices. A document describing how to configure them is available here.
- Install NodeJS ("node -v" from a shell/command-line to verify)
- Ubuntu: "sudo apt-get install nodejs" and then "sudo ln -s /usr/bin/nodejs /usr/sbin/node"
- Windows: Install the Windows version of node.js and add it to your path (as an install option or manually)
- OSX: "brew install node"
- Install ImageMagick (necessary for screen shot and video processing, "convert -version" from a shell/command-line to verify)
- Ubuntu: "sudo apt-get install imagemagick"
- Windows: Install the Windows version of ImageMagick and add it to your path (as an install option or manually)
- OSX: "brew install imagemagick"
- Install ffmpeg with x264 support (necessary for video processing)
- OSX: "brew install ffmpeg"
- Install python 2.7 and supporting libraries
- Install Pillow library: "pip install pillow"
- Install ujson (for faster trace parsing): "pip install ujson"
- Install the agent code
- Using git directly (assumes in ~/wpt but can be installed anywhere):
git clone http://github.com/WPO-Foundation/webpagetest.git wpt
- Installing manually you need the agent/js directory
- Connect the phone(s)
Android-specific Host Configuration
- Install adb and make sure it is in your path ("adb devices" from a shell/command-line to verify once the devices are connected)
- Ubuntu: "sudo apt-get install android-tools-adb"
- Otherwise, from the Android SDK
- Configure udev rules (Linux only)
iOS-specific Host Configuration (work in progress)
- Requires OSX Yosemite (10.10) or later for video capture
- As of right now, OSX can only capture video from one device. This effectively requires a dedicated Mac for each iPhone that will be used for testing (at least where video capture is required).
- Requires XCode on OSX for Screen Shots
- Copy /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport to wpt/agent/js/lib/ios/DeviceSupport
- Make sure the directory names only contain the OS versions (no build information)
- Install libimobiledevice - 'brew install libimobiledevice'
- Install ios_webkit_debug_proxy - 'brew install ios_webkit_debug_proxy'
Connect and Verify the configuration
- Authorize and verify the adb connection to the device
- "adb devices" - Make sure the device is listed (click the authorization on the device screen when prompted)
- Verify root
- Verify that the network is up
adb shell netcfg | grep wlan
wlan0 UP 22.214.171.124/26 0x00001043 ac:47:e8:4b:3a:81
adb shell ping yahoo.com
adb shell am start \
-n com.android.chrome/com.google.android.apps.chrome.Main \
Configure the locations.ini on the WPT server
Modify the WPT Server settings to add your test location and browser, e.g.:
Test location: Example
Browser: Nexus4 - Chrome
by editing the settings/locations.ini:
browser=Nexus4 - Chrome,Nexus4 - Chrome Beta
To use the Chrome and Chrome beta from the play store, the browser names need to end in "- Chrome" and "- Chrome Beta" respectively.
Start the agent
Start the agent:
-m debug \
--browser android:0088a434deadbeef \
--serverUrl example.com \
--location Example_Nexus4 \
It should print:
node src/agent_main ...
I 0913Z22:06:45.073 wpt_client.js:293 Client.requestNextJob_ : \
Get work: http://example.com/work/getwork.php?location= Example_Nexus4&pc=0088a434deadbeef&f=json
which will poll the above URL every 10 seconds.
To optionally keep the agent running if/when you logout, wrap the above command as:
nohup ... &>0088a434deadbeef.log &
Optionally, specify a specific Chrome package:
Other available command-line options:
--alive <name> // Touch a <name>.alive file while running (for integration with adbwatch.exe to auto-restart dead agents)
--apiKey <key> // Use the specified api key when polling for work--checknet yes // Check for a valid assigned IP address before polling for work
--chromePackage com.android.chrome // Specify an explicit browser package
--maxtemp <temp> // Maximum allowed battery temp - i.e. --maxtemp 36.8
--name <friendly name> // Use the specified friendly device name for reporting instead of the device ID
--processvideo yes // Process video capture locally instead of on the server. Requires python 2.7 in the path and visualmetrics.py in the agent directory (run "python visualmetrics.py -c" to validate config).
--rotate <degrees> // Rotate the screen shots (useful for landscape testing)
--tcpdumpBinary <path> // Path to an arm build of tcpdump to be installed on the agent dynamically
--trafficShaper <info> // See details below in "Traffic Shaping"
--exitTests <test count> // Exits after running the specified number of tests (helpful to keep node's memory use under control)
The agent will save debug info in the results/testId directory.
On your desktop browser, submit a job to http://example.com, e.g.:
Advanced Settings > Script > Enter Script:
driver = new webdriver.Builder().build();
Download a precompiled tcpdump arm binary, e.g. from omappedia.org
Gunzip and copy the binary to ~/wpt/lib/tcpdump
Rerun wptdriver.sh with additional args: --tcpdumpBinary ~/wpt/lib/tcpdump
When you submit jobs, enable tcpdump via:
Advanced Settings > Advanced > Capture network packet trace (tcpdump)
Verify that the agent uploads the pcap.
On-device traffic shaping:
TODO Eg. Android tc (requires on-device /proc/net/psched and /system/bin/tc).
Can likely implement via the below "trafficShaper" script approach.
The easiest configuration is to assume that all traffic passes through the desktop, e.g:
Add a second ethernet card to your desktop.
Attach WiFi access point to this additional card.
Verify that Linux enabled IP forwarding (how-to)
Configure your mobile device to use the WiFi access point.
Verify that your device's WiFi traffic is routed through your desktop (via tcpdump/wireshark?)
If traffic is not routed through your desktop, create a local "trafficShaper" script that (e.g.) uses passwordless ssh to configure the remote switch.
issue/147 adds support for user-defined "trafficShaper" script:
Create a traffic shaper script, e.g. ~/wpt/shaper
Run ./wptdriver.sh with "--trafficShaper ~/wpt/shaper" and optional "--trafficShaperArg anyString" (e.g. foo126.96.36.199).
Modify the WPT Server's settings/locations.ini to remove the "connectivity=" line (to allow the job options).
Submit job with Advanced Settings > Connection set to (e.g.) "56K Dialup-up", which the server's connectivity.ini defines as:
label="56K Dial-Up (49/30 Kbps 120ms RTT)"
Before each run, the agent should call:
~/wpt/shaper -s 0088a434deadbeef start --arg foo188.8.131.52 --bwIn 49 --bwOut 30 --latency 120
where "0" values are not passed (e.g. the above plr=0)
The agent will check the script's stdout for an optional line matching "stop=VALUE" (e.g. "stop=bar42").
Verify that pages load slowly due to your traffic shaper.
After each run, the agent should call:
~/wpt/shaper -s 0088a434deadbeef stop --arg bar42
Install ipfw3, sudo chmod 7755 /sbin/ipfw, verify that ipfw3 list prints "65535 allow ip from any to any"
Get the device IP from `adb shell netcfg | grep wlan" (e.g. 184.108.40.206), run ./wptdriver.sh with "--deviceAddr 220.127.116.11"
Modify the WPT Server's settings/locations.ini to remove the "connectivity=" line.
Submit job with Advanced Settings > Connection set to (e.g.) "56K Dialup-up".
Verify that the agent prints the expected ipfw commands (e.g. `ipfw add ...`, `ipfw pipe ...`)
Verify that pages load slowly due to traffic shaping.
Video capture (non-android)
Create capture script, e.g. ~/wpt/video/capture
Run ./wptdriver.sh with "--captureDir ~/wpt/video" and optional "--videoCard anyString" (e.g. to pass the card name to the script).
~/wpt/video/capture -f pid_video.avi -s 0088a434deadbeef -t mako -d anyString -w
where "mako" is the `adb shell getprop ro.product.device` codename for Nexus4.
When the agent wants to stop the capture, it'll kill the above process, then read the "pid_video.avi".
Primarily used to kill any zombie processes, e.g. leftover video captures.
Create a user for this device, e.g. via:
sudo useradd -d /home/0088a434deadbeef -m 0088a434deadbeef
Switch to this user (optionally via /etc/sudoers.d/wpt file and `sudo -u`)
Run ./wptdriver.sh with "--killall 1"
The agent will `killall -9` all pids owned by its user, except its own pid. This `killall` ensures that there are no zombie processes between jobs, but note that it'll also kill any login shells, so make sure to run with nohup!