The Chromium Projects
 
Search this site

Except as otherwise noted, the content of this page is licensed under a Creative Commons Attribution 2.5 license, and examples are licensed under the BSD License.

The Chromium OS designs and code are preliminary. Expect them to evolve.
For Developers‎ > ‎

Testing

Child pages

Overview

Chromium development places a high premium on tests, tests, and more tests. In order to maintain a rapid rate of development across multiple platforms and an ever increasing set of features, it is imperative that test suites be updated, maintained, executed, and evolved. The Chromium Buildbots are employed to run these tests 24x7.

Expectations

Developers contributing code are expected to always run all tests. Given the volume of tests combined with the multitude of platforms, this can be a daunting task. But fear not, your Chromium teammates will help you too. The goal is to keep the source tree always building so that regressions are minimized and developer productivity is maximized. When contributing code, always consider whether your change has enough testing. If it is a new feature or module, it should almost certainly be accompanied by tests.

After committing a change, you should use the Chromium Buildbot to monitor that the automated tests pass with your change. Please be reachable by email or IRC after check-in in case something unexpected happens.

Inventory of Existing Tests

Here is a list of some of the tests currently used by Chromium:
  • automated_ui_tests
  • base_unittests - Test the base module
  • installer_unittests - Test the Chromium installer
  • interactive_ui_tests - Tests some of the interactive UI elements, such as the find-in-page feature and tab dragging.
  • ipc_tests - Tests the IPC subsystem for communication between browser, renderer, and plugin processes.
  • mini_installer_test - Tests the mini installer.
  • net_perftests - Performance tests for the disk cache and cookie storage.
  • net_unittests - Tests the network module.
  • npapi_layout_test_plugin - A NPAPI plugin used with the layout tests.
  • npapi_test_plugin - A NPAPI plugin used with the plugin_tests and ui_tests.
  • perf_tests - Tests performance for some submodules - JSON, Safe Browsing, URL parsing, etc.
  • plugin_tests - Tests the plugin subsystem.
  • reliability_tests - Tests to verify Chromium recovery after hanging or crashing of renderers.
  • security_tests - A set of security tests for Chromium.
  • selenium_tests - A driver for running selenium tests.
  • startup_tests - Test startup performance of Chromium.
  • tab_switching_test - Test tab switching functionality.
  • test_chrome_plugin
  • test_shell - Test Shell is a standalone application for running the renderer (WebKit).
  • test_shell_tests - A collection of tests within the Test Shell.
  • ui_tests - The kitchen sink for UI tests. UI tests are tests which launch Chromium and control it through Automation.
  • unit_tests - The kitchen sink for unit tests. These tests cover several modules within Chromium.

Test Development Infrastructure

To assist with building tests, several pieces of infrastructure exist.  A truism with testing is that more is always needed.

Here are some tools you might find useful:
  • chrome/test/automation - Chromium includes a mechanism for driving the browser through automation.  This is primarily used with the UI tests.
  • gtest - GTest is Chromium's C++ test harness.
  • image_diff - A mechanism for comparing bitmaps.

Running basic tests

Windows

  1. In Visual Studio, press F7 (rather than F5) to build the entire solution
  2. In a cmd window, run the test of interest in src\chrome\Debug, e.g. src\chrome\Debug\base_unittests.exe

Linux

  1. cd to src/chrome
  2. build everything with hammer -j5
  3. run the test of interest from the Hammer subdirectory, e.g. Hammer/base_unittests

Selecting a particular subtest

The above test executables are built with gtest, so they accept command line arguments to filter which sub-tests to run.  For instance, base_unittests --gtest_filter=FileUtilTests.*

SSL tests

On Windows, the SSL tests (both in net_unittests.exe and ui_tests.exe) require you to import the test root CA certificate to your trusted root CA store. (On Linux and Mac OS X this is not needed, as the certificate is temporarily imported by the test itself. If you know how to import a root CA certificate temporarily on Windows, please help us fix issue 8470.)

To do this on Windows XP:
  1. double-click the test root CA certificate file (net/data/ssl/certificates/root_ca_cert.crt in the source tree), this opens the Certificate Windows dialog
  2. click Install Certificate
  3. click Next
  4. select the option 'Place all certificate in the following store'
  5. click Browse.., select 'Trusted Root Certificate Authorities', click OK
  6. click Next and Finish
To do this on Windows Vista:
  1. in the Start menu, type "cmd" in the Start Search box (Note: note the Run... box) and then type ctrl-shift-Enter to run the Command Prompt as Administrator
  2. in the Command Prompt, cd into net\data\ssl\certificates in the source tree, and run the root_ca_cert.crt file
  3. in the "Open File - Security Warning" dialog, click OK
  4. in the Certificate dialog, click Install Certificate
  5. In the Certificate Import Wizard, click Next
  6. select the option "Place all certificates in the following store"
  7. click Browse.., check the "Shown physical stores" box, expand "Trusted Root Certification Authorities", select "Local Computer", click OK
  8. verify that the "Certificate store" field says "Trusted Root Certification Authorities\Local Computer"
  9. click Next and Finish

Layout tests

WebKit has a large suite of tests that typically verify a page is laid out properly.  We use them to verify much of the code that runs within a Chromium renderer.
To run these tests, build test_shell, change into the webkit directory, and run ./tools/layout_tests/run_webkit_tests.sh --debug .

More information about running layout tests or fixing layouts tests can be found on the WebKit Layout Tests page.

Inducing a crash

This can be useful to test breakpad:
  • about:crash - will cause a renderer crash.
  • about:inducebrowsercrashforrealz - will cause a browser crash.
Get dumps for Chromium crashes on Windows

Before running the tests, make sure to run crash_service.exe. We use this program to intercept crashes in chromium and write a crash dump in the Crash Reports folder under the User Data directory of your chromium profile.
If you also want crash_service.exe to intercept crashes for your normal Google Chrome or Chromium build, add the flag --noerrdialogs.
You can also use the flag --enable-dcheck to get assertion errors in release mode.

Subpages (2): Chromium build infrastructure WebKit Layout Tests