Chromium uses an open-source system called Buildbot to run continuous builds and tests. The Chromium Buildbot waterfall shows the status of those tests. That waterfall shows a lot of information that can be hard to understand at first, so let's take a quick tour.

Buildbot is made up of a set of build and test machines, called "slaves", plus a single master to coordinate it all. The master is the one watching the Subversion repository, telling slaves to start building and testing, and constructing the "waterfall" page that shows all the results. (For network reasons, Chromium actually has a separate machine showing the waterfall, delayed by up to two minutes. Usually this doesn't matter, but every once in a while it can lead to confusion.)

Let's take a look at the waterfall page. For now, skip over the blue header and the box at the top with lots of links and horizontal stripes (we'll come back to them later), and look down at the row of grey boxes with "changes" at the left. Those boxes show one column per build slave.

The set of slaves changes pretty often, so any list we could put here would go out of date quickly, but the names are generally pretty good at describing what kinds of machines they are and what kinds of tests they're running. A "perf" slave is running performance tests, a "dbg" slave is running a Debug build rather than Release, the "Webkit" slaves are building and testing our WebKit port without the rest of Chromium, and the "modules" slaves are working on the independent base, net, and sandbox code modules. Some machines both build and test, but "builder" machines only build the executables, then upload them to "tests" machines to run the tests.

Steps

Now look down one of the columns, starting at the grey box with the slave's name. Each box shows one step in that slave's build/test sequence, with the oldest one at the bottom and the current or latest one just below the name box.  Just above the name box is the slave's current activity, and above that is the final result of the last build/test sequence that finished.

A yellow build step is still in progress, green finished successfully, red finished with errors, gold finished with warnings, and purple had an internal Buildbot error.

The steps slaves may run are subject to change, especially adding more kinds of tests, but at the moment the choices are as follows:

Step	Script	Description
svnkill	(internal)	Kill any leftover svn processes before starting a new test cycle.
update scripts	(internal)	Update the internal buildbot scripts that describe all the other steps the slave will run.
update	(internal)	Run gclient sync to update the checkout.
taskkill	kill_processes.py	Kill a bunch of other possible leftover processes (test_shell.exe, ui_tests.exe, etc.) that would interfere with a clean run.
check deps	checkdeps.py	Ensure that source dependencies stay clean.
compile	compile.py	Build the executable(s).
archive build	archive_build.py	Save the executables and symbols into "snapshots".
upload build	(internal)	Save a build for use by test slaves.
download build	(internal)	Download a build for testing.
(various tests)	runtest.py, debugger_unittests.py, chrome_tests.py, etc.	(See testing information.)

If you click on the "stdio" link for a step, you can see the exact command that was run, the environment it was run in, and any output it produced.

Changes

Seeing More and Seeing Less

Banner and Box

Now back to the top of the waterfall. At the very top is a banner showing the current state of the tree. If the tree is closed because of build or test failures, it should be mentioned here. If there's an announcement about a new build process, expected downtime, or some other aspect of Chromium development, it'll generally be shown here, too.

Sheriffs

Source