Findit

(site being deprecated, try the new one https://sites.google.com/chromium.org/cat/findit)

What is Findit?

Findit (https://findit-for-me.appspot.com/) identifies culprits or suspects for compile and test failures on Chromium Waterfall, and regression ranges & culprits for flaky tests on Chromium Waterfall and Commit Queue.

Who are working on it?

Shuotao Gao (TL, stgao@), Chan Li (chanli@), Jeff Li (lijeffrey@), Prasad Vuppalapu (prasadv@), Roberto Carrillo (robertocn@), Yuke Liao (liaoyuke@).

Who can use it?

  • Chromium tree sheriffs (Findit is integrated with Sheriff-o-Matic)
  • Anyone who triages a compile/test failure or a flaky test

What is supported?

  • Compile/test failures:
    • Compile failures, Swarmed gtests and Android Instrumentation tests
    • All build/test configurations (except ios) in the 9 tree-closer masters on Chromium Waterfall
  • Flaky tests:
    • Swarmed gtests and Android Instrumentation tests
    • Any test configuration on Chromium Waterfall and Commit Queue

What is not supported yet?

  • Non-swarmed gtests
  • Telemetry-based tests
  • Webkit layout tests
  • Compile/test failures on non-tree-closer masters

Flow Overview


How it works for compile/test failures?

Findit takes two complementary approaches to identify the culprits or suspects:

  • Heuristic-based analysis: correlate a CL with error messages in the failure log.
    • Instant: 1~2 minutes
    • May yield false positives
    • Requires manual verification for the identified suspects
  • Try-job-based analysis: rerun failed compile or tests and search for the culprit in the regression range.
    • Reliable as verified by rerun of failures.
    • Fast:
      • Compile failures: ¬†median = 14 minutes
      • Swarmed gtest failures: median = 24 minutes

How it works for flaky tests?

Given a flaky test at a specific build cycle
, Findit uses post-submit build artifacts on Chromium Waterfall of the same test configuration to bypass compile, triggers Swarming tasks directly to rerun the flaky test N times (currently up to 400) at different revisions in a variant of exponential search, and then narrows down the regression range into a single build cycle on Chromium Waterfall. Once a regression range is identified with high confidence, a series of try-jobs are run to compile binaries and identify the exact culprit.

    In particular, for a flaky test at a try-bot build on Chromium Commit Queue reported by Chromium-Try-Flakes, Findit maps the CQ try-bot to Waterfall buildbot before the above flow.

    File bug or feature request

    Click this link to file a bug for Findit

    Other project info

    Comments