ExperimentS

Read

How to run wild experiments just like (Google) X, Peter Diamandis on SingularityHub.com. I have posted excerpts from this essay that I found particularly valuable below.

"Sometimes shifting your perspective is more powerful than being smart."

• Astro Teller,

The Three Principles of a Good Experiment

Astro explains that the following three principles describe a good experiment:

Principle 1: Any experiment where you already know the outcome is a BAD experiment.

Principle 2: Any experiment when the outcome will not change what you are doing is also a BAD experiment.

Principle 3: Everything else (especially where the input and output is quantifiable (i.e., measurable)) is a GOOD experiment.

Seems simple enough, right?

You need to be asking questions to which you don’t know the answer but such that if you did know the answer, you’d change the way you operate.

If you already know the answer, or if you are testing an insignificant detail that doesn’t matter, you’ll just be wasting time and money.

How to Ask Good Questions and Design Good Experiments

In any organization, you get what you incentivize.

In order to get good questions/experiments, you have to create a culture that incentivizes asking good questions and designing good experiments.

Astro describes a very unique approach to doing just this:

“At X, we set up a ‘Get Weirder Award.’ The whole point of the Get Weirder Award was to focus the team on experiments and to drive home why they needed to think in terms of experiments.”

Teams would be challenged to ask “weird” questions — to put forth crazy ideas around framing problems differently and to design experiments that really push the limits.

Critically, Astro gives out the Get Weirder Award before the experiments are run.

“If you give out the award before you’ve run the experiment, then people start to really feel that you don’t actually care about the outcome. You care about the quality of the question. So every two weeks, we would give out an award for the best experiment.”

Doing so constantly (and viscerally) reinforced the behavior of asking good questions — and as such, at X, they’ve built a culture around celebrating the questions themselves.

Google Book: Experimentation Matters: Unlocking the potential of new technologies for innovation.