Skip to content

Hypotheses and Experiments

What do we believe, and how could we find out?

Series identity. Learn in cycles. Move in a line. The product keeps moving forward, while understanding improves through repeated PDSA learning loops.

Use this guide to: turn assumptions into testable predictions and small experiments.

PDSA emphasis: Plan to Do, with prediction as the bridge.

Purpose

This guide helps teams move from vague confidence to explicit hypotheses. A hypothesis states what the team believes and why. An experiment creates contact with reality without pretending the team has to build the whole solution first.

When To Use It

  • When the team has competing beliefs about user behavior or system performance.
  • When a large build can be reduced to a smaller learning test.
  • When leadership asks for certainty that the team cannot honestly have yet.

Core Questions

  • What do we believe is true?
  • Why do we believe it?
  • What do we predict will happen if we try this?
  • What is the smallest useful test?
  • What evidence would support, weaken, or change the hypothesis?

Facilitation Pattern

  1. Name the belief as a falsifiable sentence.
  2. Write the prediction before designing the test.
  3. Decide whether the test needs a prototype, concierge workflow, data pull, interview, or production slice.
  4. Define the smallest audience and safest scope.
  5. Agree how the result will be studied and what decision it will inform.

Working Example

The team believes new users abandon setup because the integration steps are confusing. The prediction: if users see three setup options with plain-language consequences, more will choose a path within two minutes. The first experiment is a clickable prototype and five moderated sessions, followed by a small production test if the pattern holds.

Common Traps

  • Calling a task an experiment when no prediction was made.
  • Testing too many variables at once.
  • Designing an experiment that cannot change a decision.
  • Using metrics that are easy to collect but unrelated to the hypothesis.

Outputs And Artifacts

  • Hypothesis statement.
  • Prediction.
  • Experiment design.
  • Evidence and measurement plan.
  • Decision rule or interpretation guide.

PDSA Linkage

Prediction is the heart of this guide. Deming's emphasis on theory means a team should be able to say what it expects and why before it studies what actually happened.

Working Worksheet

  • Write the hypothesis as a belief, not a feature.
  • Write the prediction before running the test.
  • Limit the test to one dominant uncertainty.
  • Choose the smallest useful exposure.
  • Decide how the evidence will be read.

Lineage Notes

Direct lineage: Walter Shewhart's statistical view of process learning influenced W. Edwards Deming, and Deming made PDSA a disciplined loop for prediction, action, study, and adjustment. Complementary quality thinkers - Juran, Ishikawa, Feigenbaum, and Taguchi - sharpen the product team's attention to fitness for use, causes, total systems, and variation. Ackoff, Drucker, Ries, and Blank are useful adjacent thinkers for systems, management, and product discovery, but they are not presented here as the historical source of PDSA.

References