Skip to content

Study the Outcome

What actually happened?

Series identity. Learn in cycles. Move in a line. The product keeps moving forward, while understanding improves through repeated PDSA learning loops.

Use this guide to: compare prediction with evidence and produce learning the team can act on.

PDSA emphasis: Study.

Purpose

Study is the discipline that keeps product learning honest. The team compares what it predicted with what happened, looks for variation and surprises, and separates evidence from narrative. The output is not a grade on whether the team was right; it is a better theory of the product system.

When To Use It

  • After a prototype test, release, operational trial, or research cycle.
  • When results are ambiguous and the team needs to make sense of them.
  • Before deciding to scale, adapt, pause, or abandon a direction.

Core Questions

  • What did we predict?
  • What actually happened?
  • Where did results vary by user, context, time, or workflow?
  • What surprised us?
  • What do we now understand differently?

Facilitation Pattern

  1. Start by rereading the prediction exactly as written.
  2. Review evidence before opinions.
  3. Look for patterns and variation, not only average results.
  4. Ask what the result says about the underlying theory.
  5. Write the learning in language useful to the next decision.

Working Example

The onboarding experiment predicted that clearer setup options would increase first-session completion. Completion improved for small teams but not enterprise teams. Study reveals that enterprise blockers are permissions and procurement, not comprehension. The result is not failure; it splits the problem into two cycles.

Common Traps

  • Moving straight from results to roadmap decisions.
  • Declaring victory from a vanity metric.
  • Explaining away surprises instead of studying them.
  • Averaging away meaningful segment differences.

Outputs And Artifacts

  • Prediction-versus-observation summary.
  • Evidence notes and limits.
  • Surprises and variation.
  • Revised theory of the problem or solution.
  • Candidate actions for Decide and Adapt.

PDSA Linkage

This is the step Deming protected by preferring Study over Check. Study asks what was learned about the theory, not merely whether the plan passed inspection.

Working Worksheet

  • Read the prediction aloud.
  • List evidence before interpretation.
  • Separate signal, noise, and unknowns.
  • Name at least one surprise.
  • Write what changed in the team's understanding.

Lineage Notes

Direct lineage: Walter Shewhart's statistical view of process learning influenced W. Edwards Deming, and Deming made PDSA a disciplined loop for prediction, action, study, and adjustment. Complementary quality thinkers - Juran, Ishikawa, Feigenbaum, and Taguchi - sharpen the product team's attention to fitness for use, causes, total systems, and variation. Ackoff, Drucker, Ries, and Blank are useful adjacent thinkers for systems, management, and product discovery, but they are not presented here as the historical source of PDSA.

References