Chat with Jurre

Blueprints

Experiment Canvas

One test, one page, one clear lesson.

The Experiment Canvas is the worksheet for a single experiment. Before the test you decide what success means; after the test you book the lessons and choose the next step explicitly. Made for product teams, growth teams and anyone validating ideas who wants to bring discipline to their testing rhythm.

Fill in this canvas in BoldcasterOr start with the scale-up audit →
BoldcasterExperiment Canvas · template

What is the Experiment Canvas?

The Experiment Canvas is based on Eric Ries's Build-Measure-Learn cycle from Lean Startup and on the Strategyzer templates by Alexander Osterwalder and David Bland. It is a detailed log for each experiment, so that experiments become genuinely learnable and shareable.

It solves two stubborn problems. First: experiments that are never documented properly, so the lessons evaporate as soon as the person running them moves on to other work. Second: experiments where it is unclear afterwards what exactly they showed, because no criterion was set in advance. The canvas imposes structure on both fronts.

The Experiment Canvas does not tell you which experiment to design; it structures how you run it. For the wider strategic context you need a Validation Canvas, and to prioritise which assumption to test you need a hypothesis matrix. The Experiment Canvas sits in between: one page per concrete test.

When do you use it?

Reach for the Experiment Canvas as soon as you have a hypothesis you want to test and you know which method you want to use. The canvas helps you impose design discipline: what exactly are you changing, what are you measuring, what counts as enough evidence?

Working on strategic assumptions that span several experiments? Then use a Validation Canvas as the overarching log. Testing something within a product feature? Combine it with a Feature Canvas. Want to test in MVP form? Link it to the MVP Canvas.

How to fill it in

Fill in the canvas in order, and above all do not touch the success criterion once the test has started. Write the result block only after the test has finished; anything you write before that is a prediction.

01

Hypothesis

The testable assumption the experiment should support or refute. Write it as 'We believe that [who] [what] because [why]'. An experiment without a clear hypothesis produces no lessons, only activity. Keep the hypothesis on the canvas word for word, even if you later have to admit it was wrong.

Questions that help

  • ·Which specific assumption are we testing here?
  • ·Who exactly does the assumption apply to?
  • ·What would show that the assumption is wrong?

Example

We believe that visitors to a pricing page with three tiers (instead of two) choose the middle tier more often, because the middle tier now gets framed as 'the best'.

Common mistake

Formulating a hypothesis that is too broad or too narrow; either makes evaluation impossible.

02

What We Test

Describe the intervention itself, concretely and completely. Which version sits next to which? What is the copy, the design, the price, the channel? What does the flow look like for a test participant? Enough detail for someone to reproduce the experiment exactly in six months' time; otherwise it is not an experiment but a one-off observation.

Questions that help

  • ·What exactly changes between the variant and the control?
  • ·What does the user flow look like?
  • ·Which variables do we deliberately keep constant?

Example

Variant A (control): two tiers, Basic at €29 a month and Pro at €79 a month. Variant B: three tiers, Basic at €29, Pro at €79 with a 'Recommended' badge, and Team at €149. The page is otherwise identical.

Common mistake

Varying too much at once (colour, price and copy). Afterwards you do not know which element made the difference.

03

What We Measure

The main metric that judges the hypothesis, plus one or two guardrails that stop a 'win' on the main metric from doing damage elsewhere. Higher conversion on the pricing page is no win if churn doubles three months later. Ideally choose metrics that directly measure what you want to know, not proxies.

Questions that help

  • ·Which metric directly judges whether the hypothesis holds?
  • ·Which metrics must stay the same so nothing gets damaged?
  • ·How do we measure: which tool, which event, which definition?

Example

Main metric: conversion to the Pro tier per page visitor. Guardrails: the 30-day retention rate of new Pro customers, and average support time per new customer.

Common mistake

Choosing a metric that is sensitive to chance with small samples, for example 'number of upgrades to Enterprise' when you only have 100 visitors per variant.

04

Success Criterion

The threshold, fixed in advance, that decides whether the hypothesis is confirmed. Make it statistically defensible: a difference that falls within the noise is not a difference. For A/B tests this is often a minimum effect plus a confidence level (for example 95%). For exploratory tests, a concrete threshold ('at least 25 demo requests in 14 days').

Questions that help

  • ·Which outcome on the main metric do we consider confirmation?
  • ·What is the minimum detectable effect with our sample?
  • ·What is the minimum time the test must run?

Example

Success criterion: Pro tier conversion in variant B is at least 20% higher than in variant A, at p<0.05 over at least 4,000 visitors per variant.

Common mistake

Not setting a threshold, or setting one that you later 'adjust to what turned out to be achievable'.

05

Result

Fill this in only after the test has finished. Write down all the relevant numbers: sample size, duration, main metric per variant, guardrails, and any segment differences that stood out. No interpretation, no 'it worked' or 'shame', only the factual outcome. Add context about how it ran: odd seasonal effects, tool outages, a bug that was fixed halfway through.

Questions that help

  • ·What is the factual outcome for each metric?
  • ·How large were the actual sample and duration?
  • ·Which external factors could have distorted the picture?

Example

Duration 16 days, 4,327 visitors in variant A, 4,298 in variant B. Conversion to Pro: A 7.2%, B 9.8% (+36%, p<0.01). 30-day retention still unknown at the time of reporting; support time identical.

Common mistake

Mixing result and conclusion, or summarising the result 'optimistically' with rounded numbers.

06

Lessons Learned

What this experiment teaches you about your business, your customer or your market, whether or not the hypothesis was confirmed. A 'failed' experiment is often the most instructive: it teaches you that an assumption is wrong. Write the insight here in full sentences, not just the number. Good lessons carry over to other experiments and decisions.

Questions that help

  • ·What do we know now that we did not know before the test?
  • ·Which assumption within our hypothesis was specifically confirmed or refuted?
  • ·What carries over to other experiments?

Example

Three-tier pricing works for our audience, even with a higher absolute price for the middle tier. The 'Recommended' badge probably plays a bigger role than the fact that there are three tiers; a follow-up experiment will isolate that.

Common mistake

Just repeating the outcome ('it worked!') without saying what it really means.

07

Next Step

Based on what you have learned, what is the next concrete action? Roll it out to all users, design a follow-up experiment, redesign and retest, or drop the idea. Make it concrete: who does what, by when? Without a concrete follow-up action, the lessons stay on paper.

Questions that help

  • ·Scale, test further, redesign or stop?
  • ·Who takes the next step, and when?
  • ·Which assumption do we want to test next?

Example

Next step: roll out three-tier pricing to all new visitors within 7 days. In parallel: experiment B2 to vary only the Recommended badge. Owner: the pricing PM, deadline: 22 April.

Common mistake

Not defining a next step and closing the canvas with 'we will evaluate later'.

Common mistakes

Testing too many variables at once

A test that changes colour, copy and price at the same time gives you no interpretable result. Change one thing per test, or acknowledge up front that it is an exploration rather than an experiment.

Filling in the success criterion afterwards

The whole point of fixing criteria in advance is that you cannot rationalise. Fill in the success criterion before the experiment runs, and do not change it afterwards.

Ignoring statistical significance

A 'difference' of 8% with 200 visitors per variant is within the noise. Work out in advance how large a sample you need, or accept that it is a directional indication, not proof.

Forgetting guardrails

Optimising a metric without watching what shifts elsewhere sooner or later puts Goodhart's Law into action. Always define one or two guardrails.

Skipping the 'lessons learned' block

Without an insight written down explicitly, the lessons evaporate. A team that experiments without documenting will repeat the same tests a year later.

Seeing a 'failed' experiment as wasted time

A refuted hypothesis is just as valuable as a confirmed one. The canvas helps the team turn failure into learning rather than embarrassment.

In Boldcaster

How Boldcaster helps you with this

Fill it in interactively

No stray PDF lost in your downloads folder. Fill in the canvas in the app, with version history and comment threads.

Bold AI does the groundwork

Paste in your business plan, annual report or rough notes and Bold AI fills in the canvas for you. Or ask it for suggestions block by block.

Connected to your strategy

Send an insight from this Experiment to your Priorities, KPIs or strategy in one click, instead of leaving it on a whiteboard.

All canvases in one place

Pilot users get access to BMC, Lean Canvas, Value Proposition Canvas and whatever comes next. One platform, one system.

Create a free account and use every canvas

Prefer some guidance?

You don't need to know the models: Bold AI helps you block by block. Want someone alongside you to run the sessions with your leadership team? Start with the scale-up audit. After that, we're happy to introduce you to a coach from our network who works with Boldcaster.

Frequently asked questions

What is the difference between an Experiment Canvas and a Validation Canvas?

+

The Validation Canvas zooms out to a hypothesis and the final go/no-go decision. The Experiment Canvas zooms in on one concrete test that should provide evidence for or against that hypothesis. You often need several experiments per validation.

How many tests should a product team run each week?

+

There is no fixed number. In growing product organisations, teams run 1 to 5 tests a week, depending on complexity and how mature their tooling is. More important than volume is that every test produces a lesson you have booked.

How do I know whether my experiment is statistically significant?

+

Work out in advance the minimum sample you need for the effect you want to detect (use tools such as Optimizely's calculator). Stopping a test 'when it looks good' is a classic mistake.

Do experiments always have to be quantitative?

+

No. Qualitative experiments (user interviews, observations with a prototype) also produce valid lessons, but the success criterion is phrased differently (for example 'at least 4 of the 6 participants understand the flow on their own').

How long should I let a test run?

+

At least one full business cycle (often a week) to average out day-to-day variation. For B2B with long buying cycles, often longer. Do not stop early, even if it looks good.

What do I do with an 'inconclusive' result?

+

Three options: collect more sample to make the test meaningful, redesign the experiment because the measurement was flawed, or accept that this hypothesis goes on hold for now.

How do I share experiment results with stakeholders?

+

The completed Experiment Canvas is your deliverable. Add a short spoken or written summary, but let the canvas lead; that prevents stories that stretch the facts.

Can I use experiments for existing products?

+

Yes. For onboarding flows, pricing models, email campaigns and feature rollouts, structured A/B testing is one of the highest-ROI activities in a mature product organisation.

Further reading

What does a filled-in Experiment Canvas look like?

An empty canvas is the easy part. The hard part is what goes in it. I'll send you a completed Experiment Canvas: a fictional company, real wording, sharp enough to base decisions on.

See a completed example

The strategy canvas of a fictional installation company, fully completed. Plus five signs that a canvas actually steers the business, rather than just being filled in. Four pages, PDF.

You'll see the example straight away and get a copy by email. We keep your address so we can help with your strategy; unsubscribe any time in one click.