Measurement and evidence
What is a minimum detectable effect (MDE)?
Free, with no card. We read your homepage, draft who buys from you and show your monthly price before you join the waitlist. Nothing is bought until you've approved the plan and the ads.
what it means here
Here is what the term means for a UK buyer
The definition above holds anywhere. Buying media in the UK adds the detail below, including where AdBuyMCP does something different.
The number that tells you whether a test is worth running before you run it. If your design's MDE is a 30% lift and a realistic lift is 5%, the experiment cannot answer your question and should not be commissioned. AdBuyMCP returns the MDE alongside an underpowered verdict precisely so the answer is actionable: you learn what would have to change as well as that the design failed.
What sets the minimum detectable effect?
Three things set it: the variance in your daily baseline, the number of days of data, and the significance and power you require. AdBuyMCP defaults to 5% significance and 80% power on daily observations.
The required days rise with the square of baseline volatility and fall with the square of the lift you want to detect. Halving the lift you are chasing roughly quadruples the test, so a design that can detect a 20% lift in three weeks needs about twelve weeks to detect a 10% one.
How do you lower it?
Every fix is a design change, so each one has to happen before the flight. Run for the full number of days the calculation asks for. Concentrate the spend in fewer test markets, because the weight in each market matters more than the total budget. Measure an outcome closer to the media and less volatile, such as qualified enquiries in place of closed revenue.
Or accept a larger detectable lift. Designing for the smallest effect you would act on, and leaving the smallest imaginable one alone, saves weeks.
Type your company website or work email to see who we think buys from you and what reaching them 3, 5 or 12 times a week would cost.
Free, with no card.
related terms
These terms sit next to this one
Most confusion in media buying comes from two adjacent terms being used interchangeably, so these are the neighbours worth reading next.
Statistical power
The probability that a test will detect a real effect of a given size, if an effect of that size genuinely exists in the population being measured. A design with low power will usually miss a genuine effect, raises the share of the results it does produce that are noise rather than signal, and exaggerates the size of the effects it does detect.
Geo-lift test
An experiment that runs advertising in some geographic markets and withholds it from comparable others, then compares outcomes between them. It can measure what the advertising caused without a login, a pixel or a match rate.
Incrementality
The share of an outcome that happened because of the advertising, measured against what would have happened anyway. It is measured by comparing an exposed group with a comparable unexposed one, which counting the conversions that followed an advert cannot do.
same group
These terms are also in Measurement and evidence
What you can prove afterwards, and how to tell a number that means something from one that does not.