---
title: "What is a holdout group? | AdBuyMCP"
description: "A group deliberately excluded from seeing a campaign and kept as a comparison against the group that was exposed to it."
url: "https://adbuymcp.com/glossary/holdout"
---

[Home](https://adbuymcp.com/) / [Glossary](https://adbuymcp.com/glossary) / Holdout

Measurement and evidence

# What is a holdout group?

A group deliberately excluded from seeing a campaign and kept as a comparison against the group that was exposed to it. Without one, the counterfactual (what would have happened anyway) has to be assumed rather than observed.

Also called control group

[Get my plan and price](https://adbuymcp.com/start) [Book a 30-minute call with the founder](https://adbuymcp.com/contact#book)

Free, with no card. We read your homepage, draft who buys from you and show your monthly price before you join the waitlist. Nothing is bought until you've approved the plan and the ads.

What it means here

## Here is what the term means for a UK buyer

The definition above holds anywhere. Buying media in the UK adds the detail below, including where AdBuyMCP does something different.

The most common way a lift test quietly measures nothing is a contaminated holdout: the control markets received some of the campaign, so the comparison is exposed against slightly-less-exposed. AdBuyMCP checks for this before the readout and refuses with a specific instruction: exclude those markets from targeting for the test window, or nominate a control the campaign does not buy.

## How big should a holdout be?

Big enough that the exposed and held-out series can each move against their own daily noise. In a geo test that means more than one market on each side, with enough conversion volume that a plausible lift clears the power calculation. AdBuyMCP takes the top three matched controls per test market, and two well-matched, high-volume exposed markets usually beat six thin ones.

A national campaign that buys everywhere has no holdout at all, so its incrementality cannot be measured this way. Withholding media somewhere is what turns a report into a test.

## How do you keep a holdout clean?

Exclude every held-out market from targeting for the test window, and check the delivery after launch. A targeting fallback that widens to national, a programmatic line delivering outside its polygons, or a search campaign with no geographic restriction will each leak spend into the control, and all three bias the result towards zero.

Where a rail reports no geography at all, a clean control can only be assumed. AdBuyMCP then reports control integrity as unverified, with the count of delivery rows that carried no geography, and an unverified check never counts as a pass.

Read next

- [Why most incrementality tests are invalid Goes deeper](https://adbuymcp.com/guides/why-most-incrementality-tests-are-invalid)
- [The four channels in an awareness plan Channels](https://adbuymcp.com/channels)
- [What UK awareness advertising costs Cost](https://adbuymcp.com/cost)
- [Designing a geo-lift test that can actually be powered Guide](https://adbuymcp.com/guides/designing-a-geo-lift-test)

Type your company website or work email to see who we think buys from you and what reaching them 3, 5 or 12 times a week would cost.

[Get my plan and price](https://adbuymcp.com/start)

Free, with no card.

Related terms

## These terms sit next to this one

Most confusion in media buying comes from two adjacent terms being used interchangeably, so these are the neighbours worth reading next.

- ### [Geo-lift test](https://adbuymcp.com/glossary/geo-lift)
  An experiment that runs advertising in some geographic markets and withholds it from comparable others, then compares outcomes between them. It can measure what the advertising caused without a login, a pixel or a match rate.
- ### [Incrementality](https://adbuymcp.com/glossary/incrementality)
  The share of an outcome that happened because of the advertising, measured against what would have happened anyway. It is measured by comparing an exposed group with a comparable unexposed one, which counting the conversions that followed an advert cannot do.
- ### [Matched market](https://adbuymcp.com/glossary#matched-market)
  A control market chosen because its historical outcome pattern closely tracks the test market's, so that a later divergence can be attributed to the campaign.

Same group

## These terms are also in Measurement and evidence

What you can prove afterwards, and how to tell a number that means something from one that does not.

- [Difference-in-differences](https://adbuymcp.com/glossary#difference-in-differences)
- [Statistical power](https://adbuymcp.com/glossary/statistical-power)
- [Minimum detectable effect (MDE)](https://adbuymcp.com/glossary/minimum-detectable-effect)
- [Attribution](https://adbuymcp.com/glossary/attribution)
- [Last-click attribution](https://adbuymcp.com/glossary#last-click)
- [Marketing mix modelling (MMM)](https://adbuymcp.com/glossary/marketing-mix-modelling)
- [Brand lift](https://adbuymcp.com/glossary/brand-lift)
- [Viewability](https://adbuymcp.com/glossary#viewability)

[All 81 terms](https://adbuymcp.com/glossary)