Prove the lift was real
A model only earns its keep if it changes what customers actually do. Hold a slice back, treat the rest, and measure the gap. Incrementality separates the spend you caused from the spend that would have happened anyway.
Where it sits: The Prove stage of the loop. Every model's play is validated here before it earns more budget.
Raw before-and-after would claim the whole targeted bar. True attribution counts only the gap above the control, the part that would not have happened without the model. Illustrative.
Control versus treated
Split the audience in two. The control group is deliberately held back and sees nothing. The treated group gets the campaign. Both are drawn from the same population, so anything they have in common, the season, the offer in the market, the mood, cancels out. Whatever is left above the control is the part the model caused.
A raw before-and-after reading would happily claim the whole treated bar. That is how good programmes get talked out of their budget: they take credit for spend that was always going to land.
Two layers of holdout
Did this send work?
A small slice of the target audience for one campaign is held back. The gap between the treated and the held-back customers is the lift of that specific send. Cheap, fast, and it keeps every play honest on its own terms.
Did the whole programme work?
A standing slice of customers is held out of everything for a period. It answers the harder question: across every campaign combined, is the programme adding value, or just reshuffling spend that would have arrived regardless?
A win has to clear the bar
A positive gap is not enough. Small samples wobble, and a lucky week can look like a result. Incrementality reports a confidence interval around the lift, and that interval narrows as more Customers accrue. You only call it a win once the whole interval sits above zero, meaning even the pessimistic read still shows real, model-driven lift.