Every attribution model answers the same question: who gets the credit for this conversion? Incrementality asks a better one: would this conversion have happened anyway? It is the difference between taking credit for the sunrise because you set an alarm, and proving you actually caused something to happen.
What is incrementality?
Incrementality measures whether a specific marketing activity caused an outcome that would not have happened otherwise. Instead of dividing credit across touchpoints after the fact, it isolates the additional conversions, revenue, or signups a channel genuinely generated, the “lift” above what you would have got for free.
The classic illustration is the movie poster. Last-click attribution is like giving the poster outside the cinema 100% of the credit for your ticket, ignoring the TV trailer, the reviews, and the friend who recommended the film. As AdRoll puts it, “incrementality strives to identify the causal event of a conversion, allowing businesses to properly allocate budget and reduce wasted ad spend”.
How incrementality testing works
The method is borrowed from clinical trials. You split your audience into two groups:
- A test group that sees the marketing (the ad, the email, the campaign).
- A control group, or holdout, that does not.
Everything else is held as equal as possible. After the test runs, you compare conversions between the two groups. The gap between them is the incremental lift: the conversions that exist only because of the marketing. If both groups convert at the same rate, the channel added nothing, no matter what your attribution dashboard claims.
Common ways to run this in practice include geo holdouts (turn a channel off in some regions, leave it on in others), audience holdouts (suppress ads from a random slice of users), and ghost ads or PSA tests on paid platforms.
Why it beats attribution
Attribution models, even the clever ones, are built on correlation. They watch which touchpoints appeared before a conversion and split the credit. They cannot tell you what would have happened in a world where the touchpoint never existed. Incrementality can, because it builds that world on purpose with the control group.
This matters most for the channels attribution flatters. Branded search and retargeting both tend to reach people who were already going to convert, so they hoover up last-click credit while adding little real lift. Incrementality exposes that. It also explains why chasing clicks can be a trap: AdRoll cites research that “clicky users are often more than 5.5 times more costly than non-clicky users, but reaching them does not impact overall sales”.
An example
A SaaS company spends heavily on branded search and sees a glowing return: every pound returns several in pipeline, according to last-click. Before doubling the budget, the team runs a geo holdout, switching branded search off in two comparable regions for four weeks.
Conversions in those regions barely move. Buyers who searched the brand name simply clicked the organic result instead. The “return” was mostly demand the company had already created elsewhere. The incrementality test just saved them from pouring more money into a channel that was capturing demand, not creating it.
The honest limitations
Incrementality is the closest thing marketing has to a truth serum, but it is not free. Tests need enough volume to reach statistical significance, they take time, and a clean holdout can mean deliberately leaving some revenue on the table for the duration. There is also a structural problem: many ad vendors profit from the channels incrementality tends to deflate, so the tooling and incentives do not always push you toward honest measurement. Run the tests anyway. One good holdout will teach you more than a year of dashboards.
Questions to ask yourself
As a modern growth or agile marketing professional, ask yourself the following about incrementality:
- For my biggest channels, do I actually know the lift, or just the last-click return?
- Which channels am I funding that mostly capture demand rather than create it?
- Could I run a geo or audience holdout on one channel this quarter?
- Am I confusing being present at the conversion with causing it?
- Do I have the volume to reach significance, or do I need to test at a higher level?
Related articles
- Marketing Attribution Models: The Definitive Guide
- What is last-click attribution?
- What is multi-touch attribution?
- Marketing mix modelling
- Wittgenstein’s attribution model
About Growth Method
Growth Method is the growth platform designed for experiment-led and data-driven marketers.
Learn more on our homepage, connect with me on LinkedIn or Twitter, or book a call here.