The Methodology Takes Resources and Patience, and Might not be Right for All Businesses

Marketers – at the request of their CMOs, CROs, and CEOs – look to analytics now more than ever to not only prove their value, but to help the top-line growth of the business they have been trusted to promote and advertise. Of the main ad tech measurement methods (incrementality, marketing attribution, and media mix modeling), incrementality is perhaps the most straightforward, yet hardest to perform correctly.

Gartner defines incrementality as “the uplift in value (sales, conversion rate, etc.) delivered by a tactic, campaign, channel, or experience, that would not have been realized had it not been executed.”

If you were to stop all of your advertising on Facebook, would your revenue – incrementally – actually go down?

Or are those Facebook ads only cannibalizing sales you would otherwise have generated via another channel? Incrementality is the impact witnessed when advertising on a medium is active versus ads on that medium not being active.

Thankfully, marketers are adopting incrementality more and more to help in their media buying decisions. Brands choosing to forego this measurement method could be missing out on additional customers that may not be obvious through ROAS only.

What is Incrementality?

Incrementality uses a more in-depth, and what may seem like a more complicated, equation to measure accurate ad performance. When you learn how conversion rates are measured, it’s actually an easy process to understand. 

By measuring the incremental lift or how likely the viewer of an ad is to buy, marketing professionals can gain insight into how truly effective an ad campaign is. ROAS, on the other hand, only measures the profit gained after a period of time in relation to the cost of the campaign.

What does Incrementality Help Answer?

Incrementality can help answer a variety of questions marketers could have about their campaigns.

1. Which marketing campaign or social media channel is working to increase the success of my desired outcome?

The outcome could be anything you’re hoping to achieve through your marketing efforts. For example, more leads, a higher ROAS, increased revenue, or even an increase in brand awareness. 

2. How can I adjust my current campaigns, using the numbers I have gathered, to increase the success of my desired outcome?

When using ROAS alone, marketers lack the detailed insight necessary to change their current campaign strategies where specific weaknesses may be hidden. On the other hand, incrementality can pinpoint what’s working and where you should be focusing your attention.

3. Which ads on what social media channels should I be using?

There are ways in which incrementality testing can answer any this question. By comparing which combinations provide the greatest lift, marketers can narrow their focus with confidence.

How can Marketers Test Incrementality?

Incrementality can be tested by developing two separate groups and measuring which group outperforms the other. One group consists of those who have interacted with a particular ad, and the other group is considered the control, or individuals who haven’t seen the ad. Once the information from both groups has been accumulated, the incremental lift can then be evaluated using the following equation:

Lift = Test (conversion rate) – Control (conversion rate) / Control (conversion rate)

You can also calculate the incremental impact on revenue using the following equation:

Incrementality = Test (conversion rate) – Control (conversion rate) / Test (conversion rate)

These equations may seem a bit confusing, but they can provide valuable insight for businesses.

Say, for example, you implement an incrementality test on your social media activity to determine how well your Facebook sponsored ads are working. You show your ad to a select group of your audience but make sure to keep a control group of similar people from seeing your ad. You then discover that the percentage of those who converted and did not see your ad is 10%, while those who converted after seeing the ad is 15%. The result is a 5% lift with a 25% incrementality.

Although incrementality is one of the best ways to measure ad success and is becoming more popular among marketers, other factors should be considered. There can be outside influences that can impact lift rates. For example, using Black Friday to test your incrementality may not provide accurate data because whether your ad was seen or not, that day has a high number of additional influences driving sales. When used effectively with a modest level of external influences, incrementality can be a useful tool for measuring true marketing campaign success.

How to do Incrementality Properly

You might need to think a bit more like a scientist and less like a marketer to accurately measure incrementality. You’ll need to segment a portion of your campaigns and treat them more like an experiment; you’ll need to determine a starting hypothesis; you’ll need to determine the independent variables that can be changed in your experiment; and, most importantly, you’ll need to control the environment as much as possible to make sure your results are valid.

Is determining incrementality hard? You bet. Is it worth it? Read on and decide for yourself. And check out our infographic for a visual look at the promise and reality of incrementality.

Establishing a Control Group

Incrementality requires establishing a control group – or baseline – right out of the gate. The group needs to be exactly the same as the test group in terms of reaching the same audience, at the same time, and through the same channels. The only variable between the control and test group is that the test group is exposed to the advertisement and the control group is not. This first step is extremely important, and without it you won’t be measuring incrementality. For some businesses, this can be a show-stopper step. Imagine the difficulty of running Facebook ads and trying to identify who was exposed and who was not.

In addition, you need enough datapoints to achieve statistical significance, which for many companies could be in the hundreds, if not thousands, of touchpoint and conversion events. These need to occur within both the control and test groups equally.

Global brands such as P&G can likely put resources toward a full-blown incrementality study, but for mid-market companies it could be quite the daunting task. Big brands have the budget and the lengthy timelines required to derive value from a significant incrementality test. For smaller brands, if you can’t control the environment, or don’t get enough data, there is simply no way to get actionable insights.

Establishing Your Controls for Incrementality Testing

The most accurate incrementality results come from deployments that at a minimum have the following controls in place:

• Seasonality – Is this a promotion that only works at a certain time of year, perhaps around the Christmas holidays, specifically? You have to compare your campaigns over the same season and same time period for an accurate incrementality reading.
• Geographic Region – Where your ads run and what audience you’re targeting matters, so you need to make sure the ad is targeting consumers from the same city, state or country – depending on the product, service and campaign.
• Offer – Is this a special offer or promotion, and is it tied to seasonality or a unique event? Are your competitors offering similar promotions tied to the same unique event? That can affect incrementality results.
• Spend/Impressions Shared – You need to determine what is your share of the advertising inventory that is available. What percentage of that overall volume of interest are you capturing? Your dominance on any particular medium has an impact. Outside forces can increase or decrease your effectiveness across a channel.
• Competition – You need to try and isolate the effects of third parties, such as your competitors or even just advertising competition in a given medium. Are there new competitors in the space with alternative products and services? Has the competition or price of advertising increased in this particular medium over time?

Incrementality and Multi-Touch Marketing

Multi-touch marketing campaigns are so multi-variate that attempting to isolate the effect of a single variable – for example, the presence of Google Ads or not – is extremely difficult. Anyone running incrementality campaigns in March 2020 likely had compromised data due to the effect of COVID-19 impacting marketing touchpoints and customer journeys.

If you “shut a channel down” to see if it is worth its salt at all – which is what the incrementality methodology requires – what do you do if you find your revenue suffered because your conscious choice to shutter than one channel did more harm than good? Your knee-jerk reaction would be to turn that channel back on, and fast. But if your revenue didn’t change, or maybe even went up, you would seem like a prophet for stopping your spend on that underperforming channel.

Sounds simplistic; it’s anything but. Is controlling the variables so tightly even possible? And is the juice worth the squeeze?

If you only have one channel in your marketing campaign – let’s hope you don’t – incrementality is a moot point. But most marketers have marketing campaigns across a spectrum of online and offline channels. To measure incrementality correctly, you need to assess how all the other channels are affected by the one channel you are incrementally testing. The “control” in control group gets complicated.

“Perfect marketing measurement requires including all touchpoints and accurately measuring incrementality, but it is expensive and not always needed,” said Jason McNellis, senior director analyst at Gartner.

Customer Journeys Are Unique

Finally, not all customer journeys are exactly the same. One journey could look something like this: the would-be customer saw a TV ad, visited your site, saw a Facebook ad, filled out a request-for-information form, then visited your ecommerce site or physical store to make a purchase. If every buyer took that same path, incrementality would be a lot easier. But each consumer shops differently, acts and reacts differently, and buys on a different timeline. If you take the Facebook ad out of the customer path listed above, then run an attribution report showing the sales/revenue impact, is that incrementality?

If that same customer didn’t click the Facebook ad, but instead took three additional days to convert, and clicked on two Google ads instead of the Facebook ad, would it be a good decision to turn off (stop funding) Facebook advertising?

Like many things, the answer is it depends. 

Marketers can’t always look at their data in a vacuum. The 3-5 other channels involved in the consumer’s decision to buy had varying impacts, and costs,as well. Not to mention, what your competitor was doing to market, and on what channels and with what frequency, all impact your effort to have an accurate control.

Interested in Incrementality? We Can Help

As a provider of impartial multi-touch attribution (MTA), we use as many as 7 attribution models to tell you exactly which channels are working, which ones are not, and which combination of channels make marketing campaigns most effective in driving revenue so you can optimize your return on ad spend (ROAS).

MTA provides an incremental look at your marketing channels, just using different methodology. And it does so in real-time, without any complicated experiments. It takes what is deployed now and optimizes it. If you want to know if revenue goes up or down with or without Google Ads, just stop the spend and your ROAS report will tell you the answer.

Incrementality is important to consider and could be described as an “ideal state” from a data perspective. MTA looks at the performance of all channels, weights them, and determines which campaigns delivered the highest ROAS. The data from these reports clearly dictate the next steps to make with your ad spend.

If you’re looking for incrementality, you are likely asking the question “are these ads working?” We would love to help answer that question using multi-touch attribution. We take a hands-on and proactive approach – from the beginning onboarding process to providing unbiased attribution data and insights. Our data scientists use our software to look at all of your marketing channels, incrementally, and will help guide marketing spend to improve ROAS. Schedule a demo today.

---