In the ever-evolving retail media landscape, measuring marketing campaign impact is a fundamental yet challenging aspect for marketers. Precise, actionable measurement is needed to maximize their retail media investments.
One of the most complex areas for marketing measurement to determine is which customer conversions were truly incremental and which conversions would have happened even without the marketing exposure.
The key to navigating this complex terrain lies in the power of incrementality measurement — an approach that accurately assesses the additional sales or revenue attributable to marketing activity.
And retail media marketers are very interested in solving for incrementality. In Skai’s 2023 State of Retail Media survey, when asked what could slow investment growth in the channel, proving incrementality of investment was one of the top critical challenges cited. 
However, the measurement approach’s effectiveness is only as robust as the tools employed to gauge it. And right now, there’s a lot of noise from vendors in the retail media space claiming to have incrementality solutions. Retail media marketers looking for the right incrementality partner must find a way to sift through the clutter to figure out which has real solutions and which vendors are just making claims that they do.
By blending innovative technology and actionable insights, Skai carves a path forward in the evolution of retail media for marketers looking to drive their marketing efforts to new heights.
Today, we chat with Skai’s VP of Retail Media, Kevin Weiss, who recently spoke on the topic of incrementality in retail media.
5 Questions on Retail Media Incrementality with Skai’s Kevin Weiss
Kevin recently presented at a Path to Purchase Institute Learning Labs event with notable industry thought leaders from around this industry.
In the webinar, The Incrementality of Retail Media: Planning, Activation & Measurement Learning Lab + Roundtable Q&A, Kevin focused on demystifying the complex dynamics of incrementality for retail media, especially in planning, activation, and incrementality.
We caught up with Kevin to delve deeper into some of the core topics he discussed at the Path to Purchase Institute event, including maximizing the incrementality of retail media investments, the significant growth objectives, and the currently trending retail media capabilities. Furthermore, he elaborated on the metrics that matter and the latest methodologies for retail media incrementality testing.
1. How can advertisers overcome the challenge of proving incrementality in retail media?
In retail media, “incrementality” means different things to different organizational stakeholders. In the Path to Purchase Institute webinar, I stressed that the first step is contextualizing what incrementality means to each stakeholder.
While there is a marketing definition of “incrementality” that is predicated on data science, in retail media, it typically refers to spending each dollar as efficiently as possible to drive growth. In that retail media world, “proving” what is incremental versus what is not incremental is typically more subjective than the objective data science involved in incrementality testing.
“In retail media, incrementality means different things to different organizational stakeholders.” – Kevin Weiss
Many advertisers will find it easy to prove incrementality, not because they have a magic solution but because they have a subjective opinion on the confidence level required to satisfy their definition of proof. Other advertisers will find it impossible to prove incrementality, not because they aren’t equipped to do so but because they cannot meet the data science rigor that incrementality testing imposes on marketers.
I suggest looking at retail media as an environment where marketers need to test in an imperfect environment for testing. In that sense, scrutinize the claims you hear, educate yourself on what incrementality means to each stakeholder, seek out new data sources, form hypotheses, conduct tests with as much scientific rigor as possible, and ensure that test & learn spend is included in your budgets as its line item.
2. What are the pros and cons of using new data sources to measure incrementality?
The pros include the ability to form new hypotheses and/or better measure existing hypotheses, the potential of gaining an edge over the competition, and the opportunity to collaborate more closely with a partner or team providing a new data source. The cons include the potential for sunk costs, the opportunity cost of using resources on an unproven data source over a proven one, and the risk of inaccurate or misleading data.
In an ecosystem like retail media, where FOMO runs high, the pros often seem to outweigh the cons when there’s a debate about using new data sources. As an opportunist myself, I tend to push the pros pretty hard. But I respect the counterargument and think those voices need to be heard when discussing new data sources.
3. Can you explain the concept of incrementality scores and how they can be used in testing and measurement?
Incrementality scores are a calculated metric used to measure existing data in a new way to identify optimization opportunities. Incremental Return on Ad Spend (iROAS) is a typical example of an incrementality score. It defines what is incremental and what isn’t by assigning different values to certain types of spend or conversions.
One possible way to define retail media incrementality’s iROAS might be to only use New-to-Brand (NTB) sales in the traditional ROAS calculation.
For example, an advertiser might decide that only New-to-Brand (NTB) customers are incremental and therefore define iROAS as NTB Ad-attributed Sales / Ad Spend. Doing this would make it easier for the advertiser — and the stakeholders across its organization — to assess which investments yield the best incremental return. In turn, the advertiser can more easily determine which investments should be paused or reduced to increase incrementality.
Some incrementality scores are more complex than this simplified example wherein an advertiser assigns a score based on a keyword’s organic rank, whether a keyword is thematically branded/non-branded/conquesting, where an ad appears, the product being advertised, the attribution window applied, the ad clicked versus the product purchased and more. Complex examples can include many different weighting scales.
As a result, when advertising investments are measured against an incrementality score, they will produce wide variances that can be analyzed and/or used to optimize spend. Typically, incrementality scores are applied against publisher-attributed conversions and revenue (e.g., [incrementality score] x ROAS = iROAS).
Yet another way marketers may define iROAS would be to leverage a proprietary ‘Incrementality Score’ metric in the calculation.
Skai supports the usage of incremental scores through various tools, including Custom Metrics, Custom Columns, Dimensions and Categories, Portfolio Subset Strategies, and Bid-to-Custom-Metrics (i.e., ML bid optimization to Custom Metrics). I encourage advertisers to test these strategies and measure how decision-making aided by incrementality scores impacts KPIs like sell-in and sell-through.
4. How important is advertising in improving organic rank, and what strategies can be employed to achieve this?
A generally observed philosophy in retail media is that advertising improves organic rank. And I’m not going to say that I 100% agree or 100% disagree with this philosophy; instead, I believe that advertising can impact organic rank and that advertisers should test and learn to what extent and in which instances they can positively impact organic rank.
In my experience running organic rank tests and building and executing organic rank strategies, there isn’t a 1:1 relationship between advertising and organic rank — meaning there are other variables involved that are as important (if not more critical) than advertising to determine where a product will rank in organic search results.
The best strategy is to analyze existing organic rankings and keyword spending, define your goals, write down your hypothesis, commit to a test where you won’t change other variables during the test, execute the test, and share your findings. If you can work with a data scientist or skilled analyst, define a test where you’ll have a high confidence level in the test results.
I’ve found that a combination of factors contributes to organic rank in retail media. Title optimization, image optimization, enhanced content, price, ratings/reviews, and conversions from external traffic will all significantly impact organic rank. Working on all factors concurrently is most likely going to achieve improvement in organic rank.
5. Can you provide some practical tips for planning and activating incrementality in retail media?
The first practical tip is to budget for testing. In the hundreds of media plans I’ve seen from advertisers, it’s extremely rare for me to see line items in there for test and learn, which means most advertisers aren’t earmarking dollars for testing. That’s not to say that advertisers aren’t testing. Still, it is more an acknowledgment that most of the “testing” is done in a non-scientific manner where the allocations within ad types or product categories are done based on subjectivity rather than being rooted in data science.
The second practical tip is to clean up existing campaigns. I’ve audited hundreds of Amazon Ads accounts, and it’s common to see 5-15% of ad sales attributed to non-branded campaigns from branded search queries. It’s also common to see product-targeted ads tagged as “Non-brand” or “Category” that are running on their product pages, which should be categorized as “Branded” or “Defensive.” For advertisers looking to implement incrementality scores, you will likely generate false positives when calculating iROAS.
The third practical tip is to try holdout tests. Holdout testing is a gold standard in the data science community regarding incrementality testing. It’s the practice of going 100% dark (i.e., pausing spending) on a test group while leaving the rest of your spending (i.e., control group) as-is. Going dark on ads can be painful, but it’s an excellent way to validate the impact of advertising on overall sales, organic rank, or other KPIs you are considering as part of your equation for incrementality in retail media.
Skai’s innovative approach to retail media incrementally and marketing
Skai’s Retail Media solution empowers marketers to plan, execute, and measure digital campaigns that meet consumers when and where they shop. As part of our omnichannel platform, connect the walled gardens and manage campaigns on 100+ retailers, including Amazon, Walmart, Target, and Instacart, alongside major publishers across paid search, paid social, and apps.
Client results include:
- 461% increase in Amazon Ads ROAS and 57% increase in page traffic for Bondi Sands
- 92% increase in share of voice on Amazon for a Fortune 500 CPG brand
- 72% increase in revenue for VTech
- 1,390% year-over-year sales growth for Kamado Joe
For more information on Skai and our solutions, please schedule a quick demo to see our innovation firsthand.
powered by Borlabs Cookie