[Guest Post from Kinase] Why You Must Experiment

Today, we feature a guest post by Kinase founder Richard Brooks on why experiments and modeling are now essential for taking digital marketing forwards. Founded in 2010, Kinase is a UK-based agency offering a comprehensive suite of digital performance marketing services including paid search, paid social, and a host of other key disciplines.

An accurate view of marketing performance is critical for a business.

Without good information, a business cannot make informed decisions on where to invest budgets or how to optimise individual channels. The result is that a large proportion of those budgets go to waste. And with greater automation of digital bidding and buying, good data is now more valuable than ever.

With UK companies investing more than £23bn in digital marketing annually, the value of good data cannot be overstated. Yet despite the staggering media spending at stake, businesses frequently underinvest in understanding and fail to improve the metrics that underpin it all.

There are two common errors:

• The wrong KPI is being used in budget and optimisation decisions
• The KPI is correct, but the measurement of it is wrong

The good news is that neither one is that hard to fix if approached in the right way.

Align KPIs to Objective

There is a simple test for good alignment. State the business objective(s) as succinctly as possible and then compare them to the KPIs being used in decision-making and optimisation. If the two are pulling in different directions—or producing a contradiction—then ask one powerful question: Which KPI would better reflect these objectives?

Some of the more obvious examples would be:

Finding the ‘perfect KPI’ can be hard, if not impossible, but finding one that better reflects your objectives is often much easier.

Accurate Measurement

It is easy to define what you would like to measure, but actually measuring it can be a little harder. There may be internal systems that require development to join together and output what you require. In some cases (e.g. long sales processes, lifetime value), modeling is involved. The data may not yet even be collected or digitised. It can be a long, but ultimately valuable process, where every incremental improvement will yield a financial benefit.

When linking first-party data back to digital marketing spending, there is an additional problem.

I’ll put it succinctly: all attribution models are wrong.

This is meant with no disrespect. There are many good attribution tools, and the data they provide is essential for day-to-day reporting and optimisations. But that does not detract from the fact that even the best ones are fundamentally unable to provide entirely accurate data. They can never actually read people’s minds as consumers consider a purchase and see ads.

All attribution tools have gaps in the data points they can model. There has always been a gap when it comes to offline interactions (offline ads, shop visits, word of mouth, in-store purchases…). This is now compounded by ever-expanding holes in online data as cookies become ever less reliable.

With large holes in the data, even the most sophisticated attribution modeling cannot provide the whole picture and will overstate the value of digital activities nearest the bottom of the funnel—though they are still far better than relying on Last Click.

The Solution: Why You Must Experiment

The problem is that the data in online attribution tools is wrong, but it’s the best data you have for making budgeting and bidding decisions.

The solution is to add experimental data to the strategy, and then use the results to calibrate the more real-time data sources (i.e. by updating the attribution model and knowing which biases will still need correcting).

A good experimental approach will meet these five criteria:

1. It will answer a specific question. e.g. “What is the incremental impact of investing in X on multichannel revenue?”
2. It will be statistically robust, utilising a genuine geographic AB split and clear confidence levels.
3. It will minimise business disruption (the lowest possible impact in terms of duration and spend/sales impact).
4. It will provide a data point that can be directly compared across channels. e.g. Incremental £Sales can be evaluated across online and offline investments, removing silos in budgeting. (Actually, this can go further – what generates a better ROI? Is it a)  store refurbishment or b) YouTube, or c) a Free Delivery offer).
5. The result will be acted upon objectively.

Having a trusted understanding of the incrementality of digital marketing spend or the true extent of digitally-driven store sales has had a transformative effect on businesses Kinase has worked with. More nuanced tests can also be extremely valuable at a more tactical level.

The good news is that with the tools now available, running a good experiment has never been easier or cheaper. And with the loss of cookies and uncertain economic conditions ahead, the learnings they can generate have never been more valuable.

Request a Demo of Skai