• DE&I Commitment
  • Careers
  • Locations
  • Book a Meeting
    Book a Meeting
  • Company
    Learn a little more about us, our values, and our team
    Our Values
    We don't just talk the talk; we live by our core values
    About Us
    Empowering brands to realize their potential with data, insights, and technology
    Leadership
    Meet our leadership team
    Secure Data Architecture
    Our promise of data security and privacy. We keep your data safe from publishers, competitors and bad actors
    Corporate Social Responsibility
    We are committed to making a positive impact on our communities and our planet
  • Platform
    Plan, manage, optimize, and measure your campaigns with our omnichannel platform
    Our Platform
    A platform that connects all walled garden media
    Connected Media
    Create and manage campaigns across search, social, retail media and apps, in one platform
    Connected Data
    Make data-driven decisions as you plan and strategize
    See the industries we serve
    Learn how our customizable solutions can help with your unique needs
    Explore our partner integrations
    See the media, retailer, and data partners we work with
  • Clients
  • News and Events
    Check out recent announcements and see what we’re up to
    News
    Check out our recent media coverage
    Events
    Join us for our next conference or webinar
    Roundtable discussion: Beyond Onsite
    Tackle the next phase of retail media over breakfast with Nectar360 and peers
    Retail Media for Grocery Webinar
    Join us to dig into trends and challenges shaping the industry in 2023
  • Resources
    From new releases, to industry trends and best practices, Skai has you covered
    Blog
    Read the latest insights and thought leadership from our industry experts
    Capabilities
    Take your campaigns to the next level by enhancing your platform capabilities
    Research
    Explore our reports and whitepapers so you can keep up on the latest industry trends
    Subscribe
    Sign up to get the latest updates straight to your inbox
    Quarterly Trends Report
    Learn digital advertising campaign performance trends from Q4 2022
Back to Blog

[Guest Post from Kinase] Why You Must Experiment

Richard Brooks, founder, Kinase

September  13, 2022
why must you experiment

Today, we feature a guest post by Kinase founder Richard Brooks on why experiments and modeling are now essential for taking digital marketing forwards. Founded in 2010, Kinase is a UK-based agency offering a comprehensive suite of digital performance marketing services including paid search, paid social, and a host of other key disciplines.


An accurate view of marketing performance is critical for a business.

Without good information, a business cannot make informed decisions on where to invest budgets or how to optimise individual channels. The result is that a large proportion of those budgets go to waste. And with greater automation of digital bidding and buying, good data is now more valuable than ever.

With UK companies investing more than £23bn in digital marketing annually, the value of good data cannot be overstated. Yet despite the staggering media spending at stake, businesses frequently underinvest in understanding and fail to improve the metrics that underpin it all.

There are two common errors:

  • The wrong KPI is being used in budget and optimisation decisions
  • The KPI is correct, but the measurement of it is wrong

The good news is that neither one is that hard to fix if approached in the right way.

Align KPIs to Objective

There is a simple test for good alignment. State the business objective(s) as succinctly as possible and then compare them to the KPIs being used in decision-making and optimisation. If the two are pulling in different directions—or producing a contradiction—then ask one powerful question: Which KPI would better reflect these objectives?

Some of the more obvious examples would be:

why you must experiment

Finding the ‘perfect KPI’ can be hard, if not impossible, but finding one that better reflects your objectives is often much easier.

Accurate Measurement

It is easy to define what you would like to measure, but actually measuring it can be a little harder. There may be internal systems that require development to join together and output what you require. In some cases (e.g. long sales processes, lifetime value), modeling is involved. The data may not yet even be collected or digitised. It can be a long, but ultimately valuable process, where every incremental improvement will yield a financial benefit.

When linking first-party data back to digital marketing spending, there is an additional problem.

I’ll put it succinctly: all attribution models are wrong.

This is meant with no disrespect. There are many good attribution tools, and the data they provide is essential for day-to-day reporting and optimisations. But that does not detract from the fact that even the best ones are fundamentally unable to provide entirely accurate data. They can never actually read people’s minds as consumers consider a purchase and see ads.

All attribution tools have gaps in the data points they can model. There has always been a gap when it comes to offline interactions (offline ads, shop visits, word of mouth, in-store purchases…). This is now compounded by ever-expanding holes in online data as cookies become ever less reliable.

With large holes in the data, even the most sophisticated attribution modeling cannot provide the whole picture and will overstate the value of digital activities nearest the bottom of the funnel—though they are still far better than relying on Last Click.

why you must experiment

The Solution: Why You Must Experiment

The problem is that the data in online attribution tools is wrong, but it’s the best data you have for making budgeting and bidding decisions.

The solution is to add experimental data to the strategy, and then use the results to calibrate the more real-time data sources (i.e. by updating the attribution model and knowing which biases will still need correcting).

A good experimental approach will meet these five criteria:

  1. It will answer a specific question. e.g. “What is the incremental impact of investing in X on multichannel revenue?”
  2. It will be statistically robust, utilising a genuine geographic AB split and clear confidence levels.
  3. It will minimise business disruption (the lowest possible impact in terms of duration and spend/sales impact).
  4. It will provide a data point that can be directly compared across channels. e.g. Incremental £Sales can be evaluated across online and offline investments, removing silos in budgeting. (Actually, this can go further – what generates a better ROI? Is it a)  store refurbishment or b) YouTube, or c) a Free Delivery offer).
  5. The result will be acted upon objectively.

Having a trusted understanding of the incrementality of digital marketing spend or the true extent of digitally-driven store sales has had a transformative effect on businesses Kinase has worked with. More nuanced tests can also be extremely valuable at a more tactical level.

The good news is that with the tools now available, running a good experiment has never been easier or cheaper. And with the loss of cookies and uncertain economic conditions ahead, the learnings they can generate have never been more valuable.

Request a Demo of Skai

Related Posts

  • [WATCH] Incrementality Testing: Connecting Media and Measurement in a Post-Cookie World
    Read More
  • Sign with the left arrow saying old way and yellow right arrow pointed that says new way.
    The Post-Attribution Era Has Begun: How to Transition Your Business to Cookieless Marketing Measurements
    Read More
  • Differences between attribution vs incrementality in digital advertising.
    The Single Biggest Difference Between Incrementality and Multi-Touch Attribution
    Read More
  • How Skai Clients Use Incrementality Testing to Lessen the Impact of Budget Cuts
    Read More
  • Test and Learn Marketing Can Guide Your Way Through Even the Toughest Market Conditions
    Read More
  • Two business people sitting down looking at incrementality testing examples.
    What Is Incrementality Testing and How You Can Understand Marketing Measurement Approaches
    Read More
  • Share on Facebook
  • Share on Twitter
  • Share on LinkedIn
  • Share via Email
  • Copy Link
    Copied!
Tags: Guest Post, Incrementality, Measure, Optimize, xTLx

Subscribe to Updates

Media that matters.
Marketing that works.
© 2023 Kenshoo, Ltd. All Rights Reserved.
Privacy Policy. Cookie Policy. Recruitment Privacy Policy.
  • Connected Data
    • Market Intelligence
    • Our Approach
    • By Need
    • By Solution
  • Connected Strategy
    • Dynamic Marketing Mix
    • Budget Forecasting
    • Strategic Consulting
  • Connected Media
    • Overview
    • Retail Media
    • Paid Search
    • Paid Social
    • App Marketing
    • Auditing
    • Expert Services
  • Measurement
    • Incrementality
    • Experiments
    • Cross-Channel Attribution
  • Resources
    • Blog
    • Glossary
    • Case Studies
    • Training & Enablement
    • Developer Hub
Privacy Preference

We use cookies on our website. Some of them are essential, while others help us to improve this website and your experience.

Privacy Preference

Save All

Save

Accept Only Essential Cookies

Manage Cookie Preferences

Cookie Details Privacy Policy Imprint

Privacy Preference

Here you will find an overview of all cookies used. You can give your consent to whole categories or display further information and select certain cookies.

Save All Save Accept Only Essential Cookies

Back

Privacy Preference

Essential cookies enable basic functions and are necessary for the proper function of the website.

Show Cookie Information Hide Cookie Information

Name
Provider Owner of this website, Imprint
Purpose Saves the visitors preferences selected in the Cookie Box of Borlabs Cookie.
Host(s) .skai.io, skai.io
Cookie Name borlabs-cookie
Cookie Expiry 1 Year
Name
Provider Owner of this website
Purpose This cookie stores selections made by the user in the Accessibe tool in order to maintain those settings on future visits. These cookies help us make our website compliant with our obligations under US law.
Privacy Policy https://accessibe.com/privacy-policy
Cookie Name acsbState, acsbReset
Cookie Expiry n/a
Name
Provider Owner of this website
Host(s) skai.io
Cookie Name wordpress_sec_,wordpress_test_cookie,wp-postpass_*, wordpresspass_*, wordpressuser_*
Cookie Expiry Session / 1 Year

We use these cookies to enhance functionality and allow for personalisation, such as live chats, videos and the use of social media.

Show Cookie Information Hide Cookie Information

Accept
Name
Provider Owner of this website
Host(s) .chilipiper.com, skai.chilipiper.com
Cookie Name fs_uid, CHILI_PIPER_CLUSTER, guest-session, _sp_ses*, _sp_id*
Cookie Expiry Session / 2 Years
Accept
Name
Provider Owner of this website
Host(s) .comeet.co, www.comeet.co
Cookie Name visid_incap_, nlbi_#######, incap_ses_, referrer22_00a, incap_ses_1364_2167377
Cookie Expiry Session / 1 Year
Accept
Name
Provider Owner of this website
Host(s) skai.io
Cookie Name moduleFormPardotDownload
Cookie Expiry 30 days

Statistics cookies collect information anonymously. This information helps us to understand how our visitors use our website.

Show Cookie Information Hide Cookie Information

Accept
Name
Provider Google Ireland Limited, Gordon House, Barrow Street, Dublin 4, Ireland
Purpose Cookie by Google used for website analytics. Generates statistical data on how the visitor uses the website.
Privacy Policy https://policies.google.com/privacy?hl=en
Cookie Name _ga,_ga_*,_gat,_gat_*,_gid
Cookie Expiry 2 Months
Accept
Name
Provider Hotjar Ltd., Dragonara Business Centre, 5th Floor, Dragonara Road, Paceville St Julian's STJ 3141 Malta
Purpose Hotjar is an user behavior analytic tool by Hotjar Ltd.. We use Hotjar to understand how users interact with our website.
Privacy Policy https://www.hotjar.com/legal/policies/privacy/
Host(s) *.hotjar.com
Cookie Name _hjClosedSurveyInvites, _hjDonePolls, _hjMinimizedPolls, _hjDoneTestersWidgets, _hjIncludedInSample, _hjShownFeedbackMessage, _hjid, _hjRecordingLastActivity, hjTLDTest, _hjUserAttributesHash, _hjCachedUserAttributes, _hjLocalStorageTest, _hjptid, _hjSessionUser_2229986, _hjIncludedInPageviewSample, _hjIncludedInSessionSample, _hjAbsoluteSessionInProgress, _hjFirstSeen
Cookie Expiry Session / 1 Year

Marketing cookies are used by third-party advertisers or publishers to display personalized ads. They do this by tracking visitors across websites.

Show Cookie Information Hide Cookie Information

Accept
Name
Provider Linkedin
Cookie Name lidc, li_gc, lang, AnalyticsSyncHistory, UserMatchHistory, li_sugr, bcookie, TDCPM, TDID, bscookie, ln_or
Cookie Expiry Session / 1 Year
Accept
Name
Provider Skai
Accept
Name
Provider 6sense
Cookie Name _gd_session, _an_uid, _gd_visitor, _gd_svisitor, 6suuid
Cookie Expiry Session / 400 Days
Accept
Name
Provider Pardot
Purpose Cookie name associated with services from marketing automation and lead generation platform Pardot. The visitor value is the visitor_id in your Pardot account. This cookie is set for visitors by the Pardot tracking code.
Host(s) .pardot.com, pi.pardot.com, skai.io
Cookie Name pardot, visitor_id*, lpv*
Cookie Expiry Session / 10 Years
Accept
Name
Provider Google Ireland Limited, Gordon House, Barrow Street, Dublin 4, Ireland
Purpose Cookie by Google used for conversion tracking of Google Ads.
Privacy Policy https://policies.google.com/privacy?hl=en
Cookie Name IDE, 1P_JAR, NID, SOCS, CONSENT, AEC, _gcl_au, OTZ, test_cookie
Cookie Expiry Session / 400 Days
Accept
Name
Provider Meta Platforms Ireland Limited, 4 Grand Canal Square, Dublin 2, Ireland
Purpose Cookie by Facebook used for website analytics, ad targeting, and ad measurement.
Privacy Policy https://www.facebook.com/policies/cookies
Cookie Name _fbp,act,c_user,datr,fr,tr,m_pixel_ration,pl,presence,sb,spin,wd,xs
Cookie Expiry Session / 1 Year

Content from video platforms and social media platforms is blocked by default. If External Media cookies are accepted, access to those contents no longer requires manual consent.

Show Cookie Information Hide Cookie Information

Accept
Name
Provider Wistia
Host(s) .wistia.com
Cookie Name cb_anonymous_id, _sp_ses.2b40, _li_dcdm_c, __hssrc, _gcl_au, _clsk, hubspotutk, _sp_id.2b40, __hssc, __hstc, _uetsid, _uetvid, _gid, _ga, _ga_GQR109DZ3Y, _lc2, fpi, _ex-pricing-cta, _fbp, cb_group_id, cb_user_id, _clck
Cookie Expiry Session / 400 Days
Accept
Name
Provider Meta Platforms Ireland Limited, 4 Grand Canal Square, Dublin 2, Ireland
Purpose Used to unblock Instagram content.
Privacy Policy https://www.instagram.com/legal/privacy/
Host(s) .instagram.com
Cookie Name pigeon_state
Cookie Expiry Session
Accept
Name
Provider Openstreetmap Foundation, St John’s Innovation Centre, Cowley Road, Cambridge CB4 0WS, United Kingdom
Purpose Used to unblock OpenStreetMap content.
Privacy Policy https://wiki.osmfoundation.org/wiki/Privacy_Policy
Host(s) .openstreetmap.org
Cookie Name _osm_location, _osm_session, _osm_totp_token, _osm_welcome, _pk_id., _pk_ref., _pk_ses., qos_token
Cookie Expiry 1-10 Years
Accept
Name
Provider Twitter International Company, One Cumberland Place, Fenian Street, Dublin 2, D02 AX07, Ireland
Purpose Used to unblock Twitter content.
Privacy Policy https://twitter.com/privacy
Host(s) .twimg.com, .twitter.com
Cookie Name __widgetsettings, local_storage_support_test
Cookie Expiry Unlimited
Accept
Name
Provider Vimeo Inc., 555 West 18th Street, New York, New York 10011, USA
Purpose Used to unblock Vimeo content.
Privacy Policy https://vimeo.com/privacy
Host(s) player.vimeo.com
Cookie Name vuid
Cookie Expiry 2 Years
Accept
Name
Provider Google Ireland Limited, Gordon House, Barrow Street, Dublin 4, Ireland
Purpose Used to unblock YouTube content.
Privacy Policy https://policies.google.com/privacy?hl=en&gl=en
Host(s) google.com
Cookie Name CONSENT
Cookie Expiry 6 Month

Borlabs Cookie powered by Borlabs Cookie

Privacy Policy Imprint