• DE&I Commitment
  • Careers
  • SIGN IN
  • Book a Meeting
    Book a Meeting
  • Company
    Learn a little more about us, our values, and our team
    Our Values
    We don't just talk the talk; we live by our core values
    About Us
    Empowering brands to realize their potential with data, insights, and technology
    Leadership
    Meet our leadership team
    Secure Data Architecture
    Our promise of data security and privacy. We keep your data safe from publishers, competitors and bad actors
    Corporate Social Responsibility
    We are committed to making a positive impact on our communities and our planet
  • Platform
    Plan, manage, optimize, and measure your campaigns with our omnichannel platform
    Omnichannel Advertising
    A platform that connects all walled garden media for more effective campaigns
    Data Connectivity
    Make data-driven decisions as you plan and strategize
    Product Capabilities
    Take your campaigns to the next level by enhancing your platform capabilities
    See the industries we serve
    Learn how our customizable solutions can help with your unique needs
    Explore our partner integrations
    See the media, retailer, and data partners we work with
  • Case Studies
  • News and Events
    Check out recent announcements and see what we’re up to
    Newsroom
    Check out our recent media mentions and announcements
    Events
    Join us for our next conference or webinar
    Omnichannel platform launch
    Read the press release to see how we're helping marketers win the walled gardens
    Cannes Lions 2023
    Don't miss your chance to meet with Skai leadership in Cannes
  • Resources
    From new releases, to industry trends and best practices, Skai has you covered
    Marketer Insights
    Read the latest blog posts and thought leadership from Skai and other industry experts
    Skai Research
    Explore our studies, reports and whitepapers so you can keep up on the latest industry trends
    Quarterly Trends Report
    See the key digital marketing trends from Q1 2023
    The Breakthrough
    The only resource hub built for marketers in the new era of omnichannel
Back to Blog

Collection & Connection: The Key to Harnessing External Data

Skai™ Blog

April  27, 2021

In recent years, investment in data analytics has been on the rise, with the International Data Corporation (IDC) projecting a worldwide market for big data and business analytics solutions that exceeds $274B by 2022. Until now, collecting and connecting internal data has been the primary focus; building connections is often the hardest part, as different teams built their own systems over the years that don’t use the same logic, language, or platform. Expanding an analytics strategy to include external data is exponentially more difficult, but it’s the necessary next step for brands that want to stay competitive.

Why internal data isn’t enough

While important, internal data doesn’t tell a complete story. For one, it’s backward-looking; sales figures, for example, only tell you how a product sold in the past, not how it will sell in the future. Internal data is also narrowly focused on a single brand or organization. For a broader view of the whole market, your brand’s potential, and all consumers (not just your customers), you must have access to external data as well.

But collecting and connecting external data is a far bigger initiative than most brands can tackle on their own, for two reasons: the sheer volume of information available and the complete lack of control over how data is organized. Every day, the internet adds 2.5 quintillion (billion billion) bytes of data, and more than 80% of that data is in totally unstructured and disparate forms. Somewhere in that data is information that could be transformative to your business, like:

  • What your competitors are selling
  • How your inventory compares to the broad category
  • How specific competitors are performing compared to you
  • What small emerging players are up to
  • What consumer needs are out there that have yet to be addressed
  • New product claims, features, ingredients, and more
  • Blog posts and product reviews about competitive products
  • What consumers and experts are saying about your product category
  • New patent filings, research papers, and conferences related to your category

Harnessing external data offers many angles or views on the same topic, creating a holistic view of the entire market. But tapping into this data and generating valuable insights from it requires the kind of advanced artificial intelligence (AI) capabilities that most organizations lack.

The solution: The right analytics partner for harnessing external data

There are plenty of firms out there that claim to offer robust external data analytics. But many of these platforms are limited to pulling in data from one particular data source or data type.

Skai has broken the mold by solving for the complexity of extracting large and disparate datasets and extracting high-quality information from that data. We offer all this information on a configurable platform that generates tailored insights for different industries and business needs. No other data analytics platform collects and connects data the way we do.

How we collect data

The first step to generating our revolutionary market intelligence is researching and choosing appropriate data sources for a given market. Data sources are widely varied across three main classes: third-party API integrations with data suppliers, internal data sources, and publicly accessible websites. Maintaining continuous access to those sources at scale is a complex engineering challenge that requires constant code writing and updating.

While internal and external ecommerce data provides a view into the competitive landscape, social conversations are key to understanding consumer sentiment. Typical social listening tools will track a single brand and perhaps a few competitors. But this data will be very limited. In contrast, Skai will track all social activity relevant to a market.

So instead of following specific yogurt brands like Dannon and Yoplait, for example, we’ll track all mentions of the yogurt category regardless of what brand is or is not associated with them. We capture similarly broad keywords in all other data sources, from specialist sources to sales data to demographic information, in order to widen the lens, present data about the category as a whole, and expose missed opportunities.

Data collection has its challenges, but harnessing external data is actually the easiest part of our process. The way we connect data, on the other hand, is completely unique and incredibly powerful.

How we connect data

Some data sets can be connected to each other easily via universal codes, such as UPCs or SKUs. But this type of identifier is only available for a few specific data sets. In order to connect patent filings (which never include SKUs) with products, for example, we need to rely on something else: the source content that can be extracted and connected to other data sets.

Our proprietary machine learning and natural language processing (NLP) algorithms extract content from unstructured data with a high level of accuracy. Normalizing that content allows us to ensure that all data sets are translated into the same language, using the same taxonomy and values so that each data set can be compared to every other.

Others in the data analytics space are limited in their ability to make these kinds of connections, so they either don’t make any connections at all and consider each data set separately or they make connections between much smaller sets that result in limited insights. The connection nodes we build span across every data set we have, and we do it at scale.

The impact of connected data

With fully connected data, the insights that can be gleaned are staggering. Through the Skai dashboard, you can choose a topic, feature, or benefit – let’s say, yogurt, flavor, or probiotic content – and drill down into every related piece of data, from consumer discussions to research papers, to achieve a complete understanding. Alternatively, you can dive into a specific type of data – online discussions, for instance – and compare consumer discussions vs. expert discussions to see if they are aligned, revealing opportunities for better consumer education.

The diversity of our data connections also makes it possible for us to offer predictive dashboards like our Customer Needs Assessment. This dashboard presents the products available in a market ecosystem alongside the consumer discussions about that ecosystem to expose any gaps between the two. With this information, a brand can proceed with new product development, improved product benefit messaging, or untapped marketing opportunities with confidence.

————————————–

*This blog post originally appeared on Signals-Analytics.com. Kenshoo acquired Signals-Analytics in December 2020. Read the press release.

Request a Demo of Skai

Related Posts

  • Skai’s Chief product officer guy cohen on market intelligenceSkai’s Chief product officer guy cohen on market intelligence
    The Skai 5: Five Questions About Market Intelligence With Skai’s Guy Cohen, Chief Product Officer 
    Read More
  • Networking cloud web with a magnifying glass looking at external data sources.
    Choosing External Data Sources: 4 Characteristics to Look For
    Read More
  • Analyzing metrics and charts
    Market Intelligence vs. Market Research: What’s the difference?
    Read More
  • The 10 Key Market Intelligence Data Sources Brands Should Focus On Right Now
    Read More
  • Woman looking and analyzing her computer screen while holding a pen in her hand.
    What is Modern Market Intelligence?
    Read More
  • Multiple networks and connections
    6 Insights You Should Get from Your Market Intelligence Data
    Read More
  • Share on Facebook
  • Share on Twitter
  • Share on LinkedIn
  • Share via Email
  • Copy Link
    Copied!

    Subscribe to Updates

    Media that matters.
    Marketing that works.
    © 2023 Kenshoo, Ltd. All Rights Reserved.
    Privacy Policy. Cookie Policy. Recruitment Privacy Policy.
    • Connected Data
      • Market Intelligence
      • Our Approach
      • By Need
      • By Solution
    • Connected Strategy
      • Dynamic Marketing Mix
      • Budget Forecasting
      • Strategic Consulting
    • Connected Media
      • Overview
      • Retail Media
      • Paid Search
      • Paid Social
      • App Marketing
      • Auditing
      • Expert Services
    • Measurement
      • Incrementality
      • Experiments
      • Cross-Channel Attribution
    • Resources
      • Blog
      • Glossary
      • Case Studies
      • Training & Enablement
      • Developer Hub
      • The Breakthrough
    Privacy Preference

    We use cookies on our website. Some of them are essential, while others help us to improve this website and your experience.

    Privacy Preference

    Save All

    Save

    Accept Only Essential Cookies

    Manage Cookie Preferences

    Cookie Details Privacy Policy Imprint

    Privacy Preference

    Here you will find an overview of all cookies used. You can give your consent to whole categories or display further information and select certain cookies.

    Save All Save Accept Only Essential Cookies

    Back

    Privacy Preference

    Essential cookies enable basic functions and are necessary for the proper function of the website.

    Show Cookie Information Hide Cookie Information

    Name
    ProviderOwner of this website, Imprint
    PurposeSaves the visitors preferences selected in the Cookie Box of Borlabs Cookie.
    Host(s).skai.io, skai.io
    Cookie Nameborlabs-cookie
    Cookie Expiry1 Year
    Name
    ProviderOwner of this website
    PurposeThis cookie stores selections made by the user in the Accessibe tool in order to maintain those settings on future visits. These cookies help us make our website compliant with our obligations under US law.
    Privacy Policy https://accessibe.com/privacy-policy
    Cookie NameacsbState, acsbReset
    Cookie Expiryn/a
    Name
    ProviderOwner of this website
    Host(s)skai.io
    Cookie Namewordpress_sec_,wordpress_test_cookie,wp-postpass_*, wordpresspass_*, wordpressuser_*
    Cookie ExpirySession / 1 Year

    We use these cookies to enhance functionality and allow for personalisation, such as live chats, videos and the use of social media.

    Show Cookie Information Hide Cookie Information

    Accept
    Name
    ProviderOwner of this website
    Host(s).chilipiper.com, skai.chilipiper.com
    Cookie Namefs_uid, CHILI_PIPER_CLUSTER, guest-session, _sp_ses*, _sp_id*
    Cookie ExpirySession / 2 Years
    Accept
    Name
    ProviderOwner of this website
    Host(s).comeet.co, www.comeet.co
    Cookie Namevisid_incap_, nlbi_#######, incap_ses_, referrer22_00a, incap_ses_1364_2167377
    Cookie ExpirySession / 1 Year
    Accept
    Name
    ProviderOwner of this website
    Host(s)skai.io
    Cookie NamemoduleFormPardotDownload
    Cookie Expiry30 days

    Statistics cookies collect information anonymously. This information helps us to understand how our visitors use our website.

    Show Cookie Information Hide Cookie Information

    Accept
    Name
    ProviderGoogle Ireland Limited, Gordon House, Barrow Street, Dublin 4, Ireland
    PurposeCookie by Google used for website analytics. Generates statistical data on how the visitor uses the website.
    Privacy Policy https://policies.google.com/privacy?hl=en
    Cookie Name_ga, _ga_*, _gat,_gat_*,_gid, GOOGLE_ABUSE_EXEMPTION, AEC, __Secure-ENID, SOCS, CONSENT
    Cookie ExpirySession / 2 Years
    Accept
    Name
    ProviderHotjar Ltd., Dragonara Business Centre, 5th Floor, Dragonara Road, Paceville St Julian's STJ 3141 Malta
    PurposeHotjar is an user behavior analytic tool by Hotjar Ltd.. We use Hotjar to understand how users interact with our website.
    Privacy Policy https://www.hotjar.com/legal/policies/privacy/
    Host(s)*.hotjar.com
    Cookie Name_hjClosedSurveyInvites, _hjDonePolls, _hjMinimizedPolls, _hjDoneTestersWidgets, _hjIncludedInSample, _hjShownFeedbackMessage, _hjid, _hjRecordingLastActivity, hjTLDTest, _hjUserAttributesHash, _hjCachedUserAttributes, _hjLocalStorageTest, _hjptid, _hjSessionUser_2229986, _hjIncludedInPageviewSample, _hjIncludedInSessionSample, _hjAbsoluteSessionInProgress, _hjFirstSeen
    Cookie ExpirySession / 1 Year
    Accept
    Name
    ProviderOwner of this website
    PurposeVWO is AB Testing software that we'll use to test different elements of the website - form placement, nav, different page layouts, etc.
    Privacy Policy https://vwo.com/compliance/cookies-notice/
    Host(s).skai.io, dev.visualwebsiteoptimizer.com
    Cookie Name_vis_opt_exp_{CAMPAIGNID}_goal_{GOALID}, _vis_opt_test_cookie, _vis_opt_exp_{CAMPAIGNID}_combi, _vis_opt_exp_{CAMPAIGNID}_exclude, _vis_opt_exp_{CAMPAIGNID}_split, _vis_opt_s, _vis_opt_out, _vwo_uuid, _vwo_uuid_{CAMPAIGNID}, _vwo_ds, _vwo_sn, _vwo_uuid_v2, _vis_opt_exp_{CAMPAIGNID}_combi_choose, _vwo_referrer, _vwo, _vwo_global_opt_out, _vwo_ssm, uuid
    Cookie ExpirySession / 10 Years

    Marketing cookies are used by third-party advertisers or publishers to display personalized ads. They do this by tracking visitors across websites.

    Show Cookie Information Hide Cookie Information

    Accept
    Name
    ProviderLinkedin
    Cookie Namelidc, li_gc, lang, AnalyticsSyncHistory, UserMatchHistory, li_sugr, bcookie, TDCPM, TDID, bscookie, ln_or
    Cookie ExpirySession / 1 Year
    Accept
    Name
    ProviderSkai
    Cookie Nameken_gclid
    Accept
    Name
    Provider6sense
    Cookie Name_gd_session, _an_uid, _gd_visitor, _gd_svisitor, 6suuid
    Cookie ExpirySession / 400 Days
    Accept
    Name
    ProviderPardot
    PurposeCookie name associated with services from marketing automation and lead generation platform Pardot. The visitor value is the visitor_id in your Pardot account. This cookie is set for visitors by the Pardot tracking code.
    Host(s).pardot.com, pi.pardot.com, skai.io
    Cookie Namepardot, visitor_id*, lpv*
    Cookie ExpirySession / 10 Years
    Accept
    Name
    ProviderGoogle Ireland Limited, Gordon House, Barrow Street, Dublin 4, Ireland
    PurposeCookie by Google used for conversion tracking of Google Ads.
    Privacy Policy https://policies.google.com/privacy?hl=en
    Cookie NameIDE, 1P_JAR, NID, _gcl_au, OTZ, test_cookie
    Cookie ExpirySession / 400 Days
    Accept
    Name
    ProviderOwner of this website
    Cookie NamePHPSESSID, sid, okto-autoposter-XXX, oktgid, oktsid, oktlogid-XXX, oktolead-XXX-Email, oktolead-XXX-Company, oktolead-XXX-FirstName, oktolead-XXX-LastName, oktolead-XXX-Phone, oktolead-XXX-Country, oktolead-XXX-Url, oktolead-XXX-State, oktolead-XXX-Comment, optout
    Cookie ExpirySession / 2 Years
    Accept
    Name
    ProviderMeta Platforms Ireland Limited, 4 Grand Canal Square, Dublin 2, Ireland
    PurposeCookie by Facebook used for website analytics, ad targeting, and ad measurement.
    Privacy Policy https://www.facebook.com/policies/cookies
    Cookie Name_fbp,act,c_user,datr,fr,tr,m_pixel_ration,pl,presence,sb,spin,wd,xs
    Cookie ExpirySession / 1 Year

    Content from video platforms and social media platforms is blocked by default. If External Media cookies are accepted, access to those contents no longer requires manual consent.

    Show Cookie Information Hide Cookie Information

    Accept
    Name
    ProviderWistia
    Host(s).wistia.com
    Cookie Namecb_anonymous_id, _sp_ses.2b40, _li_dcdm_c, __hssrc, _gcl_au, _clsk, hubspotutk, _sp_id.2b40, __hssc, __hstc, _uetsid, _uetvid, _gid, _ga, _ga_GQR109DZ3Y, _lc2, fpi, _ex-pricing-cta, _fbp, cb_group_id, cb_user_id, _clck
    Cookie ExpirySession / 400 Days
    Accept
    Name
    ProviderMeta Platforms Ireland Limited, 4 Grand Canal Square, Dublin 2, Ireland
    PurposeUsed to unblock Instagram content.
    Privacy Policy https://www.instagram.com/legal/privacy/
    Host(s).instagram.com
    Cookie Namepigeon_state
    Cookie ExpirySession
    Accept
    Name
    ProviderOpenstreetmap Foundation, St John’s Innovation Centre, Cowley Road, Cambridge CB4 0WS, United Kingdom
    PurposeUsed to unblock OpenStreetMap content.
    Privacy Policy https://wiki.osmfoundation.org/wiki/Privacy_Policy
    Host(s).openstreetmap.org
    Cookie Name_osm_location, _osm_session, _osm_totp_token, _osm_welcome, _pk_id., _pk_ref., _pk_ses., qos_token
    Cookie Expiry1-10 Years
    Accept
    Name
    ProviderTwitter International Company, One Cumberland Place, Fenian Street, Dublin 2, D02 AX07, Ireland
    PurposeUsed to unblock Twitter content.
    Privacy Policy https://twitter.com/privacy
    Host(s).twimg.com, .twitter.com
    Cookie Name__widgetsettings, local_storage_support_test
    Cookie ExpiryUnlimited
    Accept
    Name
    ProviderVimeo Inc., 555 West 18th Street, New York, New York 10011, USA
    PurposeUsed to unblock Vimeo content.
    Privacy Policy https://vimeo.com/privacy
    Host(s)player.vimeo.com
    Cookie Namevuid
    Cookie Expiry2 Years
    Accept
    Name
    ProviderGoogle Ireland Limited, Gordon House, Barrow Street, Dublin 4, Ireland
    PurposeUsed to unblock YouTube content.
    Privacy Policy https://policies.google.com/privacy?hl=en&gl=en
    Host(s)google.com
    Cookie NameCONSENT
    Cookie Expiry6 Month

    Borlabs Cookie powered by Borlabs Cookie

    Privacy Policy Imprint