Roy Feig
Product Manager
In my last blog post, I reviewed Facebook Split Testing and why it’s important to use. Today, I’d like to provide some simple steps to make your testing even more successful.
Step 1: Clarify your hypothesis
What are you testing? What is the most important decision you need to make? What data will help you measure the opportunity?
The most important thing is to come up with a hypothesis that you wish to prove or reject. Even better, try to combat your own biases of what you believe to be successful with a real look at whether the data can disprove your strongly held assumptions. Confirmation-only tests are a danger we all have to be on the guard against.
Example Hypothesis: Including the price on an image would drive a larger ROI than images without a price.
Step 2: Define a single goal
You need to choose one KPI metric to determine how to measure the effectiveness of your campaign.
Example: ROI and CTR might both be important, but when testing, there should be only one metric for comparison.
Step 3: Test only one variable
If there are multiple variables in each ad, it is difficult to determine which variable drove the increased or decreased performance. By having only one variable, you can easily isolate the factor that affected overall performance.
Example: Bid type, headline and CTA are probable attributes to influence an ad’s performance, together with the price on the image. But to add all these as tested variables won’t give a good answer as to which attribute made the difference. So if the hypothesis concerns the price on the image, all the other attributes should remain the same.
Step 4: Select the audience to split
The ads should be targeting the same audience at the same time. The split tool is critical to make sure that you are testing only one variable. This test splits the targeted audience of the tested ads so that no person will see both tested ads.
Example: There are many attributes for defining an audience, like age, gender, placement, interest and much more. To simplify, it would be best to split between ad-sets targeting the same Persona (by definition – same audience), guaranteeing that the targeted audience is 100% non-overlapping.
Step 5: Start the test
Change the status of the ad sets from paused to active. Make sure that the split test is already running before ad sets are starting to deliver.
Example: If one of the tested ad sets is already active for a week, it’s ads may be fatigued and a large portion of the audience may have already been exposed to the ads. That will derive unreliable test. To prevent this, be sure to run the split test before activating the ad sets.
Step 6: Monitor the outcome and select the winner
Compare the tested ads based on the selected KPI metric. Make sure there is enough volume before identifying the winning ad. And most importantly, when identifying the winning ad, go with it.
Don’t forget to stop the split test! The winning ad should now target 100% of the audience segment.
Summary
Split testing is technical, but decision making becomes much easier and the insights that can be derived about creatives, bid types, and audiences can be enormous. Here’s to success in your next split!
