Dunelm and Crealytics ran a randomized controlled experiment to compare the attribution system ROAS against incremental impact - with surprising results.
Incrementality: theory vs practice
Exploring new ways to move the needle
Retargeting vs Prospecting
A 5.9x higher return from prospecting
There is more than meets the eye; any attribution model is likely to over- or undervalue the incremental value of any campaign.
In classic holdout tests, a test group gets to see ads while a control group does not. Comparing the resulting buying behavior of both cohorts delivers first estimates on how much revenue would be lost without advertising in that channel. This is what we define as the incremental impact.
Knowing incremental campaign value is only the first step. The final goal is to act on insights by re-allocating budgets to maximize the incremental return on ad spend (3).
Prospecting campaigns to target people with no prior visit to the retailer’s website
Retargeting campaigns to advertise to past website visitors
What is Facebook's Conversion Lift tool?
Conversion lift compares the actions of real people in randomized test and control groups to measure the additional online, offline or mobile app business driven by Facebook, Instagram and Audience Network ads.
During the test, budgets for both segments were balanced to achieve the same nominal ROAS according to client’s Multitouch Attribution. Under a traditional valuation, both campaigns delivered the same advertising efficiency.
The remarketing campaigns incremental ROAS was only 44%, while the acquisition campaigns drove a staggering incremental ROAS of 298%. In other words, the multitouch attribution model underestimated its true value by two-thirds!