In our last discussion with ad and martech expert Andrew Covato (ex-eBay, Google, Meta, Netflix), we explored how marketers should navigate common obstacles to adopting an incremental approach to measurement.
This follow-up Q&A delves into how optimizing for correlative (vs causal) data has led many marketers to waste ad spend on campaigns that aren’t delivering results…and how to break free from the post-exposure attribution matrix.
Q: Jumping right in: How is post-exposure attribution leading to misallocated ad spend?
Andrew: The single biggest influence that post-exposure attribution has had on ad tech is that it became the de-facto way that ad optimization algorithms were trained at just about every major ad platform.
Post-exposure signals became the objective function, providing feedback to machine-learning algorithms and guiding ad delivery.
Post exposure attribution describes a correlative relationship between ad exposure and users’ actions;
ie “Here’s what a user does after they see and/or interact with an ad”.
While causality illustrates how ads have influenced a users’ actions versus what actions are associated with ad exposure;
ie “Here’s what a user does when exposed to an ad, versus what they do when not exposed to an ad”.
When you start comparing behaviors of users in presence or absence of ads, you can get at the “holy grail” of measurement: incrementality.
In other words, how many sales or conversions occur as a result of your ads versus the baseline of sales/conversions that would have occurred regardless of your ads.
According to Growth by Science’s own research, using a post-exposure methodology actively encourages shifting spend AWAY from campaigns that are more incremental, and TOWARD those that appear to perform correlationally—but are less incremental.
Q: Wow…so what does it mean for the industry that ad optimization algorithms have been trained on correlative data versus causal data?
Andrew: Post-exposure optimization and measurement is the crux of what I like to call “the self-fulfilling prophecy of ad tech”. Ad optimization models have been trained on correlative signals—things that happen together—versus a cause and effect relationship.
The success of ad delivery is measured by seeing how many events followed ad exposure.
But this whole circular affair only assesses how effectively ad platforms show ads to people who will perform an action marketers care about—whether they see the ad or not. It does reflect how many actions the ads drove!
Now…there is a subtlety here. Sometimes showing ads to people who have an intrinsically high probability of converting WILL cause more of them to convert than would have without ads.
I like to call this “causality by coincidence” and it’s a nice result of a sub-optimal system.
It also means that the broken system I have described isn’t totally broken…but it is far from ideal.
My belief is that many ad platforms today are only partially delivering results for advertisers, and mostly are taking a tax on performance that advertisers would have gotten anyway.
There are tens of billions of wasted ad dollars every year for this exact reason.
The solution? Advertisers must step out of “the Matrix” of post-exposure measurement by taking the proverbial “red pill” and dump post-exposure attribution once and for all.
Q: And what should advertisers use instead of “post-exposure” measurement?
Andrew: Incrementality. The only “red pill” to get out of the Matrix is to run a true causal lift experiment, making sure you have a “counterfactual”. This is a real or correctly modeled group of users who do not see ads that are statistically indistinguishable from users who do.
Comparing actions from the treated group (i.e., those who see ads) to the counterfactual will enable you to understand the incrementality of an ads program. This is the classic “test vs. control” setup, where treated = test, and counterfactual = control.
Q: Privacy has obviously had a huge influence on ad tech and measurement. What is the biggest blindspot you see in marketers evaluating the measurement landscape?
Andrew: So many advertisers are trying to keep the MTA dream alive! We have been addicted to granular data for so long—even though the way we used it was wrong. As a result, too many marketers are looking at measurement solutions with the mindset “iOS 14 and the privacy changes have been a roadblock for me and I just want to find a way to just keep multi-touch attribution approach going”.
But this mindset is going to lead them to evaluate a totally different set of solutions–like cleanrooms and privacy enhancing technology. And while those technologies are incredible and could be transformative for ad tech, they’re misused if they are trying to answer the question: “How can I revive a path-to-conversion dataset and keep using post-exposure attribution?”.
Rather—if marketers start to understand that the past wasn’t working and use the privacy changes as an opportunity to reevaluate how they have been thinking about measurement, many will land on a triangulated incrementality/MMM/MTA solution, like Crealytics’ offering.
Q: Last question. Let’s say an advertiser has bought into what you’re saying here and wants to update their measurement approach. What are a few questions they should ask vendors when evaluating measurement solutions?
Andrew: Sure…I would ask these questions:
- Do you facilitate ground-truth experiments, ideally using geo-testing with modern synthetic control methods? If you get blank stares, or some “tech tap dancing”, thank the person for their time and move on. No point in pursuing the discussion.
- What other methodologies do you use, and how do your ground-truth experiments calibrate them?
- How hard is it to get up and running with your system?
- Walk me through the timeline, including development work to be done, how much data do I need to give you, what is the level of granularity, what do you do with that data?
- Show me a sample report and walk me through how I would use this report to action results.
- What other clients are you working with in my vertical? In my media spend bracket?
Next Steps:
Ready to take the red pill and break free from the post-exposure matrix?
Explore the possibilities with Crealytics’s incrementality measurement platform and start optimizing your ad spend. Reach out via the contact form.
