POV: You’re a performance marketer in a new role, and you have a pot of money to spend. It’s time to make some key decisions around how to allocate your budget to specific marketing channels and campaigns.
But…you’re beginning to lose confidence in the results of your attribution model. You’re seeing:
- A big difference between reported and actual orders in the backend
- Major differences across different reporting (like ad platform vs attribution)
- Strong tendency for lower funnel activations and a high number of new customers for those campaigns
Getting a good sense of the real performance of each channel is critical before spending your budget.
But all bets are off unless you start with the right measurement methodology.
Last month, we sat down with a few measurement experts to understand how performance marketers must adapt to a soon-to-be cookie-less world by leveraging a triangulated approach to measuring ROI in marketing. This should include a media mix model (MMM), multi-touch attribution (MTA), and incrementality testing.
Here’s what they shared:
When you’re ready to move away from ineffective measurement models…
If you’re an advertiser ready to take that plunge and move away from antiquated measurement technologies, I recommend starting by performing an incrementality test on as much of your media investment as possible. Performing an incrementality test will give you a sense of the overall performance from a scientific perspective of your whole media program.
From there, develop a roadmap and learning agenda to dive in and understand where performance is lacking and where it’s flourishing.
Once completed, progress towards a triangulated approach where you don’t have to do only incrementality tests all the time.
While tests are fundamental, running them continuously is not realistic.
The next step is deploying an MMM specifically calibrated to the results of your incrementality test. This will allow you to extend the lifespan of the results from your incrementality tests.
Then, you’ll want to supplement this methodology with post-exposure attribution, or correlative methods (like MTA) but only to compare the relative performance of sub-campaign elements. This will allow you to perform more day-to-day or week-to-week optimizations between times when you have a more robust testing program activated.
What if your incrementality test results challenge your current approach?
Running incrementality tests is one of the best ways to measure the causal relation between the advertisement and any commercial gains. Overall, with incrementality tests we are aiming to answer the question “Did this advertisement really contribute to our growth?” Plus:
- What would have happened if I turned off this advertisement?
- What is the impact on customer lifetime value?
- How many new customers would I have lost?
If you’re investing into channels that are eligible to run incrementality tests—like Google Ads, Meta or Microsoft—then this is the most sensible way to collect evidence to understand how much the advertisement contributes in your company’s growth.
One thing to be aware of: If you haven’t done any incrementality tests in the past, you may get surprising results compared to your previous methods of measurement, and this may challenge the approach that you had been following in the past. It is important at that point to manage this change within your company.
Explain to your team why there’s a difference between these two methodologies, and why incrementality tests are offering more valid evidence compared to the previous methodology that the company was formed. Reach out to us if you are struggling to socialize this change ; we’d love to help.
Why do we recommend Media Mix Modeling as the backbone of your marketing attribution stack?
At Crealytics, we recommend using an MMM as the backbone of your marketing attribution stack. Why? This model will continuously attribute your company’s performance or revenue across main marketing channels and indicate how much is incremental.
Additionally, we recommend backing this up with incrementality tests, like Andrew mentioned. Incrementality tests help make sure that your media mix model reflects reality, especially for your biggest channels.
By running tests and integrating those results into your MMM, it can ensure accurate results during non-tested periods.
We recommend MTA in certain areas because MMM will struggle to attribute to the campaign level. This is a known issue with MMM. Moreover, you won’t be able to do incrementality tests at the campaign level, as you won’t reach significance with those.
While it’s still fairly reliable, we recommend using MTA, but you must control for the biases inherent in the model.
What can a Media Mix Model reveal?
Running an MMM gives a good understanding of your baseline of marketing – unaffected sales, versus incremental sales that were driven by marketing. The beauty of this model is that you can include many different channels from offline to online, like Print, TV, or OOH. Hence, it will give you a good understanding of where your budget has a positive impact and is effectively utilized. It will also reveal which channels are truly incremental, clarifying where you should invest more.
How do MMM + incrementality tests work together?
As Stephen said, the best MMMs can ingest “priors” through incrementality tests. This measurement methodology comes as close to reality as possible. I would always recommend choosing an MMM that can calibrate it’s output based on this information.
What is Multi-Touch Attribution (MTA) and when to use it?
MTA is a measurement methodology to measure the effectiveness and efficiency of your marketing on campaign-level. An MTA model takes in all marketing touchpoints (campaign-level) a user used on a certain period of time (also known as a customer journey).
Some of the touchpoint sequences lead to success, which is a conversion in the end, but some sequences probably not. By analyzing the different customer journeys and touchpoint sequences an MTA model can measure the effectiveness of each touchpoint in a journey and provides a certain weight to each touchpoint.
In the past, this methodology proved very useful for campaign managers when it was easy to measure on cookie-level. Nowadays we need to reinvent this methodology and move away from the classic MTA approach to not lose the insights on a campaign-level but come closer to the real performance of each campaign, enabling campaign managers to operationalize the results in their day-to-day work.
Why continue to use MTA despite its drawbacks?
Why do we still propose using multi-touch attribution (MTA) as a part of your measurement stack? We know that we can’t reliably test small channels with incrementality tests because the results will be insignificant.
Additionally, we recognize that MMM, which we propose as our main measurement tool in the future, may struggle to provide accurate results at the campaign level. However, marketers still require insights into where they should allocate their budget across channels. So, while we have some kind of MTA modeling available, we can still use it to make campaign-level decisions.
At Crealytics, we’re building an MTA model aimed at mitigating the inherent biases associated with MTA. This involves discounting anything that comes from lower-funnel channels like brand, because we know that they are often mostly capturing upper funnel traffic.
Additionally, we discount transactions from existing highly engaged customers. This model helps us predict the level of an engagement of an existing customer. Transactions from new customers retain their full value, ensuring we don’t discount channels and campaigns that drive business growth by acquiring new customers.
We also force the MTA to agree with the MMM results at a channel level, because we know that MMM is a more holistic attribution tool. We only allow the MTA to attribute the value the MMM attributes to the channel, down to campaigns. This ensures that even MTA campaign-level attribution is based in reality.
Basically, what we’re doing is using the MTA to fill the gaps left by MMM and incrementality testing, while we still have some kind of ability to stitch together customer journeys before the deprecation of cookies.
Can you summarize MTA vs Incrementality Measurement vs MMM?
In summary, here’s what you should know about the three pillars of triangulated measurement:
The strongest evidence comes from incrementality tests, but they can be costly, slow, and impractical for certain channels.
This is where media mix modelling and multi-touch attribution step in.
While MMM offers relatively faster insights at a strategic level compared to MTA, it may lack the granularity needed for day-to-day budgeting decisions.
MTA is the fastest and often the most cost-effective model, but privacy and browsing restrictions pose challenges for its reliability.
Combining these methods compensates for their individual constraints and provides a more comprehensive measurement solution.
By incorporating previous incrementality tests into MMM and adjusting MMM accordingly, you can achieve highly accurate results compared to using these methods alone.
To address MMM’s limitations, particularly with small campaigns and sub-channels, leveraging MTA can help capture additional touchpoints, optimizing insights for better decision-making.
What sets Crealytics’ measurement solution apart from others?
As a measurement consultant, when I look at Crealytics’ solution, there are a few differentiating factors that makes them a really unique approach for certain verticals in the industry.
Number one, their incrementality-based product leans into triangulation, which I believe to be the future of measurement and what all modern performance marketers should be adopting.
Second, their deep expertise in retail media sets them apart from other measurement providers.
And third: Crealytics’ hands-on, consultative approach allows its clients to develop flourishing measurement practices.
What’s Next?
- Interested in Crealytics’ incrementality measurement platform? Learn more here or reach out via our contact form
- Elevate your measurement strategies: follow Andrew Covato (@andrewcovato) and @Crealytics on LinkedIn and join us in advancing your incrementality practices today.
