Humans of Measurement: Navigating Modern Marketing Measurement with Professor Koen Pauwels

For years, marketers relied on hyper-granular user tracking to prove their worth. But as privacy changes restrict data and CFOs demand real business impact, the old playbook is dead. In this episode of the “Humans of Measurement” segment, Rajiv Nayer sits down with Dr. Koen Pauwels—Distinguished Professor, former Amazon Ads expert, and Editor-in-Chief of the International Journal of Research in Marketing.

They discuss why unified measurement is winning, how open-source Marketing Mix Modeling (MMM) is leveling the playing field, the dangers of AI “cognitive surrender,” and how to combine historical data with real-world experimentation to uncover true incrementality.

The End of the “Creepy Tracking” Era

Individual user tracking is not just becoming obsolete due to privacy laws; according to Prof. Pauwels, it has always been fundamentally flawed. It is often “creepy,” distracting, and inherently lacks a holistic view since it completely ignores massive offline drivers like TV, out-of-home advertising, and word-of-mouth.

Today, financial leaders are actively rejecting vanity ROI metrics. They don’t want to see how many cookies were tracked or how many clicks were attributed; they want to know true incrementality—what marketing spend actually drove net-new business.

The Pitfalls of AI, Open-Source, and “Vibe Coding”

AI and open-source software have drastically accelerated data ingestion for MMM, reducing timelines from quarters to mere weeks or days. This massively lowers the barrier to entry, allowing smaller teams to build models and gain organizational buy-in.

However, Pauwels warns against “cognitive surrender.” If you rely purely on automated code generation or open-source defaults without applying human business context, you risk building “misspecified models.” AI is fantastic for routine maintenance and improving efficiency for experienced analysts, but human expertise remains strictly essential to map out KPIs, understand customer decision pathways, and provide strategic direction.

Attribution vs. Incrementality: Behavior Does Not Equal Intent

Many marketers confuse digital activity with buying intent. Prof. Pauwels points out that while attribution metrics provided by ad platforms (like Google or Meta) are highly useful for operational micro-decisions—such as comparing which ad creative performs best within a specific channel—they cannot establish true cross-channel incrementality.

Case in point: Pauwels cites a Microsoft study revealing that 55% of website visitors were actually existing product owners seeking customer support, not prospective buyers. Relying purely on click-based attribution would erroneously credit digital ads for “driving” these visits.

The Synergy Effect

Marketing channels should not be measured in isolation. Pauwels explains the concept of positive synergy (where 1 + 1 = 3). For example, TV advertising might create broad upper-funnel interest, which in turn makes lower-funnel retail media ads much more effective at converting those shoppers. Demonstrating this positive synergy is crucial for justifying overarching marketing budgets to finance teams and proving that channels are working together rather than cannibalizing each other.

The Framework: “Model-Experiment-Model”

Organizational risk aversion is the biggest bottleneck to smart budget allocation. Marketers are terrified to turn off campaigns. To overcome this safely, Pauwels and Nayer recommend the “Model-Experiment-Model” framework:

1. Model: Use historical MMM to identify potential inefficiencies. (For example, your model might suggest that your branded paid search is yielding zero incremental sales.)

2. Experiment: Conduct a controlled field experiment. Turn the channel off entirely in a small, low-risk geographic region (e.g., affecting just 5% of revenue for 3 to 4 weeks) to establish an actual baseline. Alternatively, use a testing platform to shift budgets (-50% in one cell, +150% in another).

3. Model: Re-run your MMM using this clean, varied experimental data to extract deeper, highly validated insights that you can confidently present to leadership.

The Rule of Analytics

“If you’re not willing to change any decision, then my ROI is zero.”

– Prof. Koen Pauwels on the true purpose of marketing analytics

Actionable Steps for Marketing Leaders

If you want to transition your team toward unified, causal measurement, start here:

1. Start with the Decision: Before touching any data, opening any software, or selecting metrics, map out the specific business decisions you need to make and the customer pathways you are trying to influence. Work backward from the decision to the data.

2. Accept Internal Definitions: Don’t get paralyzed waiting for the industry to agree on standard definitions for terms like “halo effect,” “baseline shifts,” or “ROMI.” Establish a consistent internal definition that your specific company agrees on, and stick to it.

3. Embrace Geo-Testing: Stop guessing. Take a small, low-risk region and go “completely dark” on a specific high-spend channel for a few weeks to prove its true baseline incrementality.

About the Guest & Recommended Reading

Dr. Koen Pauwels is a Distinguished Professor and Associate Dean of Research at Northeastern University. He brings a wealth of practical and academic experience, having worked as a startup entrepreneur in Belgium and spending five years at Amazon Ads in New York