In the latest Humans of Measurement podcast episode, Lifesight’s Rajeev Nair sits down with UC San Diego’s Professor Kenneth C. Wilbur, the newest member of our Scientific Advisory Council, to talk about incrementality washing, the industry grading its own homework, and why better models will never replace a valid control group.
Few people have spent as long looking at how advertising actually works as Professor Kenneth C. Wilbur. Over two decades, his research has covered TV advertising markets, the ad tech supply chain, and the causal measurement of ad effectiveness. He is Professor of Marketing and Analytics at the Rady School of Management, University of California, San Diego. His open-access Introduction to Advertising Measurement has become essential reading for anyone who works on incrementality and causality, on either side of the table.
He is also the newest member of Lifesight’s independent Scientific Advisory Council. That made this episode of Humans of Measurement, a special segment of The Profit and Proof Podcast, a conversation we had been looking forward to for a long time.
Here’s what stood out.
From Software Engineer to Measurement Skeptic
Wilbur studied economics and communication as an undergraduate. He had been coding since he was six, so his first job was as a software engineer. That turned out to be useful preparation for 20 years of technological upheaval in advertising.
In graduate school he was taught a tidy piece of economic theory: companies only buy advertising when they know it will be profitable. The more he learned about how the industry really runs, the more gaps he found between that theory and actual practice. So he moved into marketing, where he could work closer to the data and alongside practitioners. He has since collaborated with more than ten companies on published research.
A Book that is Never Finished And Always Free
Wilbur’s deck is his version of a textbook, with two differences: he revises it constantly and gives it away. He wrote it for three audiences.
The first is the growing group of highly technical people in measurement roles who came from economics, statistics, computer science, physics, or bioinformatics rather than marketing. They need unbiased information about the data-generating process behind the numbers they study. In this industry, that kind of information is surprisingly hard to find.
The second is the marketing and finance leaders who eventually have to agree on how advertising gets measured and funded.
“A lot of marketing executives don’t have the econometrics background to fully understand the measurement techniques. And a lot of finance executives don’t fully appreciate just how challenging and complicated this advertising measurement world is.”
The third audience, by his own admission, is himself. Publishing openly invites practitioners to correct him, challenge him, and add to the work. The deck’s final slide thanks the dozen people whose data, questions, and disagreements have shaped it.
What Incrementality Actually Means
Rajeev asked about a trend everyone in measurement has noticed: incrementality has become fashionable, so nearly everyone now claims to measure it, whether or not a counterfactual is involved.
Wilbur defines it precisely. Incrementality is causal lift: the effect on conversions, from upper funnel through post-purchase, that can be attributed solely to the ads themselves.
That is harder than it sounds. Correlational measurement only needs data. It also mixes two different things together: the effect of the ads, and all the work that goes into putting those ads in front of people who were already likely to respond. Both matter. But you can’t evaluate your targeting and optimization efforts until you separate them from the outcomes those efforts produce.
Where does he see the term misused most often? In marketing mix modeling.
“A lot of people seem to misunderstand that without a valid causal identification strategy, a marketing mix model is fundamentally a correlational advertising measurement tool. And we’ve known this since the 1950s.”
MMM is a valuable tool, he stressed, as long as it is used together with valid causal measurement.
Causality is a Data Problem, Not a Modeling Problem
This may be the most important idea in the episode. One of the key lessons from econometrics over the past 30 to 40 years is the distinction between an identification strategy and a model. Wilbur pointed to work by Stanford’s Guido Imbens, who won the Nobel Prize in 2021 for his contributions to causal inference.
The identification strategy is the logic that guarantees you’re comparing like with like. It might come from random assignment in an experiment or from a valid quasi-experiment in historical data. Once that logic is in place, you can choose among MMM, difference-in-differences, regressions, or neural networks. The model is just how you perform the comparison.
“The model is not magic. It cannot turn an orange into an apple.”
This matters a great deal right now, when many products promise incrementality from observational data alone, with no holdout, no test, and no waiting period, only a better algorithm. As Rajeev noted, research using some of the richest advertising data in the world found that sophisticated machine learning still couldn’t recover true incremental effects without a control condition.
The Industry is Grading its Own Homework
Wilbur’s deck also covers incentive conflicts and principal-agent problems. That theme resonates at Lifesight, because a measurement nobody acts on is only a fashionable metric. Platforms sell the media, deliver it, and then report how well it performed. Agencies have historically been paid on spend instead of lift.
Wilbur notes that this problem isn’t new. What has changed in the last decade is how mature in-house data science has become. For him, the fix is mostly a management question:
“If we set up our marketing team such that they will be punished when they fail, either they will not try new things or they will try to make everything look like a success, whether it was or not.”
When uncertainty is real, teams need room to explore. Companies now have the ability to build cultures of experimentation, so they should reward teams for learning, documenting what they learn, and building on it. The benefits compound: the more you experiment, the better you get at experimenting.
Incrementality is Not Optimality
One of the frontier topics in Wilbur’s deck deserves more attention than it gets. Proving that ads worked is not the same as proving you got the most out of them. For Wilbur, optimality means profit maximization, and he sees two clear routes toward it.
The first is measuring more outcomes. That includes the profitability, retention, and long-term development of the customers a campaign brings in, not just the conversions.
The second is looking at how advertising interacts with other strategic levers. Price promotion is his current favorite example. A campaign’s response depends on the depth of the discount, and it’s entirely possible to set that discount wrong and end up paying to give away your margin. Optimizing advertising and pricing jointly, rather than one after the other, is where much of the untapped profit may be.
Learning Measurement by Making Decisions
To teach all of this, Wilbur built a simulation game. Students set ad budgets across 12 instruments, commission experiments, run a marketing mix model, and watch the game calibrate the MMM’s Bayesian priors to the results of their experiments. He recently ran it with 120 graduate students.
“There’s a visceral reaction when you realize that measurements are only valuable to the extent that they improve our decisions.”
One of his more revealing observations came from students who played alongside AI assistants. The language models filled gaps with assumptions, some played very conservatively, and some simply trusted the attribution numbers reported by vendors. It’s a neat illustration of why understanding causal logic still matters when an AI is helping.
The game is also sharpening Wilbur’s own research. He has two projects under way on how companies should best combine costly experiments with correlational tools like MMM, and under which conditions each combination works best. That question is at the center of what we build at Lifesight. It’s also why Rajeev suggested that a simulator like this could one day be part of the Lifesight Academy.
If you’d like to try the game, Wilbur is happy to share access codes. Just message him on LinkedIn.
Why it’s Worth the Brainpower
Wilbur closed with a reminder that advertising measurement reaches well beyond any one company’s P&L. Advertising funds much of the news, entertainment, and information the economy produces, so how well we measure it affects what gets made and what people get to consume.
His reading recommendations: Building a StoryBrand by Donald Miller for marketers, which he calls remarkably easy to absorb and highly practical, and The Mountain in the Sea by Ray Nayler for a thought-provoking piece of modern sci-fi.
Advertising measurement is a hard problem, and there may be no universal solution. There is plenty of room, though, for honesty, rigor, and better decisions. We’re glad to have Professor Wilbur bringing his skepticism and scientific discipline to our research.
Explore Professor Wilbur’s work and open-access materials at kennethcwilbur.com.
Listen to the full episode on The Profit and Proof Podcast →
Joie Roberts 
You may also like
Essential resources for your success
