The holes are where the money is
When I take over marketing at a large international operation, the first thing I look for is what to cut.
This is truest in the hardest markets to run — the sprawling, populous, multicultural ones like Brazil, Mexico, Indonesia, or Nigeria, where “the market” is not one thing but hundreds.
Countries like these can run to nearly two hundred cities, spread across regions that differ in language, income, infrastructure, and behavior, and where the business itself is not uniform from place to place: different verticals lead in different regions, the marketplace that dominates one city barely exists in another, and demand is shaped by local realities a headquarters spreadsheet never captures.
Every time I’ve arrived to lead marketing in a market like this, I’ve found the same thing — spend spread across all those cities, and almost none of it distributed in proportion to where it actually produces revenue. Some cities are genuinely efficient places to acquire users and are starved; others absorb budget for years out of habit, momentum, or the simple fact that no one ever turned them off.
And the first real thing I do is unglamorous: I start switching campaigns off, in Google, in Meta, wherever the spend is running without earning its place, and I watch what actually happens to the business.
This is the whole game, and it is worth stating plainly because it is so often buried under dashboards and channel jargon: marketing efficiency is an allocation problem.
You have a finite amount of money, and its value depends entirely on where it goes. Move it out of the cities and channels where it’s wasted and into the ones where it compounds, and you won’t see any loss in business, but will observe efficiency.
But here is the catch, and it is the reason this is a genuinely hard technical problem rather than a matter of common sense: to reallocate money toward what works, you have to know what actually works — and that is far harder than it looks. The obvious signals lie to you.
Start with the simplest entanglement. Paid acquisition and organic traffic look like separate lines in a budget, but they bleed into each other. A share of the users who arrive through a paid ad would have found the product anyway; the paid channel captured demand that already existed and took the credit. Cut the paid campaign and you often discover organic quietly rises to absorb much of what you thought paid was producing. The same is true of lifecycle messaging — push notifications, banners, in-app prompts — which cannibalize each other and the other channels in exactly the same way. The naive fix, in every case, is to credit whichever touch came last before the purchase. It is easy to implement and almost always wrong, because proximity in time is not causation. Large global marketplaces such as Amazon face this at extreme scale: a customer receives so many messages that something always precedes any given order, and crediting that last message confuses “after” with “because.”
So the efficiency problem and the measurement problem are the same problem wearing two faces. You cannot allocate well if you cannot see true incremental value — and true incremental value is obscured by channels that overlap, by demand that would have converted anyway, and by metrics that reward the last touch instead of the real cause. Everything that follows is about seeing through that fog.
Which efficiency question you’re even asking depends on the market
Before choosing any measurement method, you have to know which question the market is actually posing — because it changes as the business matures, and I’ve watched good analysts reach confident wrong answers just by asking the wrong one.
On a young or fast-growing market, user acquisition is the business. New users make up a large share of everyone active, so almost anything you do to acquisition shows up in the topline. In that world the crude signals — did revenue grow this month, did it grow versus last year — genuinely tell you something, because acquisition is moving the whole business at once.
On a strong, established marketplace, that logic quietly collapses. New users are no longer the story; the business runs on the existing base’s retention – how often established users transact. Acquisition becomes a small tributary feeding a large river. And this is where a simple, unforgiving piece of arithmetic takes over: direct-effect measurement only works when new users are a large share of the active base — roughly half or more.
When new users are a small fraction of your monthly active users, a marketing change to that fraction is invisible against the mass of existing-user activity. To move the topline visibly you would have to roughly double acquisition efficiency; anything short of that disappears into the noise.
The numbers make the point concrete. In a mature market, new users might be on the order of ten percent of the active base; in a less-established one, perhaps a quarter to a third; only in genuinely new markets does the share climb high enough that acquisition alone drives visible growth.
So on established markets — where most of the money and most of the stakes actually sit — the direct signals are structurally blind. Not merely noisy: blind. The effect you’re trying to measure is real, but it is too small a share of the whole to register in month-over-month or year-over-year movement.
This is the reason the toolkit has to escalate. It is not that analysts enjoy complexity. It is that once new-user share drops below the threshold where direct effects are visible, you are forced to methods that can isolate a small incremental signal from a large, moving baseline — methods that can see the retention and reactivation effects the topline hides, and that can attribute movement to a source rather than to the calendar. The crude metrics are the young-market tools. Everything sophisticated that follows exists because mature markets took those tools away.
The direct signals — and why even they fail on their own terms
The direct growth metrics are the tools of the young market, where new users are a large enough share of the base to move the topline. But even inside that home territory, each one has a failure mode you have to know, because reaching for the wrong one is how a confident reallocation goes wrong.
The crudest is month-over-month growth: revenue rose this month versus last, so credit the change you made. But a calendar month contains far more than your decision — seasonality, paydays, holidays, weather, competitor moves, price changes. In large, multicultural markets that noise is enormous: a religious holiday or a regional festival reshapes demand for reasons that have nothing to do with your allocation.
Month-over-month cannot separate your effect from any of it. It is least misleading precisely where new-user share is highest and the signal is large relative to the noise — but even there it is correlation in time, not proof of cause.
Year-over-year is the usual correction, since comparing a month to the same month a year earlier holds the season roughly constant. But it collapses in exactly the markets where acquisition matters most — the young ones. If a city or a vertical barely existed a year ago, year-over-year reads as several hundred or several thousand percent: technically true, useless for deciding where the next dollar goes.
There is a more refined move: compare the dynamics rather than the levels — this period’s growth rate against the same period’s growth rate a year earlier. If a month grew modestly over the prior month last year but grew far faster over the prior month this year, the ratio of those two growth rates shows the pace itself has accelerated, stripped of the recurring seasonal shape and of the base-effect distortion that breaks raw year-over-year.
It is a second-order signal — the growth of growth — and it genuinely rescues you where cruder comparisons fail. But it has its own breaking point: it assumes last year’s seasonal shape repeats, and it turns unstable when the prior-year growth it divides by is very small, because then the ratio explodes.
None of these is worthless; each is the right rough instrument in some situation and a trap in another. But two limits are fatal for the allocation problem specifically.
First, as the previous section established, they go blind the moment new-user share falls below roughly half the active base — so on mature markets they simply cannot see your effect. Second, even when they can see it, not one of them tells you which channel or city produced the movement.
They measure the whole business at once. And an allocation decision is precisely a question about sources — move money from where to where — which the topline can never answer. That is what forces the step up to a model.
The counterfactual: measuring against what would have happened anyway
The direct signals fail on mature markets because they compare the business to its own past, and the past is contaminated — by season, by trend, by everything that would have happened with no marketing at all. The way through is to stop comparing periods to each other and start comparing reality to a forecast of what would have happened without the intervention.
According to Say Agency’s marketing mix modeling overview, MMM helps identify opportunities for budget reallocation and evaluate channel efficiency, while Improvado’s 2026 MMM guide describes MMM as a regression-based technique that uses historical spend, impressions, sales, and external factors to estimate how each channel contributes to business outcomes.
Conceptually, that is the counterfactual move this section needs: you take the history, decompose it into its underlying components — trend, seasonality, and the organic baseline the business produces with no active marketing — and project the path it would have followed if you had changed nothing. That projection is the counterfactual. Make your reallocation, and the gap between what actually happened and the baseline is the incremental effect: what your money bought over and above the growth that was already coming.
Here it’s worth being precise about which tool you actually need, because that itself is an efficiency decision — and the line is finer than it first appears. Some of this genuinely is spreadsheet work. A moving average or an exponential moving average will smooth a noisy series enough to see a trend and spot a deviation from it.
According to Wikipedia’s advertising adstock entry and Fusepoint’s adstock explainer, adstock captures advertising carryover: the prolonged, lagged, and decaying effect of advertising after exposure or after a campaign ends. Even that carryover can be applied in a spreadsheet if you already know the decay rate: it’s a single recursive column, each period retaining a fraction of the last. But that is the trap. Applying a known decay is arithmetic; finding the right one is not.
As per Improvado’s 2026 MMM guide, actual channel parameters are estimated from the data, and decay rates typically differ by channel. In a real model you do not guess the decay rate, or the point where each channel saturates — the model estimates them from the data, searching for the parameter values that best fit history.
And it does this for every channel at once: each channel’s carryover, each channel’s saturation curve, and each channel’s contribution, all disentangled simultaneously from one another and from the underlying baseline and seasonality.
That is a many-parameter nonlinear fit, not a formula you drag down a column. A spreadsheet can apply a transformation someone handed you; it cannot discover the right transformations for a dozen channels competing to explain the same sales. That is the real boundary between spreadsheet and model.
And this counterfactual, however well built, still stops exactly where the allocation decision begins. It tells you whether your money moved the business, and roughly how much. It does not tell you which part of your spend did the moving — no split by channel, no split by city, no return you can compare across options.
You learn the intervention worked; you learn nothing about where the next dollar should go relative to the last. To decompose the effect into sources, and to compare returns at the margin, you need the full model.
The full model: decomposition, and the difference between average and marginal return
Everything so far tells you whether marketing worked. A marketing mix model tells you which part worked, and by how much — which is the first time you have something you can actually reallocate against.
According to Improvado’s 2026 MMM guide, MMM uses aggregated historical data and regression to estimate how each marketing channel contributes to outcomes, applying adstock and saturation transformations to isolate incremental contribution. According to Say Agency’s MMM overview, this kind of analysis helps marketers evaluate marginal effectiveness and optimize budget allocation.
The mechanics, at the level worth understanding: the model takes the outcome you care about — revenue, transactions, new users — and the spend on each channel, and uses regression to separate how much of the outcome each channel drove, on top of the baseline that absorbs trend, seasonality, and organic demand.
Two transformations make it more than naive regression, and both were invisible to every method before this one. According to Fusepoint’s adstock explainer, adstock helps account for delayed consumer responses and the lingering effects of marketing; according to Wikipedia’s advertising adstock entry, it is a model of how advertising response builds and decays over time. The second transformation is the saturation curve, and this is the heart of the whole thing.
Saturation is the formal statement of a fact every operator feels intuitively: more spend does not buy proportionally more result. According to Improvado’s 2026 MMM guide, doubling paid search spend does not double conversions once you are already bidding on the available relevant queries, and the saturation shape determines marginal ROI and the direction of budget reallocation.
According to Wikipedia’s advertising adstock entry, a linear increase in advertising exposure does not create a similar linear effect on demand. The curve is steep at low spend and flattens as you push higher, because you have already reached most of the addressable audience; beyond a point you are just paying to hit the same people more often.
As per Qonto’s MMM series on Medium, an S-shaped saturation model introduces a threshold, or the idea that some minimum level of marketing activity must be reached before returns appear. That threshold idea rhymes with the new-user-share floor from earlier: below a certain level, effort simply does not register.
Now the single most important point in this entire piece, and the reason the model exists at all. Last year’s average return on a channel tells you almost nothing about where the next dollar should go. A channel that returned three-to-one on average may be deep on the flat part of its curve, where the next dollar returns far less; a channel that looks worse on average may still be on the steep part, where the next dollar returns more.
Allocation is a question about the margin — the return on the next dollar moved — and the average return, which is all the simpler methods can ever give you, is the wrong number for it. This is why decomposition alone isn’t enough and saturation is essential: reallocation means moving money out of channels flattened on their curves and into channels still climbing theirs, until the marginal return is level across all of them. That balancing point — not “cut the low average-ROI channel,” but “equalize the marginal return” — is what optimal allocation actually means.
And here the “which tool” question resolves cleanly. This is unambiguously model territory, not spreadsheet territory — because estimating every channel’s decay, every channel’s saturation curve, and every channel’s contribution simultaneously against a shared baseline is a many-parameter nonlinear fit, not a formula.
But the model has a price that decides whether you should reach for it at all: it needs a long, clean spend-and-outcome history — typically a couple of years or more — before its estimates can be trusted, plus the analytical capability to build and maintain it. On a young market, or a small budget, or a city that launched last quarter, that history doesn’t exist, and the model will happily fit noise.
There the simpler methods aren’t inferior approximations of the model — they are the correct tool, because they’re the ones the data can actually support. Matching the method to your data maturity is the same discipline as matching your spend to where it works.
When the model isn’t enough: proving cause with experiments
A model, however sophisticated, has one weakness it can never fully escape: it learns from correlations in history. It can tell you the shape of a channel’s saturation curve, but that shape was inferred from the past, and the past is full of confounders — you spent more on a channel in exactly the months demand was rising anyway.
Correlation is not the cause, and a reallocation decision is a causal claim: if I move this dollar, the outcome will change. To trust that, at some point you have to stop modeling and run an experiment.
This is a different tool, not a better model — and it’s worth being clear about that, because it sits at a distinct rung of the ladder. The direct signals were spreadsheet work.
The counterfactual baseline was light modeling. The full model was heavy modeling. Experiments are none of those: they are deliberate interventions in the real world, designed to isolate cause by construction rather than infer it from data.
The cleanest form is a geo-experiment. You vary spend in some geographies and hold comparable ones as controls, then measure the difference in outcome between them. Because the only thing you changed between test and control was the spend, the gap between them is the causal effect — no model required to disentangle it.
According to PyMC Labs’ discussion of MMM calibration with lift tests, lift tests serve as a reality check because they ground model predictions in real-world outcomes and help address unobserved confounders. This is also, increasingly, how the best practitioners keep their models honest: the published frameworks now fold experimental results back into the model, using lift-test outcomes to calibrate its parameters rather than trusting the regression alone, and using a time-varying baseline when the market itself shifts underneath you.
As per PyMC Labs’ complete MMM guide, companies such as HelloFresh and Bolt have already put Bayesian MMM into production for real budget decisions. The major platforms and a number of large marketplaces have moved in exactly this direction — experiments and models used together, each correcting the other’s blind spot.
The whole thing lives or dies on one design choice: the comparability of test and control. The geographies have to be genuinely alike — similar in size, in population, in the structural balance of the market — because if they differ, the difference you measure is the difference between the places, not the effect of the spend.
Compare a megacity to a small town and you’ve measured nothing about your money. Choosing the geographies well is most of the work, and it is harder than it sounds precisely in the large, multicultural markets from the opening, where two cities in the same country can behave like two different countries.
Sometimes the cleanest experiments aren’t even designed in advance — conditions in live operations line up so that one region can serve as a natural control for another, and the skill is recognizing that usable comparison when it appears, rather than only running tests you planned.
One honest note on how far this goes. The frontier of this field is formalizing the loop — feeding experimental evidence into models as statistical priors, so causal results discipline the correlational estimates.
According to PyMC Labs’ lift-test calibration post, Bayesian methods allow prior knowledge to be incorporated into the model, and PyMC-Marketing can add lift-test measurements through custom likelihood functions on saturation curves rather than relying on ROAS priors alone. That direction is real and worth knowing as the state of the art.
But the core practical point stands without any of that machinery: a model tells you what probably happened; an experiment tells you what actually did; and allocating serious money demands both. Neither the spreadsheet, nor the model, nor the experiment is sufficient alone — the discipline is knowing which one the decision in front of you actually requires.
The frontier: where every method above goes blind
Everything to this point — the spreadsheet, the baseline, the model, the experiment — shares one hidden assumption: that marketing acts on demand, demand becomes transactions, and measurement’s job is to trace that line. In a two-sided marketplace, that assumption quietly breaks, and it breaks in a way none of the tools can see. This is the part the literature barely touches, and it is where the real difficulty of the job actually lives.
In a marketplace, generating demand is only half of a transaction. A campaign can succeed completely at the top of the funnel — bring users in, get them to the point of placing an order — and still produce zero incremental revenue, because the order has to be fulfilled by the other side of the market.
A user arrives, sees a price, requests a service; whether that request becomes a transaction depends on the other side being there to meet it. If that side can’t or won’t fulfill the demand you generated, the demand simply evaporates — and your genuinely effective campaign looks like a failure in every model you have, because the model sees spend going up and revenue not following.
This confounding runs in both directions, which is what makes it so hard to handle.
On the downside: a shock on the supply side that has nothing to do with marketing can sink your measured results. When a cost shock hits the providers a marketplace depends on, they pull back; the available supply contracts; the orders your campaign generated go unfulfilled; revenue falls during your campaign.
No model attributes this correctly, because it has no variable for “the other side of the market pulled back for reasons of its own.” It sees spend up, revenue flat, and concludes the campaign failed. The model can learn recurring, seasonal shocks — holidays, festivals — because they repeat and history teaches them. It is structurally blind to a novel shock that changes one side’s behavior for the first time.
On the upside, the same confounding hides in reverse, and this is where the retention thread from earlier comes back. Suppose the business improves retention on one side of the market through some operational change, so there is more of that side available, fulfilling more demand, and the whole funnel grows.
That growth is real, but it is not a marketing effect, and yet it lands in the same time window as your spend and gets tangled up with it. On a mature market this is exactly the regime flagged at the start: new-user acquisition is a small share of the base, the business runs on retention and reactivation, and the movements that matter most are precisely the ones your marketing models are least equipped to attribute.
Credit that growth to media and you’ll over-invest; credit it to the operational change and you may starve a channel that was genuinely working.
This is why, past a certain point, marketing measurement stops being a marketing problem at all. The marketing model can tell you what happened above the funnel. Whether that demand became a completed transaction depends on fulfillment — on the other side of the market, on execution, on the operational health of the marketplace — which lives with a different team and a different set of models entirely.
Neither view is complete alone: the marketing model is blind to fulfillment, the operational model is blind to demand generation, and the true incremental effect of your money lives in the overlap the two rarely measure together.
There is no tool on the ladder for this. It is a cross-functional problem wearing the costume of a measurement problem, and mistaking one for the other is how sophisticated teams reach confident, wrong answers.
There is no final method — only the right one for the situation
Marketing measurement is not a single technique to be mastered and then trusted. It is a ladder of tools, each one blind exactly where the next one sees, and the skill is knowing which rung the decision in front of you actually requires.
Simple growth comparisons work when new users are a large share of the base and fail silently when they aren’t. The counterfactual baseline can surface a small incremental lift a topline would hide, but it can’t tell you which channel produced it.
The full model decomposes by channel and — through saturation — tells you the one thing that actually governs allocation: not what a channel returned on average, but what the next dollar into it will return. Experiments prove cause where the model can only infer it.
And beyond all of them, in a two-sided marketplace, the demand you measure so carefully can still die at fulfillment, where none of these tools can follow it.
The efficiency you’re chasing — more customers, more transactions, more revenue from the same budget — is never won once. It comes from matching three things at every step: the tool to the market’s maturity, the spend to where it actually compounds, and the measurement to what it can honestly see.
A spreadsheet used where a model is needed will mislead you; a model built where the data can’t support it will fit noise; an experiment skipped where causation matters will leave you confidently wrong. The discipline is not sophistication for its own sake. It is refusing to ask a tool a question it cannot answer.
And here is the turn that makes all of this worth doing. When measurement is honest, its failures are not dead ends — they are directions. If you find that a campaign works at the top of the funnel but the effect dissolves before it becomes revenue, you have not just failed to measure something — you have located a problem.
The demand is real and the marketing did its job, so the leak is downstream, and in a marketplace it almost always sits in one of three places. It may be the product — people arrive, but the experience doesn’t carry them to the point of transacting. It may be the other side of the marketplace — the demand exists, but there aren’t enough of whoever has to meet it: sellers on a commerce platform, hosts on a booking platform, couriers on a delivery platform, providers of whatever the service is.
Or it may be the balance between the two sides — the pricing, the matching, and the incentives that keep supply and demand in equilibrium, which when they slip cause perfectly good demand to go unmet. What looks like a marketing result that failed to convert is, read properly, a diagnosis pointing at the product, at the other side of the market, or at the balance between them.
This is the real job, and it is why measurement matters beyond the marketing team.
The marketer who can only report ROAS is replaceable. The one who can look at a top-of-funnel win that didn’t convert and say “the marketing worked — the constraint is on the supply side of the market in these regions,” or “the demand is real, but pricing is choking the match between the two sides,” is doing something the organization cannot get anywhere else: using the marketing signal as an instrument to find where the whole system is breaking, and then stressing that to the team that owns it.
That is where a marketing function stops being a spend center and becomes a diagnostic layer for the entire business.
So the honest state of the field is not a solved science but a discipline of seeing clearly — matching the tool to the question, refusing to ask a method what it cannot answer, and treating every measurement failure as a signal about where the real problem lives.
The most advanced measurement available still stops at the edge of the marketplace. But an operator who understands that edge doesn’t stop there — they follow the signal across it, into product, into the other side of the market, and into the balance between them, which is where the next real gain in efficiency was always going to come from.
