Beer Advertising and Retail Metrics Challenges in the Post-Pandemic Era: The New E-commerce Boom

Online sales of beer have grown exponentially since the pandemic. As a result, there is growing advertising investment by the big players in the e-commerce space. With this comes additional demand to measure the return on that advertising spend to maximize media performance. However, there are several challenges to be kept in mind.

First, there is a lack of standardization on data-sharing across retailers. Within the same market, different retailers may share data at different granularities, frequencies, or formats. Some might not share impressions, clicks, conversions, etc., which come under digital marketing KPIs. Even in cases where similar metrics are shared at similar granularity, the method of calculation may be different. For example, one retailer might attribute all sales in a 7-day window to an ad, while another uses a 14-day window, thereby making the comparison of the same KPI among them invalid.

The second challenge revolves around the nature of the metrics and insights being shared. Commonly, retailer reports dwell on campaigns in isolation. Compiling these reports and insights of many campaigns will be extremely time-consuming. Retailers also usually use ROAS as a KPI. Besides those various calculation methods mentioned above, ROAS also tends to be an overinflated KPI in that it is focused on simple attribution and not true incrementality for an ad. For instance, there are regular consumers of a product who simply clicked on the ad for convenience-easy to reach location on website or app-or because of the discount.

This article focuses on a methodology that addresses these challenges, enabling the CPG firms to standardize their retail media performance measurement across markets, retailers, brands, and even SKUs.

Hypothesis Testing 

Various drivers of online sales were identified and their corresponding metrics & data sources were gathered. Hypotheses on the impact of these factors were formed and a combination of significance testing (T-tests) and correlation analysis was used to generate the following findings:

1. Distribution is the largest driver of sales – Product availability & high share of shelf drives sales
2. Both top-of-funnel media (TV, Facebook, Youtube ads etc) and eCommerce media have a significant short-term impact on sales
3. Price has a significant impact on sales. This includes own brand price, competitor price and discounts
4. Major events such as COVID lockdown dates, and retailer events such as Prime Day & Black Friday have a significant local impact on sales
5. There are seasonal patterns of rise & fall in sales for various brands that needs to be accounted for

Model Selection

There is no direct way to trace the spends on advertising and attribute it to sales uplift directly which necessitated the use of a statistical model. Guided by research and previous use cases, the market mix model was chosen to solve this problem.

Model Customization in Business Context

Step 1. Multiplicative Model

A conventional additive model assumes impact of every explanatory variable on response variable to be independent of other explanatory variables. Impact of distribution, price and advertising is mashed with each other in reality. Thus, a multiplicative model was chosen.

Step 2. Mixed Effects Hierarchical Model

In many cases there may exist relationships across levels of the data that need to be captured even though this can show up in a typical model as multicollinearity. To that effect, a hierarchical model is used to capture relationships between predictors.

For example:

Price has a significant impact on sales, but can vary across retailers, or by pack sizes. Assume that shoppers on Retailer 1 are more price inelastic compared to shoppers on Retailer 2, or that consumers are more responsive to a discount on a 24-pack compared to a 4-pack.

This model has two kinds of effects: fixed effects and random effects. A fixed effect captures the overall effect across the groups of the variable, for instance, the effect of a specific format on overall sales, say homepage ad. And the random effect would be able to capture the subject-specific impact, let’s say, the impact due to the homepage ads by retailer or by pack size. A model that possesses both the fixed and the random effects would be the mixed-effects model.

Multilevel or hierarchical modeling has been used in the analysis of data with a hierarchical structure. It is particularly useful in cases where the assumptions made for a linear regression are violated or in cases where we hold an interest in checking how group-level factors influence the individual-level outcomes.

This also enabled us to scale the model to an extent that had never been achieved earlier. Typically, MMM models would have to be built for each retailer-brand combination, which, in this case, would have been time-consuming, labor-intensive, and expensive in terms of resources required. However, hierarchical modeling is allowed for market level models to be built with ~10 retailers and brands in each market.

Step 3. Ad-stock or Carryover Effect

One of the features of advertising is its lagged impact. There is some effect of the advertisement that comes out a few days after it goes on air. More precisely, I can see a TV ad today and make a purchase a few days after. We will capture this phenomenon using an ad-stock or carryover transformation on our media. Typically, video ads have longer ad-stocks than image ads.Where N₀ is the true value of media spend/causals at the time the campaign aired;

t is the amount of time since the campaign aired, usually measured in weeks;

λ is the half life i.e. time taken for media to decay to half of its initial impact. λ of 1 means that media impact of 50% after 1 week, for instance. For most of the offsite media, adstock has been observed to be between 1 and 3 weeks. Since E-retail Media was majorly conversion focused we took a mid value of 1-2 weeks for it;

N(t) is the impact of media after t time.

Step 4. Saturation

The other consideration would be decreasing media return. For example, buying 100 GRPs of television in a week may gain 100L additional volume for a brand. But that does not mean buying 1000 GRPs will push 1000L because at some point, we would be bombarding the same audience with the same message repeatedly. There is an upper saturation limit. Similarly, buying 1 GRP is virtually akin to buying no GRPs since practically nobody sees the ad. Also, there is a bottom saturation limit as well. To capture this, an S-curve is fitted to each media type.

Testing & Validation

The series of tests below have been administered for the stability and accuracy of the model.

1. eRetail media ROI benchmarked vs Retailer reported ROAS. Since ROAS is attributed sales, ROI could not be greater than ROAS in any case. This was true in the model output.

2. Model volume-contribution break-up validated with the results of prior hypotheses tested and business validation

3. Ran a few example cases to ensure attribution was done appropriately: For example, the spike in sales during COVID did not artificially overinflate media ROI, and attribution to the movement regarding the drop in sales in the UK during Dry January was correct.

4. Stress testing by removing levels or time periods to see if the results were directionally consistent.

This model helped retail account managers measure performance at the country, region, brand, SKU, retailer, ad format levels, or any other combination of these levels provided adequate investment was available. This would provide for data-driven decision making in eRetail media planning and improve sales and ROI from eRetail advertising.