Virtual Quantitative Marketing Seminar

Upcoming Talks

• Monday, November 10, Noon ET - Samsun Knight (Toronto)

Measuring Heterogeneity in TV Advertising Elasticities: Evidence from 135 Retail and Restaurant Brands

Abstract: We estimate the heterogeneity of TV advertising effectiveness across store characteristics and advertising levels using a large-scale panel of 135 US retail and restaurant brands, and then use these estimates to assess strategies for improving TV advertising performance. We find that only 48% to 56% of brands exhibit diminishing marginal returns at median advertising levels, suggesting that simply reducing ad expenditure across-the-board may not reliably lead to higher marginal effectiveness. We find significant heterogeneity in TV advertising elasticity across characteristics for over 93% of brands, but show that firms' observed allocations generally fail to fully exploit this estimated heterogeneity and instead covary much more closely with simple heuristics. For example, we find that firms tend to advertise in areas where they already have high revenue, rather than in the areas estimated to have the highest incremental revenue from advertising.  We project that brands could improve ad lift by a median 2.35 percentage points (relative to <0.5% median baseline ad lift) and earn tens of millions in additional revenue under identical-budget reallocations that better leverage this heterogeneity, and that 14-16 percentage points of brands with negative return-on-investment (ROI) from TV advertising could achieve positive ROI through such reallocations.

• Monday, November 17, Noon ET - Ashesh Rambachan (MIT)

Large Language Models: An Applied Econometric Framework

Abstract: How can we use the novel capacities of large language models (LLMs) in empirical research? And how can we do so while accounting for their limitations, which are themselves only poorly understood? We develop an econometric framework to answer this question that distinguishes between two types of empirical tasks. Using LLMs for prediction problems (including hypothesis generation) is valid under one condition: no ``leakage'' between the LLM's training dataset and the researcher's sample. No leakage can be ensured by using open-source LLMs with documented training data and published weights. Using LLM outputs for estimation problems to automate the measurement of some economic concept (expressed either by some text or from human subjects) requires the researcher to collect at least some validation data: without such data, the errors of the LLM's automation cannot be assessed and accounted for. As long as these steps are taken, LLM outputs can be used in empirical research with the familiar econometric guarantees we desire. Using two illustrative applications to finance and political economy, we find that these requirements are stringent; when they are violated, the limitations of LLMs now result in unreliable empirical estimates. Our results suggest the excitement around the empirical uses of LLMs is warranted -- they allow researchers to effectively use even small amounts of language data for both prediction and estimation -- but only with these safeguards in place.

• Monday, December 1, Noon ET - Eric Bradlow (Wharton)