Session VII (May 16, 1:30pm-3:00pm): Experimental Design in Transportation Studies, organized by Feng Guo 

• Title: Data-Driven Switchback Designs: Theoretical Tradeoffs and Empirical Calibration 

• Speaker: Ruoxuan Xiong, Emory University

• Abstract: We study the design and analysis of switchback experiments conducted on a single aggregate unit. The design problem is to partition the time space into intervals and switch treatments between intervals, in order to minimize the estimation error of the treatment effect. We observe that the estimation error depends on four factors: time heterogeneity, carryover effects, serially correlated outcomes, and impacts from simultaneous experiments. We derive a rigorous bias-variance decomposition and show the tradeoff of the estimation error from different factors. The decomposition provides three insights in choosing a design: First, balancing the time heterogeneity between treated and control intervals reduces the variance; second, switching less frequently reduces the bias from carryover effects while increasing the variance from correlated outcomes, and vice versa; third, randomizing interval endpoints reduces both bias and variance from simultaneous experiments. Lastly, we propose a new empirical Bayes design approach that accounts for all factors by using prior experiments. We illustrate this approach through a case study on a ride-sharing platform, yielding a design that reduces MSE by 33% compared to the status quo design.