Booking.com ML Meetup #1 

Tackling Interference in Promotion Optimisation 

      Bram van den Akker | Senior Machine Learning Scientist - Booking.com

Uplift modelling is central to promotion optimisation, but its effectiveness is often limited by the Stable Unit Treatment Value Assumption (SUTVA). This assumption, where promoting one product does not affect another’s conversion, is frequently violated in practice due to product interference and cannibalisation. Ignoring these effects leads to suboptimal promotion strategies that merely cannibalise conversions instead of generating true incremental lift. Unfortunately, addressing interference in uplift modelling is a non-trivial challenge with limited coverage in existing literature. This talk provides a comprehensive, practical walkthrough to how we at Booking.com approached the problem of interference, by

• Creating synthetic interference simulations to model complex interactions.

• Creating interference-aware Qini curves for offline evaluation.

• Combining state-of-the-art interference estimation techniques with existing uplift modelling methods.

Attendees will leave with the technical tools to implement these solutions and key insights from the successful academic collaborations that made them possible.

Experimentation that Prioritises Causal Decision-Making 

Olivier Jeunen | Principal Research Scientist - Aampe

Consumer applications provide ample opportunities to surface and communicate various forms of content to users. From promotional campaigns for new features or subscriptions, to evergreen nudges for engagement, or personalised recommendations; across e-mails, push notifications, and in-app surfaces. The conventional approach to orchestration for communication relies heavily on labour-intensive manual marketer work, and inhibits effective personalisation of content, timing, frequency, and copy-writing. We formulate this task under a sequential decision-making framework, where we aim to optimise a modular decision-making policy that maximises incremental engagement for any funnel event. Our approach leverages a Difference-in-Differences design for Individual Treatment Effect estimation, and Thompson sampling to balance the explore-exploit trade-off. We present results from a multi-service application, where our methodology has resulted in significant increases to a variety of goal events across several product features, and is currently deployed across 150 million users.