Bayesian Optimization for Balancing Metrics in Recommender Systems
International Joint Conference on Artificial Intelligence (IJCAI-PRICAI) 2020, Yokohama
Abstract
Most large-scale online recommender systems like newsfeed ranking, people recommendations, job recommendations, etc. often have multiple utilities or metrics that need to be simultaneously optimized. The machine learning models that are trained to optimize a single utility are combined together through parameters to generate the final ranking function. These combination parameters drive business metrics. Finding the right choice of the parameters is often done through online A/B experimentation, which can be incredibly complex and time-consuming, especially considering the non-linear effects of these parameters on the metrics of interest.
In this tutorial, we will talk about how we can apply Bayesian Optimization techniques to obtain the parameters for such complex online systems in order to balance the competing metrics. First, we will provide an in-depth introduction to Bayesian Optimization, covering some of the basics as well as the recent advances in the field. Second, we will talk about how to formulate a real-world recommender system problem as a black-box optimization problem that can be solved via Bayesian Optimization. We will focus on a few key problems such as newsfeed ranking, people recommendations, job recommendations, etc. Third, we will talk about the architecture of the solution and how we are able to deploy it for large-scale systems. Finally, we will discuss the extensions and some of the future directions in this domain.
Original proposal
Complete Presentation (IJCAI website)
Schedule
Date: 4:00 -- 7:00 , Jan 6th 2020
Introduction to Bayesian Optimization (Part 2)
Reformulating a Recommendation System Problem (Part 1, Part 2)
Infrastructure (link)
Extensions of Bayesian Optimization
Materials
Please download the slides here at SlideShare
https://www2.slideshare.net/ViralGupta2/ijcai-2020-240992740
Presenters
Dr. Kinjal Basu is a Staff Applied Researcher at LinkedIn focusing on broad Machine Learning/AI algorithms for several applications and leads the efforts on Bayesian Optimization at LinkedIn. He received his Ph.D. in Statistics from Stanford University and has several published papers in many top journals and conferences such as The Annals of Statistics, Foundations of Computational Mathematics, SIAM Journal of Numerical Analysis, NeurIPS, KDD, etc. He has been serving as a reviewer and program committee member in multiple top journals and conferences such as NeurIPS, ICML, KDD, WWW, etc.
Dr. Cyrus DiCiccio is a software engineer at LinkedIn, where he works on Bayesian Optimization as well as other machine learning algorithms. He received a Ph.D. in Statistics from Stanford University where he focused his research on non-parametric statistics, econometrics, data privacy, and high-dimensional variable selection. His work has been published in top journals such as the Journal of the American Statistical Association. He has also served as a visiting assistant lecturer at Stanford University where he has taught courses on Statistics and Data Science.
Dr. Brendan Gavin is a software engineer at LinkedIn, where he builds and improves relevance libraries for online notifications. He has a background in high performance numerical algorithms and equation solvers, and has published articles in leading journals such as Numerical Linear Algebra with Applications, the Journal of Computational Science, and the Journal of Chemical Physics (Google Scholar). He previously served as the teaching assistant for the undergraduate Probability and Stochastic Processes course at the University of Massachusetts Amherst Electrical and Computer Engineering department.
Viral Gupta works as a Relevance Tech Lead for near real-time Notifications recommendation on the LinkedIn platform. He works on several problems including onboarding new notification onto the Relevance platform efficiently, improving the overall quality of the recommendations by incorporating multi-objective optimization, etc. He is one of the key players who designed and implemented the scalable hyper-parameter optimization library at LinkedIn. He spends time consulting several teams at LinkedIn on how can the parameter search needs for the specific use-cases be formulated as a multi-objective optimization problem and solved using the hyper-parameter optimization library.
Dr. Yunbo Ouyang is a software engineer at LinkedIn who has expertise and works extensively on automatic hyperparameter tuning. He is one of the key players building LinkedIn’s offline and online hyperpameter tuning libraries. He obtained his Ph.D. in Statistics from University of Illinois at Urbana-Champaign. He has published papers in top conferences such as SIAM International Conference on Data Mining. See List of publications. He has been serving as a reviewer for multiple top conferences such as NeurIPS, KDD and AAAI. He has taught and TAed multiple advanced statistics and machine learning courses in UIUC. For a full list of courses, see Courses Taught.