Implicit feedback (e.g., user clicks) is an important source of data for modern search engines. While heavily biased, it is cheap to collect and particularly useful for user-centric retrieval applications such as search ranking.

To develop an Unbiased Learning-To-Rank system with biased feedback, previous studies have focused on constructing probabilistic graphical models (e.g., click models) with user behavior hypothesis to extract and train ranking systems with unbiased relevance signals.

Recently, a novel counterfactual learning framework that estimates and adopts examination propensity for unbiased learning to rank has attracted much attention.

This tutorial provides an overview of the fundamental mechanism for unbiased learning to rank, and a systematic comparison of the unbiased learning-to-rank frameworks based on graphical models and counterfactual learning. It describes the theory behind existing frameworks, and give detailed instructions on how to conduct unbiased learning to rank in practice.