This is the companion web page for the paper:

Information Retrieval Evaluation Measures Defined on Some Axiomatic Models of Preferences. Fernando Giner. ACM Transactions on Information Systems, 42, 3, Article 69, 35 pages, 2023, (issue date: May 2024), https://doi.org/10.1145/3632171

Abstract

Information retrieval (IR) evaluation measures are essential for capturing the relevance of documents to topics, and determining the task performance efficiency of retrieval systems. The study of IR evaluation measures through their formal properties enables a better understanding of their suitability for a specific task. Ranking axioms, heuristics or desirable properties of retrieval measures can be expressed with pairwise comparisons on documents, leading to preference or ordering relationships on the set of ranked lists of documents. Each of these orderings constitute an Axiomatic Model of Preferences (AMP), which can be considered as an "ideal" scenario of retrieval. Based on lattice theory, this work formally explores the numeric, metric and scale properties of effectiveness measures defined on these AMPs. In some of these scenarios, the retrieval measures are completely described from the values of a subset of document rankings: join-irreducible elements. The possible metrics and pseudometrics, defined from these structures are determined, with explicit expressions in terms of the join-irreducible elements. The deduced scale properties of the precision, recall, F-measure, RBP, DCG and AP confirm the state-of-the-art results. It is found that when the relevant documents are prioritised, the associated distance to some of these retrieval measures are metrics. However, this is not the case when the swapping of documents is considered.