Formal Tasks and Metrics in Data Mining on the Basis of Measurement Theory

This is the companion web page for the paper:

Enrique Amigó, Julio Gonzalo, Stefano Mizzaro: What is my problem? Identifying Formal Tasks and Metrics in Data Mining on the Basis of Measurement Theory. In IEEE Transactions of Knowledge and Data Engineering, 2021 (in press).

Abstract

The design and analysis of experimental research in Data Mining (DM) is anchored in a correct choice of the type of task addressed (clustering, classification, regression, etc.). However, although DM is a relatively mature discipline, there is no consensus yet about what is the taxonomy of DM tasks, which are their formal characteristics, and their corresponding metrics. In this paper, we formalize DM tasks in terms of Measurement Theory, which is a cornerstone of quantitative research in many disciplines, but has not yet been incorporated (in a consensual way) into some areas of Computer Science, including DM. The proposed formal framework provides a methodology to precisely define DM tasks for any given scenario and identify appropriate metrics. We validate this framework via (i) its coverage of existing DM tasks, (ii) its capability to group existing metrics into families, and (iii) its coverage of actual DM research problems, using about 250 papers from ACM KDD 2019 and IEEE ICDM 2019 conferences as reference sample.

Dataset

The classification of the almost 250 papers from ACM KDD 2019 and IEEE ICDM 2019 conferences can be downloaded here:

readme.txt

Analysis of ICDM 2019 papers

Analysis of KDD 2019 papers