The Epistemology of Machine Learning: Induction, Causality, Explanation
UPDATE! We have uploaded the tutorial slides and materials. You can find them at the following link! You can check the link to stay updated with references and suggested readings.
Summary
Scientists have typically divergent attitudes towards philosophy, ranging from those who think that science without it is “primitive and muddled” (Einstein, 1949) to those who claim we should not expect philosophy to provide researchers “with any useful guidance about how to go about their work” (Weinberg, 1993). Although both positions contain an element of truth, in this tutorial we shall side with the former, in the conviction that a greater awareness of the philosophical questions underpinning one’s research field can only be beneficial. In particular, we shall take the view that machine learning can arguably be thought of as “a continuation of epistemology by other means,” and will try to critically examine some of the most fundamental (and often tacit) assumptions of our field through a philosopher’s lens.
In fact, key questions pertaining to categorization, abstraction, generalization, induction, explanation etc. have been on the agenda of the epistemological inquiry, under different names and guises, since its inception. This could be an opportunity for reflection, reassessment, and possibly some synthesis, with a view to providing the field a self-portrait of where it currently stands and where it is going as a whole.
Outline
Introduction and motivations
Induction and its discontents
Causality: The strange life of a principle
Philosophical Aspects of Explanation
Summary and outlook
Prerequisites
We target expert and non-expert AI researchers. We shall assume no pre-existing knowledge of philosophy by the audience, thereby making the tutorial self-contained and understandable by a non-expert.
About the Speakers
Marcello Pelillo is a Full Professor of Computer Science at Ca’ Foscari University, Venice, where he leads the Computer Vision and Machine Learning Lab which he established in 1995. He has been the Director of the European Centre for Living Technology (ECLT) and has held visiting research/teaching positions at Yale University (USA), McGill University (Canada), University College London (UK), University of Vienna (Austria), York University (UK), NICTA (Australia), Wuhan University (China), Huazhong University of Science and Technology (Wuhan, China), South China University of Technology (Guangzhou, China). He is an external affiliate of the Computer Science Department at Drexel University (USA) and of the Italian Institute of Technology. His research interests are in the areas of computer vision, machine learning, and pattern recognition where he has published more than 300 technical papers in refereed journals, handbooks, and conference proceedings. He has been the General Chair for ICCV 2017 and Program Chair for ICPR 2020. He has also been Track Chair for ICPR 2018 and is regularly an Area Chair for the top-tier conferences in his field (ICCV, ECCV, CVPR, IJCAI, ICPR, etc.). He has served as Program Chair for several conferences, including workshops at NIPS (1999, 2011) and ICML (2010), many of which he initiated (e.g., EMMCVPR, SIMBAD, IWCV). He is the Chief Editor of Frontiers in Computer Science – Computer Vision, and serves or has served on the Editorial Boards of several journals, including IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Pattern Recognition, IET Computer Vision, Frontiers in Computer Image Analysis, Brain Informatics, IEEE Transactions on Neural Networks and Learning Systems (Guest Editor), Pattern Recognition Letters (Guest Editor), and serves on the Advisory Board of Springer’s International Journal of Machine Learning and Cybernetics. He is regularly invited as a Keynote Speaker in international conferences and summer schools and has been a Tutorial Lecturer at top-notch conferences in his area (e.g., CVPR, ECCV, ICPR, ACL, ECML). Prof. Pelillo is Fellow of the IEEE, the IAPR, and the AAIA, and is an IEEE SMC Distinguished Lecturer.
Martina Mattioli is a Ph.D. Student in the Department of Environmental Sciences, Informatics, and Statistics (DAIS) at the University of Ca' Foscari (Venice), under the supervision of Professor Marcello Pelillo. She is part of the Italian National Ph.D. Program in Artificial Intelligence for Industry 4.0., with Polytechnic of Turin as the leading university. She got a bachelor’s degree in Communication, Innovation, and Multimedia studies from the University of Pavia and a master's degree in Theory and Technology of Communication from the University of Milano-Bicocca. Her research interests lie at the intersection of epistemology and machine learning. Currently, she is dealing with the philosophical aspects of explanation. Her other research topics include the foundation of machine learning from the standpoint of the philosophy of science, and the philosophical aspects of similarity and categorization.
References
Balazka, D., & Rodighiero, D. (2020). Big data and the little big bang: An epistemological (r) evolution. Frontiers in Big Data, 3.
Bovens, L., & Hartmann, S. (2004). Bayesian Epistemology. Oxford, UK: Oxford University Press.
Corfield, D., Schölkopf, B., & Vapnik, V. (2009). Falsificationism and statistical learning theory: Comparing the Popper and Vapnik-Chervonenkis dimensions. Journal for General Philosophy of Science, 40, 51-58.
Dennett, D. C. (1988). When philosophers encounter Artificial Intelligence. Daedalus, 283-295.
Duda, R. O., Hart, P. E., & Stork, D. G. (2000). Pattern Classification. New York: John Wiley & Sons.
Duin, R. P. (2001). Four scientific approaches to pattern recognition. Fourth Quinquennial Review, 331–337.
Duin, R. P., & Pekalska, E. (2007). The science of pattern recognition. Achievements and perspectives. Challenges for Computational Intelligence, 221-259.
Eco, U. (2000). Kant and the Platypus: Essays on Language and Cognition. Harvest Books.
Glymour, C. N., & Cooper, G. F. (Eds.). (1999). Computation, Causation, and Discovery. Cambridge, MA: MIT Press.
Good, I. J. (1983). The philosophy of exploratory data analysis. Philosophy of Science, 50(2), 283-295.
Goodman, N. (1972). Seven strictures on similarity. In N. Goodman, Problems and Projects (pp. 437-448). Indianapolis: Bobbs-Merrill.
Goodman, N. (1983). Fact, Fiction, and Forecast. Harvard University Press.
Hahn, U., & Ramscar, M. E. (2001). Similarity and Categorization. Oxford, UK: Oxford University Press.
Halpern, J. Y., & Pearl, J. (2005). Causes and Explanations: A Structural-Model Approach. The British Journal for the Philosophy of Science, 56(4), 843-887.
Joyce, J. M. (1999). The Foundations of Causal Decision Theory. Cambridge, UK: Cambridge University Press.
Kanal, L. N. (1993). On pattern, categories, and alternate realities. Pattern Recognition Letters, 14(3), 241-255.
Khun, T. S. (1962). The Structure of Scientific Revolutions. University of Chicago Press.
Kitchin, R. (2014). Big Data, new epistemologies and paradigm shifts. Big Data & Society, 1(1), 2053951714528481.
Koller, D., & Friedman, N. (2009). Probabilistic Graphical Models: Principles and Techniques. Cambridge, MA: MIT Press.
Korb, K. B. (2004). Introduction: Machine Learning as philosophy of science. Minds and Machines, 14, 433-440.
Lakatos, I. (2014). Falsification and the Methodology of Scientific Research Programmes. In I. Lakatos, Philosophy, Science, and History (pp. 89-94). Routledge.
Lakoff, G. (2008). Women, Fire, and Dangerous Things: What Categories Reveal about the Mind. University of Chicago Press.
Lewis, D. (1981). Causal decision theory. Australasian Journal of Philosophy, 59(1), 5-30.
Mattioli, M., Cinà, A. E., & Pelillo, M. (2024). Understanding XAI Through the Philosopher's Lens: A Historical Perspective. ArXiv. https://arxiv.org/abs/2407.18782
Minsky, M. (1961). Steps toward Artificial Intelligence. Proceedings of the IRE, 49(1), 8-30.
Pearl, J. (2009). Causality. Cambridge, UK: Cambridge University Press.
Pelillo, M. (2013). Similarity-Based Pattern Analysis and Recognition. London: Springer.
Pelillo, M. (2015). Special issue on" Philosophical Aspects of Pattern Recognition". Pattern Recognition Letters, 64, 1-2.
Pelillo, M., & Teresa, S. (2013). How mature is the field of machine learning? AI* IA 2013: Advances in Artificial Intelligence: XIIIth International Conference of the Italian Association for Artificial Intelligence, Turin, Italy, December 4-6, 2013. Proceedings 13 (pp. 121-132). Springer.
Popper, K. R. (1959). The Logic of Scientific Discovery. Hutchinson & Co.
Quine, Willard V. (1969). Natural kinds. In W. V. Quine, Ontological relativity and other essays (pp. 114-138). Columbia University Press
Salmon, W. C. (1990). Four Decades of Scientific Explanation. University of Pittsburgh Press.
Serra, J. (2000). Is pattern recognition a physical science? Proceedings 15th International Conference on Pattern Recognition. ICPR-2000 (pp. 29-36). IEEE.
Spirtes, P., Glymour, C., & Scheines, R. (2001). Causation, Prediction and Search. Cambridge, MA: MIT Press.
Thagard, P. (1990). Philosophy and Machine Learning. Canadian Journal of Philosophy, 20(2), 261-276.
Tversky, A. (1977). Features of similarity. Psychological Review, 84(4), 327–352.
Von Luxburg, U., & Schölkopf, B. (2011). Statistical Learning Theory: Models, Concepts, and Results. In D. Gabbay, S. Hartmann, & J. Woods, Handbook of the History of Logic (pp. 651-706). Elsevier.
Watanabe, S. (1985). Pattern Recognition: Human and Mechanical. New York: John Wiley & Sons.