Empowering LLMs with Logical Reasoning: Challenges, Solutions, and Opportunities
The 34th International Joint Conference on Artificial Intelligence (IJCAI) Tutorial
August 29, 2025, Langham Place
Guangzhou, China
Embedded Files
Abstract
Large language models (LLMs) have achieved remarkable successes on various tasks. However, recent studies have found that there are still significant challenges to the logical reasoning abilities of LLMs, which can be categorized into the following two aspects: (1) Logical question answering: LLMs often fail to generate the correct answer within a complex logical problem which requires sophisticated deductive, inductive or abductive reasoning given a collection of premises and constrains. (2) Logical consistency: LLMs are prone to producing responses contradicting themselves across different questions. For example, a state-of-the-art question-answering LLM Macaw, answers “Yes” to both questions “Is a magpie a bird?” and “Does a bird have wings?” but answers “No” to “Does a magpie have wings?”.
In this tutorial, we comprehensively introduce the most cutting-edge methods with a proposed new taxonomy. Specifically, to accurately answer complex logic questions, previous methods can be categorized based on reliance on external solvers, prompts, and fine-tuning. To avoid logical contradictions, we discuss concepts and solutions of various logical consistencies, including implication, negation, transitivity, factuality consistencies, and their composites. In addition, we review commonly used benchmark datasets and evaluation metrics, and discuss promising research directions, such as extending to modal logic to account for uncertainty and developing efficient algorithms that simultaneously satisfy multiple logical consistencies.
Materials
Tutorial Slides:
TBA
Survey Paper:
Fengxiang Cheng, Haoxuan Li, Fenrong Liu, Robert van Rooij, Kun Zhang, Zhouchen Lin. Empowering LLMs with Logical Reasoning: A Comprehensive Survey. In IJCAI, 2025.
Paper Link: https://arxiv.org/pdf/2502.15652
Citation:
@inproceedings{cheng2025empowering,
author={Cheng, Fengxiang and Li, Haoxuan and Liu, Fenrong and Van Rooij, Robert and Zhang, Kun and Lin, Zhouchen},
title={Empowering LLMs with Logical Reasoning: A Comprehensive Survey},
booktitle={Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence},
year={2025}
}
Schedule
Time: August 29, 2025
Location: Langham Place, Guangzhou
Outline
- Part I: Introduction to Logical Reasoning in LLMs
- Part II: Logical Question Answering Tasks
- Part III: Logical Question Answering Methods
- Part IV: Logical Consistency Types and Modeling Techniques
- Part V: Benchmarking and Evaluation of Logical Consistency
- Part VI: Future Directions in Symbolic and Modal Reasoning
- Part VII: Q&A and Interactive Discussion
Tutors' Bios
Haoxuan Li is an assistant researcher at Center for Data Science, Peking University. His research interests include causal machine learning theory, information retrieval, and large language models. He has published more than 40 papers in top conferences including ICML, NeurIPS, ICLR, AAAI, SIGIR, ICDM, SIGKDD, WWW, SIGIR, ICDE, etc., and received the NSFC Young Scientists Fund (2024) and Young Elite Scientists Sponsorship Program by CAST - Doctoral Student Special Plan (via CCF). He have been selected as the 2024 Peking University Person of the Year. He have served as the area chair (AC) or program committee (PC) member for top-tier conferences including ICML, NeurIPS, ICLR, SIGKDD, WWW, AAAI, IJCAI, CVPR, ICCV, and the invited reviewer for prestigious journals such as TOIS, TPAMI, TKDE, TKDD, TNNLS, JASA, SCIENCE CHINA Information Sciences, and The Innovation.
Fengxiang Cheng is currently a Ph.D. candidate in Logic at the University of Amsterdam’s Institute for Logic, Language and Computation (ILLC). She holds a Master of Logic from Tsinghua University and dual Bachelor’s degrees in Philosophy and Economics from Sichuan University. Her research lies at the intersection of logic, artificial intelligence, and causal reasoning, with a focus on enhancing the logical and causal capabilities of large language models (LLMs). She has co-authored many related works, including a comprehensive survey on empowering LLMs with logical reasoning (under review at IJCAI 2025) and a AAAI 2025 workshop paper analyzing causal methods to resolve spurious correlations in language models. She served as a program committee member for ICLR and AAAI workshops. She has received the National Scholarship (China’s top honor), Tsinghua University Scholarship for Excellence in Social Work, and the China Scholarship Council (CSC) Scholarship.
Fenrong Liu is currently a Professor in the Department of Philosophy, Tsinghua University. She is now the Director of the THU-UvA Joint Research Centre for Logic and Amsterdam-China Logic Chair in ILLC, University of Amsterdam. She is a Fellow of the Institut International de Philosophie (IIP), ASL Code of Ethics committee, International Academy of the Philosophy of Science (AIPS). She is the president of the Beijing Logic Association and Vice-President of the Chinese Society of Logic. She has hosted tutorials at the Annual Conference of the Australasian Association of Logic. Her research interests are Mathematical Logic, Philosophical Logic, Modal Logic, Logics in Artificial Intelligence, and large models.
Robert van Rooij is currently a professor of Logic and Cognition (with special attention to the analysis of language) at ILLC (Faculty of Science, the University of Amsterdam), and serves as the scientific director at ILLC. He works on the formal semantics (e.g., generics, comparatives, questions, dynamic semantics) and pragmatics (e.g. conversational implicatures, presuppositions) of natural language, philosophy of language (e.g. propositional attitudes, reference), philosophical logic (e.g. conditionals, vagueness, truth) and cognition (causality, decision making, learning, bounded rationality). He was involved in the AI-oriented Marie Curie ESSENCE project, the philosophy-oriented `Communication and Context' project, the linguistic-oriented VAAG project, and a project with mostly psycholinguists. Moreover, he has published a paper entitled ``ChatGPT: Five priorities for research" in Nature in 2023 (Citation: 1769) for advancing LLMs research.
Kun Zhang is currently an associate professor at Carnegie Mellon University and a Professor (also the department chair) of Machine Learning at Mohamed bin Zayed University of Artificial Intelligence. His research on causal discovery and inference spans theory and applications, especially causal representation learning and causal principles for intelligent systems. He serves as an Associate Editor of ACM Computing Surveys and Pattern Recognition, guest editor of special issues for ACM TIST (2018), and guest co-editor for JSAIT (2022) and TNNLS (2021). He also co-organized multiple workshops at UAI, NeurIPS, ICCV, ACM SIGKDD and SDM. He was a co-founder and general & program co-chair of the Conference on Causal Learning and Reasoning (CLeaR 2022), a program co-chair of the 38th Conference on Uncertainty in Artificial Intelligence (UAI 2022), and a general co-chair of UAI 2023, and is a program co-chair of ICDM 2024.
Zhouchen Lin is currently a Distinguished professor in the School of Intelligence Science and Technology at Peking University. He is a Fellow of the Asia-Pacific Artificial Intelligence Association (AAIA), Fellow of the China Society of Image and Graphics (CSIG), Fellow of the Institute of Electrical and Electronics Engineers, Fellow of the International Association of Pattern Recognition (IAPR), and ACM Distinguished Member. His main research interests are about machine learning, large models, and causal inference. He earns the Okawa Research Grant and Microsoft SPOT Award. He is a senior area chair (or senior program committee member) of ICML, NeurIPS, ICLR, AAAI and IJCAI. He is an associate editor of the IEEE Transactions on Pattern Analysis and Machine Intelligence and the International Journal of Computer Vision.
BibTeX
@article{ijcai-logicllm-tutorial,
author={Li, Haoxuan and Cheng, Fengxiang and Liu, Fenrong and Van Rooij, Robert and Zhang, Kun and Lin, Zhouchen},
title={IJCAI 2025 Tutorial: Empowering LLMs with Logical Reasoning: Challenges, Solutions, and Opportunities},
journal={IJCAI 2025},
year={2025}
}
Page updated
Google Sites
Report abuse