Winter Semester 2026/27 - Seminar

Solving Complex Tasks Using Large Language Models (LLMs)

The seminar explores how Large Language Models (LLMs) address complex tasks across a range of research areas, including reasoning, evaluation, scientific discovery, and multimodal AI. The seminar is organized around 27 presentation topics, each accompanied by a set of assigned research papers. Students are expected to study, critically analyze, and present the research covered in these papers. The assigned papers form the basis of the seminar assessment; additional literature may be consulted but is not required.

Organization

• This seminar is organized by Dr. Jennifer D'Souza.

• The seminar is available for master students of the "Electrical Engineering and Information Technology" or "Computer Science" programs at the Leibniz University of Hannover and at present is offered in Winter Semester 2026/2027 (WiSE 26/27).

• Seminar Grading: Talk 50% · Written report 40% · Participation 10%. All three aspects need to be passed individually. 

• Seminar Registration (19 October – 1 November 2026): A maximum of 27 seminar places are available, corresponding to the 27 presentation topics. Students must register for the VbP examination in QIS and select an available topic in the Stud.IP Wiki. QIS registration alone does not secure a seminar place. If no suitable topic remains available, students should not register for the examination or, if already registered, should deregister within the official deregistration period. Detailed instructions are available in Stud.IP.

• Student talks will begin on 17.11.2026. This is also the official examination start date for this seminar.

  • Each presentation should be approximately 30–35 minutes, followed by 10–15 minutes of discussion. 

• Seminar appointments (day/time):

  • From October 13th till October 20th: Tuesday, 09:00 - 10:30, weekly

  • October 27th, November 3rd, and November 10th: No class, time for seminar talk preparation

  • From November 17th till January 26th: Tuesday, 09:00 - 11:15, weekly (Please note: The finalized seminar schedule, including whether individual sessions will need to run until 11:15, will be communicated after the examination registration period closes on 1 November 2026.)

  • 29 December 2026 and 5 January 2027: No class, Winter break days

Goals

In this seminar, you will

• read and develop a thorough understanding of the scientific papers assigned to your presentation topic.

• critically analyze, synthesize, and explain the key ideas, methodologies, and findings presented in these papers.

• deliver a presentation (talk) based on the assigned papers, followed by discussion, and subsequently submit a written report.

Please note: Your presentation and report will be assessed based on your understanding, critical analysis, and coverage of the assigned papers. You are welcome to consult additional literature, but this is not required. 

Topics

Evaluating LLM Capabilities

1. Beyond Traditional Benchmarks: Evaluating LLM Intelligence - I

   • Center for AI Safety., Scale AI. & HLE Contributors Consortium. A benchmark of expert-level academic questions to assess AI capabilities. Nature 649, 1139–1146 (2026). https://doi.org/10.1038/s41586-025-09962-4   

2. Beyond Traditional Benchmarks: Evaluating LLM Intelligence - II

• Foundation, A. R. C. "ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence." arXiv preprint arXiv:2603.24621 (2026). https://arxiv.org/abs/2603.24621 

3. Beyond Traditional Benchmarks: Evaluating LLM Intelligence - III

   • Xu, Wanghan, Yuhao Zhou, Yifan Zhou, Qinglong Cao, Shuo Li, Jia Bu, Bo Liu et al. "Probing scientific general intelligence of llms with scientist-aligned workflows." arXiv preprint arXiv:2512.16969 (2025). https://arxiv.org/abs/2512.16969 

4. Can LLMs Solve Advanced Mathematics? 

• Dekoninck, Jasper, et al. Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs 

• Pu, Lushi, et al. MA-ProofBench: A Two-Tiered Evaluation of LLMs for Theorem Proving in Mathematical Analysis 

5. When LLMs Make Things Up: Hallucinations and Factuality

• Ravichander, A., Ghela, S., Wadden, D. & Choi, Y. "HALoGEN: Fantastic LLM Hallucinations and Where to Find Them." Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, 1402–1425 (2025). https://doi.org/10.18653/v1/2025.acl-long.71

• Fatahi Bayat, F., Zhang, L., Munir, S. & Wang, L. "FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation." Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, 33090–33110 (2025). https://doi.org/10.18653/v1/2025.acl-long.1587

• Bang, Y., Ji, Z., Schelten, A., Hartshorn, A., Fowler, T., Zhang, C., Cancedda, N. & Fung, P. "HalluLens: LLM Hallucination Benchmark." Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, 24128–24156 (2025). https://doi.org/10.18653/v1/2025.acl-long.1176 

6. Are LLMs Fair, Safe, and Resistant to Jailbreaks?

• Cantini, R., Orsino, A., Ruggiero, M. & Talia, D. Benchmarking adversarial robustness to bias elicitation in large language models: scalable automated assessment with LLM-as-a-judge. Machine Learning 114, 249 (2025). https://doi.org/10.1007/s10994-025-06862-6 

• Jiang, F. et al. SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities. Findings of ACL 2025, 23303–23320 (2025). https://doi.org/10.18653/v1/2025.findings-acl.1197 

• Hagendorff, T., Derner, E. & Oliver, N. Large reasoning models are autonomous jailbreak agents. Nature Communications 17, 1435 (2026). https://doi.org/10.1038/s41467-026-69010-1 

Reasoning and Knowledge Augmentation 

7. Eliciting Reasoning in LLMs 

   • Liao, H., Hu, S., Zhu, Z., He, H. & Jin, Y. Forest for the Trees: Overarching Prompting Evokes High-Level Reasoning in Large Language Models. Proceedings of NAACL 2025, 1433–1453 (2025). https://doi.org/10.18653/v1/2025.naacl-long.66 

   • Liu, Y., Li, Z., Fang, Z., Xu, N., He, R. & Tan, T. Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory. Proceedings of ACL 2025, 27962–27994 (2025). https://doi.org/10.18653/v1/2025.acl-long.1356 

   • Zhang, Y. et al. Logit Arithmetic Elicits Long Reasoning Capabilities Without Training. Findings of ACL 2026, 24943–24959 (2026). https://doi.org/10.18653/v1/2026.findings-acl.1249 

8. Can LLMs Correct Their Own Mistakes? Self-Reflection and Refinement 

   • Srivastava, G., Bi, Z., Lu, M. & Wang, X. DEBATE, TRAIN, EVOLVE: Self-Evolution of Language Model Reasoning. Proceedings of EMNLP 2025, 32764–32810 (2025). https://doi.org/10.18653/v1/2025.emnlp-main.1666 

   • Choi, H. K., Zhu, X. & Li, S. Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models? Advances in Neural Information Processing Systems 38 (2025). https://doi.org/10.52202/085713-3405 

   • Zhu, X., Zhang, C., Chi, Y., Stafford, T., Collier, N. & Vlachos, A. Demystifying Multi-Agent Debate: The Role of Confidence and Diversity. Findings of ACL 2026, 33909–33930 (2026). https://doi.org/10.18653/v1/2026.findings-acl.1694 

9. From RAG to GraphRAG: Reasoning over Connected Knowledge 

   • Gutiérrez, B. J., Shu, Y., Qi, W., Zhou, S. & Su, Y. From RAG to Memory: Non-Parametric Continual Learning for Large Language Models. Proceedings of ICML 2025, 267, 21497–21515 (2025). https://proceedings.mlr.press/v267/gutierrez25a.html 

   • Luo, H. et al. Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learning. Proceedings of ICML 2026, 306, 82220–82240 (2026). https://proceedings.mlr.press/v306/luo26c.html 

   • Xiang, Z., Wu, C., Zhang, Q., Chen, S., Hong, Z., Huang, X. & Su, J. When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation. International Conference on Learning Representations (ICLR) (2026). Official paper 

LLMs in Real-World Applications 

10. Talking to Databases: From Natural Language to SQL and Knowledge Graph Queries 

    • Wang, Z. et al. AutoLink: Autonomous Schema Exploration and Expansion for Scalable Schema Linking in Text-to-SQL at Scale. Proceedings of AAAI 2026, 40(40), 33809–33817 (2026). https://doi.org/10.1609/aaai.v40i40.40672 

    • Nascimento, E. R. et al. A Text-to-SQL strategy based on large language models and knowledge graphs for real-world databases. Data & Knowledge Engineering 164, 102580 (2026). https://doi.org/10.1016/j.datak.2026.102580 

    • Zhang, Z. et al. CypherSmith: Transforming Text-to-Cypher Generation for LLMs with Synthetic Data. Proceedings of ACL 2026, 34665–34682 (2026). https://doi.org/10.18653/v1/2026.acl-long.1601 

11. Can LLMs Become Software Engineers? From Code Generation to Bug Fixing 

    • Lu, Z., Ren, H., Yang, Y., Wang, K., Zong, Z., Zhan, M. & Li, H. FullStack-Agent: Enhancing Agentic Full-Stack Web Coding via Development-Oriented Testing and Repository Back-Translation. Proceedings of ICML 2026, 306, 81478–81512 (2026). https://proceedings.mlr.press/v306/lu26ab.html 

    • Choi, C., Kaya, Z. N., Wu, S., Ma, T., Hashimoto, T. & Schmidt, L. Anchored Self-Play for Code Repair. Proceedings of ICML 2026, 306, 20045–20067 (2026). https://proceedings.mlr.press/v306/choi26f.html 

    • Deng, X. et al. SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? Proceedings of ICML 2026, 306, 23917–23934 (2026). https://proceedings.mlr.press/v306/deng26f.html 

12. LLMs in Educational Applications 

    • Li, Z., Zhu, Q., Wang, M., Li, J. & Huang, H. Planning-Guided Tutoring with Assessment-Driven Memory for Pedagogical LLM Tutors. Proceedings of ACL 2026, 7165–7188 (2026). https://doi.org/10.18653/v1/2026.acl-long.325 

    • Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö. & Mariman, R. Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences 122(26), e2422633122 (2025). https://doi.org/10.1073/pnas.2422633122 

    • Xu, B. et al. EduBench: A Comprehensive Benchmarking Dataset for Evaluating Large Language Models in Diverse Educational Scenarios. Proceedings of ACL 2026, 21615–21645 (2026). https://doi.org/10.18653/v1/2026.acl-long.987 

13. Can LLMs Be Lawyers? Legal Reasoning, Case Retrieval, and Evidence 

    • Lee, J., On, K.-W., Han, S. S., Cohan, A. & Hockenmaier, J. Evaluating Legal Reasoning Traces with Legal Issue Tree Rubrics. Proceedings of ACL 2026, 3299–3322 (2026). https://doi.org/10.18653/v1/2026.acl-long.150 

    • Tao, M., Hu, J., Zhou, X., Hu, W., Cheng, J., Cao, Y., Luo, Z. & Geng, G. When Rules Learn: A Self-Evolving Agent for Legal Case Retrieval. Findings of ACL 2026, 43348–43365 (2026). https://doi.org/10.18653/v1/2026.findings-acl.2152 

    • Huang, Z., Ma, Y., Wang, L., Wang, Y., Shi, D. & Xu, C. Mitigating Legal Hallucinations via Symbolic Constraints and Analogical Precedents. Proceedings of ACL 2026, 13894–13920 (2026). https://doi.org/10.18653/v1/2026.acl-long.633 

Video, Music, and Creative AI  

14. Can LLMs Really See? The Frontiers of Visual Intelligence 

• Törtei, B. M. et al. VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs. Proceedings of CVPR 2026, 33185–33195 (2026). Official paper 

• Zhang, J. et al. Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models. Proceedings of ACL 2026, 24381–24404 (2026). https://doi.org/10.18653/v1/2026.acl-long.1119 

• Janjua, M. K., Silva, H., Niu, D. & Rashidi, B. Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs. Proceedings of CVPR 2026, 5165–5174 (2026). Official paper 

15. From Words to Images: AI Image Generation and Editing

• Tian, R., Gao, M., Gang, H., Lu, J., Gan, Z., Yang, Y., Wu, Z. & Dehghan, A. UniGen-1.5: Enhancing Image Generation and Editing through Reward Unification in RL. Proceedings of CVPR 2026, 29367–29378 (2026). Official paper 

• Liu, Z., Peng, W., Chen, Y. & Wu, Z. Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization. Proceedings of CVPR 2026, 36604–36614 (2026). Official paper  

• Yin, F. et al. ReasonEdit: Towards Reasoning-Enhanced Image Editing Models. Proceedings of CVPR 2026, 23283–23293 (2026). Official paper  

16. Lights, Camera, AI! Generative Models for Video Creation 

    • Gu, J. et al. STARFlow-V: End-to-End Video Generative Modeling with Autoregressive Normalizing Flows. Proceedings of CVPR 2026, 9084–9094 (2026). Official paper 

    • Yuan, Y., Wang, X., Wickremasinghe, T., Nadir, Z., Ma, B. & Chan, S. NewtonGen: Physics-consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamics. International Conference on Learning Representations (ICLR) (2026). Official paper  

    • Gu, J. et al. PhyWorldBench: A Comprehensive Evaluation of Physical Realism in Text-to-Video Models. International Conference on Learning Representations (ICLR) (2026). Official paper  

17. Can AI Compose the Next Hit Song? Music and Song Generation 

    • Li, S., Choi, D. & Sung, Y. MIDILM: A Dual-Path Model for Controllable Text-to-MIDI Generation. Proceedings of AAAI 2026, 40(28), 23160–23168 (2026). https://doi.org/10.1609/aaai.v40i28.39483 

    • Wang, Y., Ji, Z., Cai, P., Li, X., Zheng, H., Song, Z., Liu, Z., Zhang, C. & Wan, P. SegTune: Structured and Fine-Grained Control for Song Generation. Proceedings of ACL 2026, 12883–12897 (2026). https://doi.org/10.18653/v1/2026.acl-long.586 

    • Ma, Y. et al. CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction. Proceedings of ICML 2026, 306, 85099–85130 (2026). https://proceedings.mlr.press/v306/ma26am.html 

18. Can AI Direct a Movie? Long-Form Video and Visual Storytelling 

• Zhang, P., Jia, Z., Liu, K., Weng, S., Li, S. & Shi, B. STAGE: Storyboard-Anchored Generation for Cinematic Multi-shot Narrative. Proceedings of CVPR 2026, 659–669 (2026). Official paper 

• Meng, Y. et al. HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narratives. Proceedings of CVPR 2026, 461–471 (2026). Official paper 

• Wu, W., Liu, M., Zhu, Z., Xia, X., Feng, H., Wang, W., Lin, K. Q., Shen, C. & Shou, M. Z. MovieBench: A Hierarchical Movie Level Dataset for Long Video Generation. Proceedings of CVPR 2025, 28984–28994 (2025). Official paper 

Scientific Discovery and Future Architectures   

19. Can LLMs Discover Scientific Laws? From Hypotheses to Equations 

    • Liu, H., Huang, S., Hu, J., Zhou, Y. & Tan, C. HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generation. arXiv preprint arXiv:2504.11524 (2025). https://arxiv.org/abs/2504.11524 

    • Shojaee, P., Nguyen, N.-H., Meidani, K., Barati Farimani, A., Doan, K. D. & Reddy, C. K. LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models. Proceedings of ICML 2025, 267, 55325–55359 (2025). https://proceedings.mlr.press/v267/shojaee25a.html 

    • Li, Y. & Pan, S. J. Robust Equation Structure Learning with Adaptive Refinement. International Conference on Learning Representations (ICLR) (2026). Official paper 

20. From Scientific Papers to Knowledge: Multimodal Scientific Information Extraction  

• Khalighinejad, G., Scott, S., Liu, O., Anderson, K. L., Stureborg, R., Tyagi, A. & Dhingra, B. MatViX: Multimodal Information Extraction from Visually Rich Articles. Proceedings of NAACL 2025, 3636–3655 (2025). https://doi.org/10.18653/v1/2025.naacl-long.185 

• Li, Y. et al. SudokuFill: A Multi-Agent Progressive Filling Framework for Document-Level Scientific Information Extraction. Findings of ACL 2026, 33112–33138 (2026). https://doi.org/10.18653/v1/2026.findings-acl.1657 

• Cao, Z. et al. Agents-K1: Towards Agent-native Knowledge Orchestration. arXiv preprint arXiv:2606.13669 (2026). https://arxiv.org/abs/2606.13669 

21. Can LLMs Become Chemists? AI for Chemistry and Materials Discovery   

• Liu, S., Xu, J., Ye, B., Hu, B., Srolovitz, D. J. & Wen, T. MatTools: Benchmarking Large Language Models for Materials Science Tools. arXiv preprint arXiv:2505.10852 (2025). https://arxiv.org/abs/2505.10852 

• Chaudhari, A., Ock, J. & Barati Farimani, A. Modular large language model agents for multi-task computational materials science. Communications Materials 7, 131 (2026). https://doi.org/10.1038/s43246-025-00994-x 

• Pagel, S., Jirasek, M. & Cronin, L. Verification and execution of the scientific literature via chemputation augmented by large language models. Communications Chemistry 9, 191 (2026). https://doi.org/10.1038/s42004-026-01993-w  

• Yu, B. et al. Tooling or Not Tooling? The Impact of Tools on Language Agents for Chemistry Problem Solving. Findings of NAACL 2025, 7635–7655 (2025). https://doi.org/10.18653/v1/2025.findings-naacl.424 

22. Do LLMs Understand Cause and Effect? Causal Reasoning in Scientific Research  

• Acharya, S. et al. CauSciBench: Can LLMs Automate Causal Inference in Real-World Scientific Research? Proceedings of ICML 2026, 306, 332–384 (2026). https://proceedings.mlr.press/v306/acharya26a.html 

• Saklad, R., Chadha, A., Pavlov, O. V. & Moraffah, R. Can Large Language Models Infer Causal Relationships from Real-World Text? Proceedings of ACL 2026, 21957–21989 (2026). https://doi.org/10.18653/v1/2026.acl-long.1003 

• Spillias, S., Rogers, J. G. D., Boschetti, F., Fulton, E. A., Guglielmo, M., Yong, S. Y. & Trebilco, R. Data-driven discovery of mechanistic ecosystem models with LLMs. Methods in Ecology and Evolution 17(4), 1303–1321 (2026). https://doi.org/10.1111/2041-210x.70244  

23. From AI to the Laboratory: Experimental Design and Self-Driving Labs  

• Mandal, I., Soni, J., Zaki, M. et al. Evaluating large language model agents for automation of atomic force microscopy. Nature Communications 16, 9104 (2025). https://doi.org/10.1038/s41467-025-64105-7 

• Brunnsåker, D., Gower, A. H., Naval, P. et al. Agentic AI integrated with scientific knowledge: laboratory validation in systems biology. Journal of the Royal Society Interface 23(240), 20260043 (2026). https://doi.org/10.1098/rsif.2026.0043 

• Iwasaki, Y., Toyama, R., Tamura, R. et al. Large-language-model-driven adaptive search space definition for autonomous closed-loop materials exploration. Communications Materials 7, 202 (2026). https://doi.org/10.1038/s43246-026-01304-9  

24. Can AI Become a Scientist? Autonomous Scientific Discovery 

• Yamada, Y., Lange, R. T., Lu, C., Hu, S., Lu, C., Foerster, J., Clune, J. & Ha, D. The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search. arXiv preprint arXiv:2504.08066 (2025). https://arxiv.org/abs/2504.08066 

• Wehr, G., Rideaux, R., Fox, A. J., Lightfoot, D. R., Tangen, J., Mattingley, J. B. & Ehrhardt, S. E. Closing the Empirical Loop: Autonomous AI Agents Conduct End-to-end Research With Human Participants. Advanced Science 13(56), e76675 (2026). https://doi.org/10.1002/advs.76675 

• Liu, T. et al. Benchmarking AI Agents for Addressing Scientific Challenges Across Scales. arXiv preprint arXiv:2606.12736 (2026). https://arxiv.org/abs/2606.12736  

25. The Illusion of Thinking? Understanding the Limits of LLM Reasoning 

• Shojaee, P., Mirzadeh, I., Alizadeh, K., Horton, M., Bengio, S. & Farajtabar, M. The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity. Advances in Neural Information Processing Systems 38 (2025). https://arxiv.org/abs/2506.06941 

• Chen, Y. et al. Reasoning Models Don't Always Say What They Think. arXiv preprint arXiv:2505.05410 (2025). https://arxiv.org/abs/2505.05410 

• Zhou, S., Ling, R., Chen, J., Wang, X., Fan, T. & Wang, H. When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling. Findings of ACL 2026, 23967–23977 (2026). https://doi.org/10.18653/v1/2026.findings-acl.1199  

26. Hugging Science: Democratizing AI for Scientific Discovery 

• Channing, G. & Ghosh, A. AI for scientific discovery is a social problem. Patterns 7(3), 101497 (2026). https://doi.org/10.1016/j.patter.2026.101497 

• Horton, M. K., Huck, P., Yang, R. X. et al. Accelerated data-driven materials science with the Materials Project. Nature Materials 24, 1522–1532 (2025). https://doi.org/10.1038/s41563-025-02272-0 

• Trišović, A. & Sivaloganathan, J. Open Science, Closed Models: How Funding Shapes AI in Science. arXiv preprint arXiv:2609.25347 (2026). https://arxiv.org/abs/2609.25347 

27. Beyond Next-Token Prediction: JEPA and the Future of World Models 

• LeCun, Y. A Path Towards Autonomous Machine Intelligence. OpenReview position paper (2022). https://openreview.net/forum?id=BZ5a1r-kVsf 

• Assran, M. et al. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. arXiv preprint arXiv:2506.09985 (2025). https://arxiv.org/abs/2506.09985