第16回最先端NLP勉強会

8月25日(日) プログラム

[09:30-10:00]  受付
[10:00-10:10]  オープニング [slide]
[10:10-11:10]  セッション1: Large Language Models

• 西田京介: The Llama 3 Herd of Models (arxiv; Llama Team, AI @ Meta) [paper][slide]

• 高瀬翔: Multi-Head Mixture-of-Experts (arxiv; Xun Wu, Shaohan Huang, Wenhui Wang, Furu Wei) [paper][slide]

• 小林颯介: Physics of Language Models: Part 3.1, Knowledge Storage and Extraction (ICML2024; Zeyuan Allen-Zhu, Yuanzhi Li) [paper][slide]

[11:20-12:20]  セッション2: Finetuning

• 篠田一聡: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (NeurIPS2023; Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, Chelsea Finn) [paper][slide]

• 水木栄: Instruction-tuned Language Models are Better Knowledge Learners (ACL2024; Zhengbao Jiang, Zhiqing Sun, Weijia Shi, Pedro Rodriguez, Chunting Zhou, Graham Neubig, Xi Victoria Lin, Wen-tau Yih, Srini Iyer) [paper][slide]

• 清野舜: Tuning Language Models by Proxy (COLM2024; Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, Noah A. Smith) [paper][slide]

[14:00-15:20]  セッション3: Multilingual

• 笹野遼平: Unveiling Linguistic Regions in Large Language Models (ACL2024; Zhihao Zhang, Jun Zhao, Qi Zhang, Tao Gui, Xuanjing Huang) [paper][slide]

• 臼井久生: MELA: Multilingual Evaluation of Linguistic Acceptability (ACL2024; Ziyin Zhang, Yikang Liu, Weifang Huang, Junyu Mao, Rui Wang, Hai Hu) [paper][slide]

• 稲葉達郎: Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models (ACL2024; Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, Ji-Rong Wen) [paper][slide]

• スポンサー（ELYZA様）

• スポンサー（オムロン サイニックエックス様）

[15:30-16:30]  セッション4: Embedding/Representation

• 山田康輔: Improving Text Embeddings with Large Language Models (ACL2024; Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei) [paper][slide]

• 上田亮: Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models (NeurIPS2023; Guillermo Ortiz-Jimenez, Alessandro Favero, Pascal Frossard) [paper][slide]

• 塚越駿: Isotropy, Clusters, and Classifiers (ACL2024; Timothee Mickus, Stig-Arne Grönroos, Joseph Attieh) [paper][slide]

[16:40-18:00]  セッション5: Linguistic Generalization

• 大羽未悠: Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs (ICLR2024; Angelica Chen, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew L Leavitt, Naomi Saphra) [paper][slide]

• 横井祥: Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve (arxiv; R. Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, Thomas L. Griffiths) [paper][slide]

• 石井太河: What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages (ACL2024; Nadav Borenstein, Anej Svete, Robin Chan, Josef Valvoda, Franz Nowak, Isabelle Augenstein, Eleanor Chodroff, Ryan Cotterell) [paper][slide]

• 賀沢秀人: Mission: Impossible Language Models (ACL2024; Julie Kallini, Isabel Papadimitriou, Richard Futrell, Kyle Mahowald, Christopher Potts) [paper][slide]

[18:00-18:10]  クロージング [slide]

8月26日(月) プログラム

[09:30-10:00]  受付
[10:00-10:10]  オープニング [slide]
[10:10-11:10]  セッション1: Evaluation

• 西田悠人: Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References (NAACL2024; Tianyi Tang, Hongyuan Lu, Yuchen Jiang, Haoyang Huang, Dongdong Zhang, Xin Zhao, Tom Kocmi, Furu Wei) [paper][slide]

• 神藤駿介: Quantifying the redundancy between prosody and text (EMNLP2023; Lukas Wolf, Tiago Pimentel, Evelina Fedorenko, Ryan Cotterell, Alex Warstadt, Ethan Wilcox, Tamar Regev) [paper][slide]

• 平澤寅庄: ConSiDERS-The-Human Evaluation Framework: Rethinking Human Evaluation for Generative Large Language Models (ACL2024; Aparna Elangovan, Ling Liu, Lei Xu, Sravan Babu Bodapati, Dan Roth) [paper][slide]

[11:20-12:20]  セッション2: Multimodal

• 大嶽匡俊: Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbing (TACL2023; William Brannon, Yogesh Virkar, Brian Thompson) [paper][slide]

• 前田航希: Vision Transformers Need Registers (ICLR2024; Timothée Darcet, Maxime Oquab, Julien Mairal, Piotr Bojanowski) [paper][slide]

• 渡邉研斗: MotionGPT: Human Motion as a Foreign Language (NeurIPS2023; Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, Tao Chen) [paper][slide]

[13:30-14:30]  セッション3: Knowledge

• 木山朔: Analyzing Semantic Change through Lexical Replacements (ACL2024; Francesco Periti, Pierluigi Cassotti, Haim Dubossarsky, Nina Tahmasebi) [paper][slide]

• 井田龍希: TTM-RE: Memory-Augmented Document-Level Relation Extraction (ACL2024; Chufan Gao, Xuan Wang, Jimeng Sun) [paper][slide]

• 辻村有輝: Context versus Prior Knowledge in Language Models (ACL2024; Kevin Du, Vésteinn Snæbjarnarson, Niklas Stoehr, Jennifer C. White, Aaron Schein, Ryan Cotterell) [paper][slide]

[15:00-16:00]  セッション4: Cognition/Dialogue

• 李凌寒: Language is primarily a tool for communication rather than thought (Nature; Evelina Fedorenko, Steven T. Piantadosi & Edward A. F. Gibson) [paper][slide]

• 三田雅人: Expectations over Unspoken Alternatives Predict Pragmatic Inferences (TACL2023; Jennifer Hu, Roger Levy, Judith Degen, Sebastian Schuster) [paper][slide]

• 銭本友樹: Planning Like Human: A Dual-process Framework for Dialogue Planning (ACL2024; Tao He, Lizi Liao, Yixin Cao, Yuanxing Liu, Ming Liu, Zerui Chen, Bing Qin) [paper][slide]

[16:10-17:30]  セッション5: Context/Interpretability

• Youmi Ma: Do Large Language Models Latently Perform Multi-Hop Reasoning? (ACL2024; Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, Sebastian Riedel) [paper][slide]

• 持橋大地: [slide]

  • What Do Language Models Learn in Context? The Structured Task Hypothesis (ACL2024; Jiaoda Li, Yifan Hou, Mrinmaya Sachan, Ryan Cotterell) [paper]

  • A Theory for Emergence of Complex Skills in Language Models (arxiv; Sanjeev Arora, Anirudh Goyal) [paper]

• 森下睦: Interpretability of Language Models via Task Spaces (ACL2024; Lucas Weber, Jaap Jumelet, Elia Bruni, Dieuwke Hupkes) [paper][slide]

• 斉藤いつみ: Lost in the Middle: How Language Models Use Long Contexts (TACL2024; Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang) [paper][slide]

[17:35-17:45]  クロージング [slide]