第15回最先端NLP勉強会

8月27日(日) プログラム

[9:20-10:00]  受付
[10:00-10:10]  1日目オープニング [slide]
[10:20-11:20] セッション1:  Large Language Models

• 西田京介: Llama 2: Open Foundation and Fine-Tuned Chat Models (arxiv; Hugo Touvron et al.) [paper][slide]

• 笹野遼平: Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM’s Translation Capability (ACL2023; Eleftheria Briakou, Colin Cherry, George Foster) [paper][slide]

• 清水周一郎: QLoRA: Efficient Finetuning of Quantized LLMs (arxiv; Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer) [paper][slide]

[11:35-12:15] セッション2:  Resource&Evaluation

• 佐々木翔大: Can Large Language Models Be an Alternative to Human Evaluations? (ACL2023; Cheng-Han Chiang, Hung-yi Lee) [paper][slide]

• 銭本友樹: Is GPT-3 a Good Data Annotator? (ACL2023; Bosheng Ding, Chengwei Qin, Linlin Liu, Yew Ken Chia, Boyang Li, Shafiq Joty, Lidong Bing) [paper][slide]

• 菅原朔: What’s the Meaning of Superhuman Performance in Today’s NLU? (ACL2023; Simone Tedeschi, Johan Bos, Thierry Declerck, Jan Hajič, Daniel Hershcovich, Eduard Hovy, Alexander Koller, Simon Krek, Steven Schockaert, Rico Sennrich, Ekaterina Shutova, Roberto Navigli) [paper][slide] （※都合により現地発表はなし）

[13:40-14:50] セッション3:  In-context Learning 1

• 西田光甫: What learning algorithm is in-context learning? Investigations with linear models. (ICLR2023; Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, Denny Zhou) [paper][slide]

• Benjamin Heinzerling: Transformers Learn In-Context by Gradient Descent (ICML2023; Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, Joao Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, Max Vladymyrov) [paper][slide] （※都合により現地発表はなし）

• 横井祥: A Theory of Emergent In-Context Learning as Implicit Structure Induction (arxiv; Michael Hahn, Navin Goyal) [paper][slide]

• スポンサー（ELYZA様）

• スポンサー（LINE様）

[15:00-15:55] セッション4:  In-context Learning 2 

• 加藤祥太: Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (ACL2023; Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, Huan Sun) [paper][slide]

• 村山友理: What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning (ACL2023 Findings; Jane Pan, Tianyu Gao, Howard Chen, Danqi Chen) [paper][slide]

• 上原康平: MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning (ACL2023; Zhiyang Xu, Ying Shen, Lifu Huang) [paper][slide]

[16:10-17:10] セッション5: Generalization/Interpretability

• 染谷大河: Neural Networks and the Chomsky Hierarchy. (ICLR2023; Grégoire Delétang, Anian Ruoss, Jordi Grau-Moya, Tim Genewein, Li Kevin Wenliang, Elliot Catt, Chris Cundy, Marcus Hutter, Shane Legg, Joel Veness, Pedro A. Ortega) [paper][slide]

• 石井太河: Transformers Learn Shortcuts to Automata. (ICLR2023; Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang) [paper][slide]

• 稲葉達郎: Backpack Language Models (ACL2023; John Hewitt, John Thickstun, Christopher Manning, Percy Liang) [paper][slide]

[17:20-18:20] セッション6: Language Model/Decoding

• 吉田遼: How Much Do Language Models Copy From Their Training Data? Evaluating Linguistic Novelty in Text Generation Using RAVEN (TACL2023; R. Thomas McCoy, Paul Smolensky, Tal Linzen, Jianfeng Gao, Asli Celikyilmaz) [paper][slide]

• 栗林樹生:

  • A Watermark for Large Language Models (ICML 2023; John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, Tom Goldstein) [paper][slide]

  • Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times? (TACL2023; Byung-Doh Oh, William Schuler) [paper][slide]

• 持橋大地: Locally Typical Sampling (TACL2023; Clara Meister, Tiago Pimentel, Gian Wiher, Ryan Cotterell) [paper][slide]
  
  

[18:20-18:30] 1日目クロージング [slide]

[19:00-] 懇親会

8月28日(月) プログラム

[9:20-9:50]  受付
[9:50-10:00]  2日目オープニング [slide]
[10:00-11:00] セッション7:  Interpretability

• 賀沢秀人: Analyzing Transformers in Embedding Space (ACL2023; Guy Dar, Mor Geva, Ankit Gupta, Jonathan Berant) [paper][slide]

• 李凌寒: What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary (ACL2023; Ori Ram, Liat Bezalel, Adi Zicher, Yonatan Belinkov, Jonathan Berant, Amir Globerson) [paper][slide]

• 田中涼太: Visual Classification via Description from Large Language Models. (ICLR2023; Sachit Menon, Carl Vondrick) [paper][slide]

[11:10-12:10] セッション8:  Multilingualism/Machine Translation

• 木山朔: The Geometry of Multilingual Language Model Representations (EMNLP2022; Tyler Chang, Zhuowen Tu, Benjamin Bergen) [paper][slide]

• 平澤寅庄: Knowledge Transfer in Incremental Learning for Multilingual Neural Machine Translation (ACL2023; Kaiyu Huang, Peng Li, Jin Ma, Ting Yao, Yang Liu) [paper][slide]

• 森下睦: When Does Translation Require Context? A Data-driven, Multilingual Exploration (ACL2023; Patrick Fernandes, Kayo Yin, Emmy Liu, André Martins, Graham Neubig) [paper][slide]

[13:10-14:10] セッション9:  Model Architectures

• 平岡達也: From Characters to Words: Hierarchical Pre-trained Language Model for Open-vocabulary Language Understanding (ACL2023; Li Sun, Florian Luisier, Kayhan Batmanghelich, Dinei Florencio, Cha Zhang) [paper][slide]

• 塚越駿: Hyena Hierarchy: Towards Larger Convolutional Language Models (ICML2023; Michael Poli, Stefano Massaroli, Eric Nguyen, Daniel Y Fu, Tri Dao, Stephen Baccus, Yoshua Bengio, Stefano Ermon, Christopher Re) [paper][slide]

• 牧野晃平: Hungry Hungry Hippos: Towards Language Modeling with State Space Models. (ICLR2023; Daniel Y. Fu, Tri Dao, Khaled Kamal Saab, Armin W. Thomas, Atri Rudra, Christopher Ré) [paper][slide]

[14:20-15:40] セッション10:  Linguistic Generalization

• 大羽未悠: How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases (ACL2023; Aaron Mueller, Tal Linzen) [paper][slide]

• 芳賀あかり: How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech (ACL2023; Aditya Yedetore, Tal Linzen, Robert Frank, R. Thomas McCoy) [paper][slide]

• 村岡雅康: When and Why Vision-Language Models Behave like Bags-Of-Words, and What to Do About It? (ICLR2023; Mert Yüksekgönül, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, James Zou) [paper][slide]

• 篠田一聡: Minding Language Models’ (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker (ACL2023; Melanie Sclar, Sachin Kumar, Peter West, Alane Suhr, Yejin Choi, Yulia Tsvetkov) [paper][slide]

[15:50-16:50] セッション11:  Unsupervised Learning/Parsing

• 芝原隆善: Weaker Than You Think: A Critical Look at Weakly Supervised Learning (ACL2023; Dawei Zhu, Xiaoyu Shen, Marius Mosbach, Andreas Stephan, Dietrich Klakow) [paper][slide]

• 上田亮: Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammars (ACL2023; Songlin Yang, Roger Levy, Yoon Kim) [paper][slide]

• 三輪誠: Prompting Language Models for Linguistic Structure (ACL2023; Terra Blevins, Hila Gonen, Luke Zettlemoyer) [paper][slide]

[17:00-18:00] セッション12:  Efficiency/Evaluation

• 井田龍希: Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models. (ICLR2023; Xiaoman Pan, Wenlin Yao, Hongming Zhang, Dian Yu, Dong Yu, Jianshu Chen) [paper][slide]

• 飯田大貴: SCOTT: Self-Consistent Chain-of-Thought Distillation (ACL2023; Peifeng Wang, Zhengyang Wang, Zheng Li, Yifan Gao, Bing Yin, Xiang Ren) [paper][slide]

• 高瀬翔: 

• Beyond neural scaling laws: beating power law scaling via data pruning (NeurIPS2022; Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, Ari Morcos) [paper][slide]

• Rogue Scores (ACL2023; Max Grusky) [paper][slide]
  
  

[18:00-18:10] 2日目クロージング [slide]