Keisuke Sakaguchi

Tohoku University

Associate Professor

Recent advances in natural language processing mainly come from Large Language Models (LLMs) such as BERT and GPT3. Many applications, such as machine translation, summarization, proofreading, and dialogue with chatbots, are deployed as profitable business services in industries and start-ups. The core engine of LLMs is the Encoder-Decoder (Transformers) with the attention mechanism, and the fuel is an enormous amount of data on the web. In this talk, I will briefly overview how LLMs work and what they have achieved, followed by discussing the limitations and future directions along with my recent projects.