Efficiency in African NLP @Deep Learning Indaba

Paul Azunre

Paul Azunre holds a PhD in Computer Science from MIT and has served as a Principal Investigator on several DARPA research programs. He founded Algorine Inc., a Research Lab dedicated to advancing AI/ML and identifying scenarios where they can have a significant social impact. Paul also co-founded Ghana NLP, an open-source initiative focused on using NLP and Transfer Learning with Ghanaian and other low-resource languages. He also serves as Director of Research at Dun & Bradstreet, a company helping businesses manage supply chain risk and other business analytics challenges. He is the author of the recently published book "Transfer Learning for NLP" by Manning Publications.

African NLP - Beyond Global Big Tech

In this talk, I will describe how GhanaNLP and Algorine Research built Khaya AI, the world's first machine translation and speech recognition app and API for several Ghanaian, Nigerian, and Kenyan Languages. I will discuss how this system was deployed on a shoestring budget to hundreds of thousands of users across available platforms. Our discourse will challenge prevailing research paradigms that have garnered support from prominent Big Tech corporations, particularly those emphasizing "zero resource translation," predicated upon extensive scraping of monolingual data and the relentless expansion of model sizes. By juxtaposing recent evaluations of our system vis-à-vis those advanced by Big Tech entities, we will demonstrate that prioritizing the curation of superior-quality multilingual data - and thereby investing in the communities that actually own the language and culture yields simultaneously smaller and better models. Furthermore, this comparative analysis will underscore the detrimental impact of exaggerated assertions and hype surrounding suboptimal models, as exemplified by Meta's NLLB, which undermine the local communities doing this important work. I will explore a collaborative initiative involving the Distributed AI Research Institute and Lesan AI to build a unified system with better coverage of African Languages by uniting local organizations serving their own communities and building the best tools for their own languages. I will argue that this, as opposed to a monopoly by Global Big Tech - is in the cultural and national security interests of our communities.

Machel Reid

Machel Reid is a research scientist at Google DeepMind working on NLP research, with a focus on multilingual NLP. Recently, He has been working a lot with LLMs and instruction tuning, developing recipes to boost multilingual capabilities in LLMs.

Extending Large Language Models Beyond English

Large language models (LLMs) have achieved state-of-the-art results on a wide range of natural language processing tasks. However, most highly performant LLMs are trained on largely monolingual data in English, which limits their ability to generalize to other languages. This talk will discuss recent advances in extending LLMs beyond English. We will cover approaches in multilingual pre-training, zero-shot cross-lingual transfer in an in-context learning setting, and multilingual instruction tuning. We will also discuss the challenges of working with LLMs in low-resource languages, and the opportunities for future research.

Monojit Choudhury

Monojit Choudhury is a Principal Data and Applied Scientist at Microsoft India. They build large universal language models that forms the backbone of various Microsoft products. Prior to this, he was a Principal researcher at Microsoft Research Lab India, and I still strongly collaborate with my colleagues from MSR. His research interests cut across the areas of Linguistics, Cognition, Computation and Society. He has a B.Tech and PhD in Computer Science and Engineering from IIT Kharagpur and had been at Microsoft Research since 2007.