Research

Efficient Generative Modeling with Residual Vector Quantization-Based Tokens

Jaehyeon Kim*, Taehong Moon*, Keon Lee, Jaewoong Cho

arXiv 2024

[paper][demo]

DiTTo-TTS: Efficient and Scalable Zero-Shot Text-to-Speech with Diffusion Transformer

Keon Lee, Dong Won Kim, Jaehyeon Kim, Jaewoong Cho

arXiv 2024

[paper][demo]

Mini-Batch Optimization of Contrastive Loss

Jaewoong Cho*, Kartik Sreenivasan*, Keon Lee, Kyunghoo Mun, Soheun Yi, Jeong-Gwan Lee, Anna Lee, Jy-yong Sohn, Dimitris Papailiopoulos, Kangwook Lee

TMLR 2024

[paper][code]

CLaM-TTS: Improving Neural Codec Language Model for Zero-Shot Text-to-Speech

Jaehyeon Kim, Keon Lee, Seungjun Chung, Jaewoong Cho

ICLR 2024

[paper][demo]

Censored Sampling of Diffusion Models Using 3 Minutes of Human Feedback

TaeHo Yoon, Kibeom Myoung, Keon Lee, Jaewoong Cho, Albert No, Ernest K. Ryu

NeurIPS 2023

[paper][code]

Mini-Batch Optimization of Contrastive Loss

Kartik Sreenivasan, Keon Lee, Jeong-Gwan Lee, Anna Lee, Jaewoong Cho, Jy-yong Sohn, Dimitris Papailiopoulos, Kangwook Lee

ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models

[paper]

DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speech

Keon Lee*, Kyumin Park*, Daeyoung Kim

ICASSP 2023

[paper][code]

RedPen: Region- and Reason-Annotated Dataset of Unnatural Speech

Kyumin Park, Keon Lee, Daeyoung Kim, Dongyeop Kang

arXiv 2022

[paper]

STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speech

Keon Lee, Kyumin Park, Daeyoung Kim

INTERSPEECH 2021

[paper][demo][code]

Academic Services

Conference Reviewer: NeurIPS (2024), ICLR (2025), ICASSP (2025)

Award

1st Papers with Code Contributor Award, 2022

by The Papers with Code Team

I’ve been selected as a recipient of the first ever Papers with Code Contributor Award by my ongoing contributions to Papers with Code!

Education

M.S. in School of Computing, Korea Advanced Institute of Science and Technology (KAIST)

Advised by Daeyoung Kim, Mar 2020 - Mar 2022

Area: Expressive Speech Synthesis

B.S. in Industrial Design & School of Computing, Korea Advanced Institute of Science and Technology (KAIST)

Mar 2014 - Mar 2020

Experience

Selected Project

PyTorch Implementations

Github Project, May 2021 - Present

• Please visit my Github repositories to see all the ongoing projects!

• Keep daily surveying on deep learning based Audio and Speech Processing, mainly by arXiv.

• Implement a paper at best in a week as the fastest and most accurate repo in the community.

• Collaborate with Github's big names to implement/resolve/improve so many exciting projects.

• I’ve been selected as a recipient of the first ever Papers with Code Contributor Award by my ongoing contributions to Papers with Code!

Pintos

Course Project: Operating Systems and Lab, Aug 2019 - Dec 2019

• Passed all the Pintos projects (1, 2, 3, and 4).

• Understood the basic concepts of operating systems and C programming in complex programs.

Speech Assistant for English Learner

Personal Project, Apr 2019

• A real-time checking system for English speaking style by STT and TTS modules.

• This project is not for general users, but for my mother's English practice.

Homepage

Personal Project, Jun 2019 - present

• Built personal web pages using the latest technology in React.

개발자그리고디자이너 (Programmers and Designers)

Personal Project, Sep 2017 - Oct 2019

• A personal project that uses Selenium, BeautifulSoup, and Pyqt5 (for GUI) to deliver programs (crawlers and automated web testers in Python) to various customers. Example tasks include extracting movie information, analyzing social media (Instagram, Facebook) posts using NLTK and KoNLTK, and collecting real-time stock price information.

• One of the various projects was carried out as part of a government project with the Korea Robot Industry Association. In that project, job information from multiple sites was crawled and analyzed according to given criteria to ascertain current employment in robotics and artificial intelligence.

• Managed the service center through the Kakao Channel and attracted customers through the freelance platform called Kmong.

Linky @ E5 KAIST

Start-up, Jun 2017 - Dec 2017

• Aimed to provide a service that overcomes loneliness by sharing the sounds of everyday life and empathizing with each other.

• Unfortunately, it could not be realized in production, but it provided an experience for the startup ecosystem.

System Design for National Science Museum

Course Project: System Design, Mar 2017 - Jun 2017

• Recruited elementary school students and parents who are interested in the project.

• The goal was to maintain a high revisit rate of National Science Museum.

• The requirements were identified by applying the system design process, such as affinity diagrams. Some of our solutions have actually been incorporated into their program.

Graduation Project @ ID KAIST

Advised by KunPyo Lee, Mar 2017 - Dec 2017

• An assistant (application) for the gold generation that helps them remember the memories of social networks.

Fruiteria

Course Project: Entrepreneurship & Startup Creation, Mar 2016 - Jun 2016

• Deliver different combinations of fruits every day to 40 people for 3 weeks. More than 90 people contacted us on the Facebook page I managed at the time.

• The response has been very good as there is not yet a fruit-related delivery platform on campus. Many students were looking forward to the launch!

• Unfortunately, I cannot proceed further due to the remaining semester. However, I do remember that a similar service came out after Frutheria.

Teaching Experience