Contact Information
Associate Professor
Department of Artificial Intelligence
402 Woojung Hall of Informatics, 145 Anam-ro, Seongbuk-gu, Seoul, 02841
Republic of Korea
E-mail: sungwoong.kim01 at gmail dot com
[CV]
About Me
I am an Associate Professor in the department of Artificial Intelligence at Korea University. I direct AGI Lab where our research focuses on realizing artificial general intelligence to make an AI agent can perform task generalization and self-learning. I received BS and Ph.D degrees from KAIST in 2004 and 2011, respectively. When I was a graduate student, my research area was machine learning, especially applied to speech and image processing, under supervision by Professor Chang D. Yoo. In the middle of my Ph.D course, I performed research internships at National ICT Australia under supervision by Dr. Alex Smola and Mircosoft Research Cambridge under supervision by Dr. Pushmeet Kohli and Dr. Sebastian Nowozin. After receiving my Ph.D degree, I was in KAIST as a postdoc researcher, and then worked as a staff engineer at Qualcomm Research Korea. In 2017, I joined Kakao Brain and worked as a research scientist conducting several research projects mostly related to artificial general intelligence. Then, in March 2023, I joined Korea University.
Education
B.S. in Electrical Engineering, KAIST, Aug. 2004.
Ph.D. in Electrical Engineering, KAIST, Aug. 2011.
Research internship at National ICT Australia (NICTA), working on structured support vector machine for signal processing supervised by Dr. Alexander J. Smola, April 2008 ~ July 2008.
Research internship at Microsoft Research Cambridge, working on task-dependent unsupervised image segmentation supervised by Dr. Pushmeet Kohli and Dr. Sebastian Nowozin, April 2010 ~ May 2010.
Employment
Post Doc. in Electrical Engineering, KAIST, Sep. 2011 ~ Mar. 2012.
Staff Engineer, Qualcomm Research Korea, Mar. 2012 ~ Aug. 2017.
Research Scientist, Kakao Brain, Sep. 2017 ~ Feb. 2023.
Associate Professor, Korea University, Mar. 2023 ~ Present.
Research Area
Artificial General Intelligence, Machine Learning, Graphical Modeling, Deep Learning, Meta Learning, Representation Learning, Generative Modeling, Reinforcement Learning, Optimization, Multi-Modal Learning, Language Modeling, Multimedia Signal Processing.
Ph.D. Dissertation
"Task-specific Image Partitioning based on Discriminative Higher-Order Correlation Clustering", Department of EE, Korea Advanced Institute of Science and Technology, Aug., 2011.
Project Experiences
Language Modeling (Mar. 2022 ~ Present).
Multi-Modal Generative Modeling (Mar. 2022 ~ Present).
Agent Learning (April 2021 ~ Mar. 2022).
AutoLearn (Sep 2018 ~ April 2021).
Meta Learning (Sep 2017 ~ Sep 2018).
Development of the on-device learning system for smart IoT (May 2016 ~ Aug. 2017).
Development of the text recognition system for hand-held devices (Jan. 2014 ~ Apr. 2016).
Development of the copyright protection system for multimedia data based on the detection of semantically coherent contents (Mar. 2009 ~ Aug. 2011).
Development of a speech based interface for human robot interaction (May 2009 ~ Aug. 2011).
Multimedia retrieval system based on the detection of semantically coherent contents (Mar. 2009 ~ Aug. 2011).
Development of the image contents annotation, search and retrieval system. (Dec. 2009 ~ June 2010).
Development of feature extraction and indexing for content-based video identification (July 2008 ~ June 2010).
Speaker recognition system (Sep. 2008 ~ Feb. 2009).
Development of the speech and music based human emotion recognition system for intelligent robotics (Apr. 2007 ~ Mar. 2008).
Speech recognizer on the embedded linux system for wearable ubiquitous computer (Mar. 2007 ~ Feb. 2008).
Audio fingerprinting system for copyright projection (May 2005 ~ Nov. 2006).
Publications
International Journal Papers
Patrick Bilic, Patrick Christ, Hongwei Bran Li, Eugene Vorontsov, Avi Ben-Cohen, Georgios Kaissis, Adi Szeskin, Colin Jacobs, Gabriel Efrain Humpire Mamani, Gabriel Chartrand, Fabian Lohöfer, Julian Walter Holch, Wieland Sommer, Felix Hofmann, Alexandre Hostettler, Naama Lev-Cohain, Michal Drozdzal, Michal Marianne Amitai, Refael Vivanti, Jacob Sosna, Ivan Ezhov, Anjany Sekuboyina, Fernando Navarro, Florian Kofler, Johannes C. Paetzold, Suprosanna Shit, Xiaobin Hu, Jana Lipková, Markus Rempfler, Marie Piraud, Jan Kirschke, Benedikt Wiestler, Zhiheng Zhang, Christian Hülsemeyer, Marcel Beetz, Florian Ettlinger, Michela Antonelli, Woong Bae, Míriam Bellver, Lei Bi, Hao Chen, Grzegorz Chlebus, Erik B. Dam, Qi Dou, Chi-Wing Fu, Bogdan Georgescu, Xavier Giró-i-Nieto, Felix Gruen, Xu Han, Pheng-Ann Heng, Jürgen Hesser, Jan Hendrik Moltz, Christian Igel, Fabian Isensee, Paul Jäger, Fucang Jia, Krishna Chaitanya Kaluva, Mahendra Khened, Ildoo Kim, Jae-Hun Kim, Sungwoong Kim, Simon Kohl, Tomasz Konopczynski, Avinash Kori, Ganapathy Krishnamurthi, Fan Li, Hongchao Li, Junbo Li, Xiaomeng Li, John Lowengrub, Jun Ma, Klaus Maier-Hein, Kevis-Kokitsi Maninis, Hans Meine, Dorit Merhof, Akshay Pai, Mathias Perslev, Jens Petersen, Jordi Pont-Tuset, Jin Qi, Xiaojuan Qi, Oliver Rippel, Karsten Roth, Ignacio Sarasua, Andrea Schenk, Zengming Shen, Jordi Torres, Christian Wachinger, Chunliang Wang, Leon Weninger, Jianrong Wu, Daguang Xu, Xiaoping Yang, Simon Chun-Ho Yu, Yading Yuan, Miao Yue, Liping Zhang, Jorge Cardoso, Spyridon Bakas, Rickmer Braren, Volker Heinemann, Christopher Pal, An Tang, Samuel Kadoury, Luc Soler, Bram van Ginneken, Hayit Greenspan, Leo Joskowicz, and Bjoern Menze, "The Liver Tumor Segmentation Benchmark (LiTS)", Medical Image Analysis, vol. 84, Feb., 2023.
Jorg H. Kappes, Bjoern Andres, Fred A. Hamprecht, Christoph Schnorr, Sebastian Nowozin, Dhruv Batra, Sungwoong Kim, Bernhard X. Kausler, Thorben Kroger, Jan Lellmann, Nikos Komodakis, Bogdan Savchynskyy, and Carsten Rother, "A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems", International Journal of Computer Vision, vol. 115, no. 2, pp. 155-184, Nov., 2015.
Sungwoong Kim, Chang D. Yoo, Sebastian Nowozin, and Pushmeet Kohli, "Image Segmentation Using Higher-Order Correlation Clustering", IEEE Transaction on Pattern Analysis and Machine Intelligence, vol.36, no.9, pp.1761-1774, September 2014.
Sungwoong Kim, Sebastian Nowozin, Pushmeet Kohli, and Chang D. Yoo, "Task-Specific Image Partitioning", IEEE Transaction on Image Processing, vol. 22, no. 2, pp. 488-500, Feb., 2013.
Sungwoong Kim, Sungrack Yun, and Chang D. Yoo, "Large Margin Discriminative Semi-Markov Model for Phonetic Recognition", IEEE Transaction on Audio, Speech, and Language Processing, vol.19, no.7, pp.1999-2012, September 2011.
O. Thomas, P. Sunehag, G. Dror, Sungrack Yun, Sungwoong Kim, M. Robards, A. Smola, D. Greene, and P. Saunders, "Wearable-Sensor Activity Analysis Using Semi-Markov Models with a Grammar", Pervasive and Mobile Computing, vol. 6, no. 3, pp. 342--350, June 2010.
Dalwon Jang, Chang D. Yoo, Sunil Lee, Sungwoong Kim, and Ton Kalker, "Pairwise Boosted Audio Fingerprint", IEEE Trans. Information Forensics and Security, vol.4, no.4, pp.995-1004, December 2009.
International Conference Papers
Gunsoo Han, Daejin Jo, Daniel Wontae Nam, Eunseop Yoon, Taehwan Kwon, Seungeun Rho, Kyoung-Woon On, Chang D. Yoo, Sungwoong Kim, "Efficient Latent Variable Modeling for Knowledge-Grounded Dialogue Generation", The Conference on Empirical Methods in Natural Language Processing (EMNLP) 2023 (Long, Findings).
Eunbi Yoon, Keehun Park, Sungwoong Kim, Sungbin Lim, "Score-based Generative Models with Levy Processes", Neural Information Processing Systems (NeurIPS) 2023 (spotlight).
Sungwoong Kim*, Daejin Jo*, Donghoon Lee*, and Jongmin Kim*, "MAGVLT: Masked Generative Vision-and-Language Transformer", IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2023 (*: equal contributions).
Daejin Jo*, Sungwoong Kim*, Daniel Wontae Nam*, Taehwan Kwon, Seungeun Rho, Jongmin Kim, and Donghoon Lee, "LECO: Learnable Episodic Count for Task-Specific Intrinsic Reward", Neural Information Processing Systems (NeurIPS) 2022 (*: equal contributions).
Daejin Jo, Taehwan Kwon, Eun-Sol Kim, and Sungwoong Kim, "Selective Token Generation for Few-shot Natural Language Generation", International Conference on Computational Linguistics (COLING) 2022 (oral).
Eric Hambro, Sharada Mohanty, Dmitrii Babaev, Minwoo Byeon, Dipam Chakraborty, Edward Grefenstette, Minqi Jiang, Daejin Jo, Anssi Kanervisto, Jongmin Kim, Sungwoong Kim, Robert Kirk, Vitaly Kurin, Heinrich Kuttler, Taehwon Kwon, Donghoon Lee, Vegard Mella, Nantas Nardelli, Ivan Nazarov, Nikita Ovsov, Jack Parker-Holder, Roberta Raileanu, Karolis Ramanauskas, Tim Rocktaschel, Danielle Rothermel, Mikayel Samvelyan, Dmitry Sorokin, Maciej Sypetkowski, and Michal Sypetkowski, "Insights From the NeurIPS 2021 NetHack Challenge", arXiv:2203.11889, 2022.
Doyup Lee, Sungwoong Kim, Ildoo Kim, Yeongjae Cheon, Minsu Cho, and Wook-Shin Han, "Contrastive Regularization for Semi-Supervised Learning", IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) L3D-IVU Workshop, 2022.
Chiheon Kim, Saehoon Kim, Jongmin Kim, Donghoon Lee, and Sungwoong Kim, "Automated Learning Rate Scheduler for Large-batch Training", International Conference on Machine Learning (ICML) AutoML Workshop, 2021.
Saehoon Kim, Sungwoong Kim, and Juho Lee, "Hybrid Generative-Contrastive Representation Learning", arXiv:2106.06162, 2021.
Byungseok Roh, Wuhyun Shin, Ildoo Kim, and Sungwoong Kim, "Spatially Consistent Representation Learning", IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Taesup Kim, Sungwoong Kim, and Yoshua Bengio, "Visual Concept Reasoning Networks", AAAI Conference on Artificial Intelligence (AAAI), 2021.
Ildoo Kim, Younghoon Kim, and Sungwoong Kim, "Learning Loss for Test-Time Augmentation", Neural Information Processing Systems (NeurIPS), 2020.
Woonhyuk Baek, Ildoo Kim, Sungwoong Kim, and Sungbin Lim, "AutoCLINT: The Winning Method in AutoCV Challenge 2019", arXiv:2005.04373, 2020.
Chiheon Kim, Heungsub Lee, Myungryong Jeong, Woonhyuk Baek, Boogeon Yoon, Ildoo Kim, Sungbin Lim, and Sungwoong Kim, "torchgpipe: On-the-fly Pipeline Parallelism for Training Giant Models", arXiv:2004.09910, 2020.
Ildoo Kim, Woonhyuk Bae, and Sungwoong Kim, "Spatially Attentive Output Layer for Image Classification", IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Sangwoo Mo, Chiheon Kim, Sungwoong Kim, Minsu Cho, and Jinwoo Shin, "Mining GOLD Samples for Conditional GANs", Neural Information Processing Systems (NeurIPS), 2019.
Sungbin Lim, Ildoo Kim, Taesup Kim, Chiheon Kim, and Sungwoong Kim, "Fast autoaugment", Neural Information Processing Systems (NeurIPS), 2019.
Sungwoong Kim, Ildoo Kim, Sungbin Lim, Woonhyuk Baek, Chiheon Kim, Hyungjoo Cho, Boogeon Yoon, and Taesup Kim, "Scalable Neural Architecture Search for 3D Medical Image Segmentation", International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), 2019.
Jongmin Kim, Taesup Kim, Sungwoong Kim, and Chang D Yoo, "Edge-Labeling Graph Neural Network for Few-shot Learning", IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, USA, June. 2019.
Taesup Kim, Jaesik Yoon, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn, "Bayesian Model-Agnostic Meta-Learning", Neural Information Processing Systems (NeurIPS), 2018.
Jorg H. Kappes, Bjoern Andres, Fred A. Hamprecht, Christoph Schnorr, Sebastian Nowozin, Dhruv Batra, Sungwoong Kim, Bernhard X. Kausler, Jan Lellmann, Nikos Komodakis, and Carsten Rother, "A Comparative Study of Modern Inference Techniques for Discrete Energy Minimization Problems", IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Portland, USA, June. 2013.
Sungwoong Kim, Sebastian Nowozin, Pushmeet Kohli, and Chang D. Yoo, "Higher-Order Correlation Clustering for Image Segmentation", Neural Information Processing Systems (NIPS), Granada, Spain, Dec. 2011.
Sungwoong Kim, Jongmin Kim, Sungrack Yun, and Chang D. Yoo, "$\nu$-Structured Support Vector Machines", IEEE International Workshop on Machine Learning for Signal Processing (MLSP), Kittila, Finland, pp. 450--455, Aug. 2010.
Sungwoong Kim, Sungrack Yun, and Chang D. Yoo, "Large Margin Training of Semi-Markov Model for Phonetic Recognition", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Dallas, USA, pp. 1910--1913, Mar. 2010.
Sungwoong Kim, Sungrack Yun, and Chang D. Yoo, "Margin-Enhanced Maximum Mutual Information Estimation for Hidden Markov Models", IEEE International Symposium on Industrial Electronics (ISIE), Seoul, Korea, pp. 1347--1351, July 2009.
Sungwoong Kim, and Chang D. Yoo, "Boosted Binary Audio Fingerprint Based on Spectral Subband Moments", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Honolulu, USA, pp. 241--244, April 2007.
Patents
Ildoo Kim, Sungbin Lim, Taesup Kim, Chiheon Kim, Sungwoong Kim, Data augmentation method and apparatus, and computer program, 10-2300903, (Republic of Korea), 2021.
Hee-Seok Lee, Kang Kim, Duck Hoon Kim, SungWoong Kim, Seok-Soo Hong, Method and device for capturing image of traffic sign, 10,325,339, (US), 2019.
Kyu Woong Hwang, Seungwoo Yoo, Duck-hoon Kim, SungWoong Kim, Te-Won Lee, Context-based access verification, 9,916,431, (US), 2018.
Sungrack Yun, Taesu Kim, SungWoong Kim, Heeman Kim, Electronic device generating notification based on context data in response to speech phrase from user, 9,946,862, (US), 2018.
Yongwoo Cho, Duck Hoon Kim, SungWoong Kim, Minho Jin, Kyuwoong Hwang, Display device adjustment by control device, 9,495,004, (US), 2016.
SungWoong Kim, Kyuwoong Hwang, Taesu Kim, Duck-hoon Kim, Minho Jin, Yongwoo Cho, Automatic authorization for access to electronic device, 9,514,296, (US), 2016.
Chang D. Yoo, Sungwoong Kim, Image Segmentation Method Using Higher-Order Clustering, System for Processing the Same and Recording Medium for Storing the Same, 9,111,356, (US), 2015.
Chang D. Yoo, Sungwoong Kim, Sanghyuk Park, Image Partitioning Method Using High-Order Correlation Clustering, System Processing The Method and Recording Medium, 10-1348904, (Republic of Korea), 2013.
Chang D. Yoo, Sungwoong Kim, Sanghyuk Park, Image Partitioning Method Using Correlation Clustering, System Processing The Method and Recording Medium, 10-1356629, (Republic of Korea), 2014.
Chang D. Yoo, Sungwoong Kim, Continuous Phoneme Recognition Method Using Semi-Markov Model, System Processing The Method and Recording Medium, 10-1359689, (Republic of Korea), 2014.
Chang D. Yoo, Sanghyuk Park, Sungrack Yun, Sungwoong Kim, Advanced Driver Assistance System for Safety Driving Using Driver Adaptive Irregular Behavior Detection and Method thereof, 10-1276770, (Republic of Korea), 2013.
Chang D. Yoo, Sanghyuk Park, Sungrack Yun, Sungwoong Kim, Augmented Reality System for Head-Up Display, 10-1359660, (Republic of Korea), 2014.
Chang D. Yoo, Sanghyuk Park, Sungwoong Kim, Face Detection System and Method Using Skin Color Filtering and Morphology Processing, 10-1143555, (Republic of Korea), 2012.
Honors
Government Scholarship (both Undergraduate and Graduate).
16th Samsung Humantech Thesis Prize, Bronze, 2010.
AutoCV1, AutoCV2 Challenges, 1st place, NeurIPS 2019.
Nethack Challenge, 2nd place in neural net track, NeurIPS, 2021.
Activities
Reviewer of IEEE Transactions on Pattern Analysis and Machine Intelligence.
Reviewer of IEEE Transactions on Image Processing.
Reviewer of IEEE Transactions on Audio, Speech, and Language Processing.
Reviewer of IEEE Signal Processing Letters.
Reviewer of several conferences such as ICML, CVPR, ICCV, NeurIPS, etc.
Area Chair of NeurIPS, ICML.
Conducting a lot of lectures and talks with regard to machine learning.