Research Interests

My research aims to build an AI agent that can think and learn like humans. To do so, I focus on challenging issues in the current learning system both in the 'model' and 'data' perspective. Specifically, there are two principal directions I have explored. First, apart from the current model design paradigm, I attempted to incorporate the attention mechanism, a central component of the human vision system, into the model. Second, to reduce expensive and error-prone human supervision, I tried to exploit self-supervision or simulation and scale the training data up to a billion-scale.

My work explores following areas

• • Visual Perception (Categorization, Detection, Segmentation)

  • Visual Reasoning (Scene Graph, Summarization, Stroytelling)

  • Image and Video Processing (Synthesis, Translation)

  • Unsupervised / Self-supervised Learning

  • Simulated Learning and Domain Adaptation

but not limited to.

Research Experience

• Adobe Research, San Jose, CA

• KAIST, Daejeon, Korea

International Publications (Google Scholar)

(* indicates equal contribution)

2021

• • The Devil is in the Boundary: Exploiting Boundary Representation for Basis-based Instance Segmentation [Paper]

2020

• • Discover, Hallucinate, and Adapt: Open Compound Domain Adaptation for Semantic Segmentation [Paper]

• • Align-and-Attend Network for Globally and Locally Coherent Video Inpainting [Paper]

• • Two-Phase Pseudo Label Densification for Self-training based Domain Adaptation [Paper]

• • Global-and-Local Relative Position Embedding for Unsupervised Video Summarization [Paper]

• • Video Panoptic Segmentation [Paper] [Code]

• • A Simple and Light-weight Attention Module for Convolutional Neural Networks [Paper]

• • Recurrent Temporal Aggregaation Framework for Deep Video Inpainting [Paper]

• • Hide-and-Tell: Learning to Bridge Photo Streams for Visual Storytelling [Paper]

• • Propose-and-Attend Single Shot Detector [Paper]

2019

• • Preserving Semantic and Temporal Consistency for Unpaired Video-to-Video Translation [Paper]

• • Video Retargeting: Trade-off between Content Preservation and Spatio-temporal Consistency [Paper]

• • Deep Video Inpainting [Paper] [Project] [Code]

• • Deep Blind Video Decaptioning by Temporal Aggregation and Recurrence [Paper] [Project] [Code]

• • Gated Bidirectional Feature Pyramid Network for Accurate One Shot Detection [Paper]

• • Discriminative Feature Learning for Unsupervised Video Summarization [Paper]

2018

• • LinkNet: Relational Embedding for Scene Graph [Paper]

• • CBAM: Convolutional Block Attention Module [Paper] [Code]

• • BAM: BottleNeck Attention Module [Paper] [Supp] [Code]

• • StairNet: Top-Down Semantic Aggregation for Accurate One Shot Detection [Paper] [Supp]

Academic Activities

• Journal Reviewer

-Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

-International Journal of Computer Vision (IJCV)

-Transactions on Image Processing (TIP)

-Transactions on Circuits and Systems for Video Technology (TCSVT)

-Transactions on Cybernetics

-Machine Vision Applications (MVA)

-Pattern Recognition (PR)

-Neurocomputing

-IEEE Access

• Conference Reviewer

-CVPR 2019, 2020, 2021

-ICCV 2019, 2021

-ECCV 2020

-BMVC 2020

-WACV 2020, 2021

-ACCV 2020

-NeurIPS 2020

-ICLR 2021

-AAAI 2020

Awards & Honors

• Awards

-KAIST-Samsung Industry-University Cooperation Best Paper Award ($3,000), July 2020

-Honorable Mention, 25th HumanTech Paper Award, Samsung Electronics Co., Ltd. ($2,000), Feb 2019

-1st Place Award in ChaLearnLAP 2018 Inpainting Challenge Track 2 – video decaptioning (ECCV2018 Challenge), Sep 2018 [factsheet]

-Gold Prize, 24th HumanTech Paper Award, Samsung Electronics Co., Ltd. ($10,000), Feb 2018

-Honorable Mention, 24th HumanTech Paper Award, Samsung Electronics Co., Ltd. ($2,000), Feb 2018

-Silver Prize, Best Paper Award, 30th IPIU 2018, Feb 2018

• Honors

-Microsoft Research Asia Ph.D. Fellowship 2020 Winner ($10,000), Nov 2020 [Certificate]

-AI summer school (PAISS 2018), July 2018