[Researchs]

Emotional Talking Face Generation

This paper proposes a novel approach called Cross-Modal Emotion Transfer (C-MET) to improve flexibility and extensibility of emotion expression in talking-face video generation.

Conventional label-based methods are limited by discrete categories, and audio-based methods struggle because emotional cues are entangled with linguistic content, making emotion disentanglement difficult.

To address these issues, C-MET learns emotion semantic vectors that capture the relationship between audio and visual domains and performs cross-modal emotion transfer by mapping these vectors from the speech space to the facial-expression embedding space.

Publication: "Cross-Modal Emotion Transfer for Emotion Editing in Talking Face Video", CVPR 2025 (2nd Author) [pdf]

Source Code: https://github.com/ChanHyeok-Choi/C-MET
Project Page: https://chanhyeok-choi.github.io/C-MET/

Multimodal Empathetic Dialogue Modeling

Developed a dialogue system that integrates non-verbal and para-linguistic cues into natural conversations with LLMs, and designed the first conversational model capable of generating para-linguistic outputs such as pitch, speaking pace, and speech monotony alongside text.

To support this, built MSenC (Multi Sensory Conversation), a multimodal dataset combining speech, facial expressions, and text to address the limitations of existing dialogue resources.

The system’s effectiveness was validated through both quantitative benchmarks and qualitative user studies.

Publication: "Towards Human-like Multimodal Conversational Agent by Generating Engaging Speech", Interspeech 2025 Oral (1st Author) [pdf]

Source Code: https://github.com/kimtaesu24/MSenC
Samples: https://kimtaesu24.github.io/

[Projects]

Recommender System in Signed Graph Neural Networks

Conducted research on session-based recommendation using Signed Graph Neural Networks, employing a graph attention network for message passing. Re-implemented the original TensorFlow model in PyTorch and performed experiments to reproduce the performance reported in the original paper. Additionally, explored several improvements to the model architecture and training strategies to enhance recommendation accuracy.

Source Code: https://github.com/ssu-dmlab/DGRec-pytorch

Semantic Segmentation for Autonomous Driving

Developed an improved semantic segmentation model for autonomous vehicles that outperformed the SegNet baseline while remaining lightweight. Integrated multiple datasets with unified color palettes to enhance training flexibility. The model incorporated local and global residual connections, CReLU, and edge maps in a lightweight design. Trained and validated the system on diverse datasets and produced a demo video showcasing its performance in real-world driving environments.

Award: Grand Prize (Issued by Jeonbuk National University, 2022)
Source Code: https://github.com/kimtaesu24/semantic_seg

Sentiment Analysis of Korean Sentences

Conducted sentiment analysis on Korean sentences by fine-tuning KoBERT, a model tailored to address the limitations of the original BERT in handling Korean text. Experiments were performed on the NAVER movie review dataset, and the results demonstrated improved performance compared to other pre-trained models for Korean sentiment classification.

Award: Excellence Award (Issued by Jeonbuk National University, 2021)
Source Code: https://github.com/kimtaesu24/KoBERT_with_NAVER

Object Detection for Custom Nutritional Supplements Service

Developed a device and mobile app that analyze dietary photos to identify nutrient deficiencies and recommend personalized supplements. As part of the software team, collected a food dataset, created bounding box annotations, and applied object detection using YOLO. Fine-tuned the YOLOv5 model to achieve improved detection performance while maintaining real-time inference.

Patent: “Custom Nutritional Supplements Service” KR-Application No. 10-2021-0110106

Game Development

Developed an idle RPG (clicker game) using Unity and C#, and released it on the Google Play Store. Designed the blueprint during the initial planning stage of development. Responsible for implementing core internal algorithms.

Award: Grand Prize (Issued by Jeonbuk National University, 2021)
Source Code: https://github.com/kimtaesu24/RasingMusk_git_Start
Playstore: https://play.google.com/store/apps/details?id=com.gpucompany.rasingmusk (expired)
Demo: https://youtu.be/3UOhGNvXz0I?si=WjZNYrOo5SEIavQZ

Page updated

Google Sites

Report abuse