Projects

Towards Calibrated Robust Fine-Tuning of Vision-Language Models  @CMU

Team research project in Large-Scale Multimedia Analysis 23f 

 Sep. 2023 ~ Dec. 2023 

Vision-Language-model  CLIP  Uncertainty calibration  Robust fine-tuning 

• reviewing research overview & literature search 

• draft writing & code writing & experiment

• publication :  [pdf] 

Implementation of Diffusion Attentive Attribution Maps  @CMU

Implementation project in Natural Language Processing 23f 

 Sep. 2023 ~ Dec. 2023 

NLP Explainable AI  Stable Diffusion Model  Attribution map

• Theoretical study and pipeline structure analysis of stable diffusion

• Implementation of Diffusion Attentive Attribution Map (DAAM) 

• Designing additional experiments and analysis for meaningful interpretation of attribution map 

• Technical report writing

Post-hoc calibration using Ornstein-Uhlenbeck process   @CAU

XAI  Uncertinty calibration SDE Ornstein-Uhlenbeck process Temperature Scaling

Team research project in Explainable Artificial Intelligence 22f  

 Sep. 2022 ~ Dec. 2022 

• Evaluation on experiment method and proposed method 

• Experiment using synthetic data and visualization  

• Report writing 

Highway Drone Control Automation  @NextLab

OpticalFlow DeepSort EfficientNet HoughTransform CannyEdgeDetection FeatureDescriptor 

Automate existing drone control work in highway, recognizing first lane region and vehicle type 

 Nov. 2021 ~ Dec. 2021 

• development of first lane recognition 

• make a pipeline between lane recognition module and vehicle type recognition 

• make a real demo at cheon-an highway

Subtitle Anomaly Detection  @NextLab

EasyOCR CRAFT multi-threading  ONNX

Create an algorithm checking subtitle anomaly in streaming videos 

 Sep. 2021 

• create subtitle datasets   

• develop a subtitle detection & recognition module 

• Implementation of whole pipeline including multi-processing 

Handwriting & News PDF OCR  @NextLab

CRAFT deep-text-recognition-benchmark CTC ResNet EasyOCR VGGNet LSTM

OCR with text image in PDF 

 Oct. 2021 ~ Dec. 2021 

• Implementation of End-to-End OCR pipeline using the model structure 

• various experiment with data augmentation and pre-, post-processing 

• make better performance than original EasyOCR in Korean recognition 

Table Detection  @NextLab

MNIST Retinex OpenCV 

recognizing table’s cell and handwritten numbers 

 Oct. 2021 

• get table’s cell contour position with image processing  

• Develop a post-processing algorithm to fill cells that are missing due to light smudging 

• make better performance of handwriting recognition with using Retinex 

P&ID (Piping and Instrumentation Diagram) @JBNU

Recognize symbols and text of existing factory drawings and digitalize them 

 Apr. 2019 ~ Aug.2021 

 P&ID TesseractOCR EasyOCR CLEval MVVM

• text detection & recognition

• test and study various OCR models

• create a pre, post processing algorithm to improve performance

• make better OCR performance

• make GUI to visualize and modify drawing recognition results 

• publication : [pdf] 

Contribution to EasyOCR   @JBNU

EasyOCR git 

Contribution to a popular OCR framework contribution 

 Aug. 2020 ~ Nov.2020

• solve a frequently uploaded issue    

• add a feature which makes possible to recognize rotated text 

• fix an error about post processing logic  

Colleful  @JBNU

Java Spring Boot git 

Development of meeting matching platform 

 Aug. 2020 ~ Nov. 2020 

• Develop back-end with java spring boot

• Implementation of team matching API 

• Implementation of User management API for admin

• Develop a department info crawler 

• Experience development rules and collaborative culture 

Mask Detector  @JBNU

Raspberry pi Embedded system MobileNet 

Mask Detector against Covid-19 using Raspberry pi 4 

 Jun. 2021

• transfer learning with Mobile Net V2 using kaggle’s mask dataset  

• Implementation of running buzzer, camera, LED module 

Op (Pregnancy Seat Assistant) @JBNU

Arduino C Hackathon

Development of Policies, Services, and an App for the Smooth Utilization of Pregnancy Seats in Subways

Sep. 2020

• Android app development

• Idea conceptualization

• Idea development

CIMR (Crop big Image and Merge detected Results)  @JBNU

OpenCV pre-, post-processing algorithm 

post-processing algorithms designed to solve memory errors which occur during large image inference 

 Dec. 2020 

• creat an algorithm