Seungjun Lee
Seungjun Lee
Google Summer of Code 2023 @ TensorFlow Archive
Made a custom text summary dataset with 12,000 examples encompassing 13 types of document and average length of summary/document is 19% using PaLM API making it superior to publicly available dataset such as Xsum or CNN-DailyMail in Summary length and content diversity
Using this created text summary ml models fine-tuning T5, and GPT2. Doing experiments found out that fine-tuning GPT2 setting loss function to give same importance to original document and summary part yields the best result in terms of Rouge-L score.
Toxic - LLama GitHub
Engineered a method to jailbreak the LLaMA-3.2-1B-Instruct model using PPO, successfully reducing the refusal rate on 131 unsafe prompts from 91.6% 6.1%.
Innovated a training approach that exclusively used a safe prompt dataset and an RLHF dataset, bypassing the conventional need for SFT with toxic prompt-response pairs.
Evaluated model performance trade-offs, showing that while overall IFEval benchmark scores slightly decreased, the model achieved significant improvements in targeted instruction-following, including a +16% gain in case handling and +10% in constrained response generation.
Llama-finetuning GitHub
Developed a modular framework for fine-tuning Llama models on any text dataset, implementing LoRA and DoRA from scratch.
Fine-tuned the Llama 3.2-1B model on SQuAD v1, achieving cross-entropy losses of 1.5245 and 1.3891 on the SQuAD v1.0 dataset for LoRA and DoRA, respectively.
Image Colorization GitHub
Developed a flexible framework for training various generator architectures (U-Net, ResNet-backbone U-Net, and Attention U-Net) based on the Pix2Pix architecture for image colorization.
Trained on the COCO2017 dataset, achieving LPIPS scores of 0.2150, 0.2098, and 0.2247 for the respective models.
Pic Scribe GitHub
Developed and implemented an image captioning model using a Vision Transformer (ViT) encoder and a Transformer decoder architecture.
Trained the model on the COCO2017 dataset, achieving 0.640 CIDEr score on COCO2017 validation set.
Successfully deployed the model as an iOS application, leveraging CoreML for efficient on-device inference and SwiftUI for a seamless user interface. The app allows users to easily upload images and receive descriptive captions.
ML Implementations GitHub
Implemented Linear, Convolution, TransposeConvolution, MaxPool, RNN, LSTM, Embedding, MultiHeadAttention, Transformer, K-means clustering, and KV Cache using NumPy.
Implemented ViT, DenseNet-BC, DDColor, ResNet, YOLO, CRNN, FlashAttention, Diffusion (DDPM), ModernBERT, Conformer, MQA, GQA, and MLA using PyTorch.
Animal Classifier GitHub
Develop a web application that classifies a provided animal photograph into one of ten categories by creating and deploying a Flask app to Google Cloud Run.
Created a model using InceptionV3 as base, and trained it dataset from Kaggle, and achieved an 85% of accuracy on validation set.
Apple Swift Student Challenge 2021
Selected as one of 350 winners by building a net-worth tracking app that shows an allocation of net-worth in two hierarchies with pie charts.
MLH hack This Fall
My team and I took the 1st place among 4,000 students and awarded with 50,000 INR and $2000 USD worth of gift cards by creating an iOS app that helps people with speaking disorders to communicate by transforming emoji/text to audio using AVFoundation framework.