Zillow Group, Seattle, WA. 06/2021 - Present

Senior Applied Scientist, Department of AI, AI Media Insights

Open-set Home Image Understanding : Developed vision-language models to achieve open-set image classification , object detection, and semantic segmentation capacity. Our model can recognize both seen and unseen objects in images, enhancing flexibility and compatibility for various Zillow applications.

Large-scale Indoor Dataset Collection :  Designed and created a large-scale indoor semantic segmentation dataset. Developed an advanced annotation tool that integrates foundational vision models (e.g., Segment Anything). This tool significantly reduces mask annotation workload, improving annotation efficiency and accuracy.

Research Works : As a intern supervisor, recruited and supervised 2 research interns on their projects. 

• Developed a large-scale indoor description dataset via GPT4 and computer vision models with human-in-the-loop supervision. Designed and trained a baseline model based on the dataset. This pioneering work enables Zillow's model to achieve home-level understanding and evaluation capacity. 

• Introduced an enhanced open-set object detection model. It balances task-specific detection performance while maintaining open-set capacity for handling unexpected input. This model enhances the robustness of Zillow’s product in real-world applications.

Applied Scientist, Department of AI, Rich Media Experience

Home Feature Extraction : Developed AI models which explores 2D and 3D home data (e.g., Zillow Indoor Dataset) in both visual and language domains to extract additional home features and insights. The learned feature improves the performances for several down-stream Zillow applications including classification, retrieval, and recommendations.

Research Works : As a intern supervisor, recruited and supervised 1 research intern. We proposed a domain adaptation-based computer vision model for the Home Layout Estimation task. This project enhances Zillow’s capacity to obtain home layout information more precisely and robustly. [PDF]

Northeastern University, Boston, MA. 09/2016 - 04/2021

Research Assistant, Department of Electrical & Computer Engineering

Multi-modal Learning : (1) Led a team in collecting a large-scale multi-modal (RGB-D, EMG, Skeleton) action dataset. Aligned, organized, and pre-processed the dataset for future research works; (2) Proposed various multi-modal methods that fully explore latent correlations across different modalities; (3) Developed generative strategies to address challenges of multi-modal scenarios such as modality missing and corruption.

Transfer Learning & Domain Adaptation : (1) Explored new training strategies that adapt large models to fit specific tasks with limited data, either in a supervised or unsupervised manner; (2) Various modules are designed for different data types (e.g., images, depth, 3D point cloud, multi-modal) and different settings (e.g., co-training, self-supervised, generative, adversarial).

Multi-label Learning : Proposed methods which predict multiple labels from a single instance. Modules are designed for tackling challenges such as complex label correlations and long-tail label distributions. Models are evaluated in various applications such as image classification, annotation, and retrieval.

Teaching Assistant,  College of Engineering

Computer Vision (EECE 5639) : Introduced conventional and advanced computer vision algorithms including image processing, 3D reconstruction, deep learning, classification, detection, segmentation, etc.

Unsupervised Machine Learning (DS 5230) :  Introduced various traditional and SOTA unsupervised learning strategies such as clustering, dimension reduction, auto-encoder, deep learning-based, self-supervised learning, etc.

Data Visualization (EECE 5642) : Introduced diverse visualization strategies in various scenarios, including presentations, reports, and research papers. Tools such as MATLAB and Tableau are introduced in assignments.

Samsung Research America, Mountain View, CA. 06/2020 - 09/2020

Research Intern, Group of Artificial Intelligence

Multi-modal Saliency Detection: Explored a novel framework for multi-modal (RGB-D) saliency detection, which effectively identifies significant objects in an image. A Knowledge-Distillation strategy is implemented  to considerably reduce the network's complexity and enhance its inference efficiency, even on mobile platforms.

NEC Labs of America, Princeton, NY. 06/2019 - 12/2019

Research Assistant Intern, Department of Data Science & System Security

Reinforced Sentiment Classification : Proposed a reinforcement learning-based NLP model which predicts sentimental polarities of a given text. It disregards task-irrelevant text and instead prioritizes identifying the most effective clues. It considerably reduces the computational resource requirements. [Research Paper] [Patent]

Graph Data Learning : Developed a novel mechanism for learning graph data representations. Graph structured data retains valuable connectivity information among instances (e.g., social networks and advertising). The model allows for inductive and unsupervised learning in a highly efficient and effective manner. [Research Paper] [Patent]

Zebra Technologies, Lincolnshire, IL. 06/2017 - 08/2017 and 06/2018 - 08/2018

Computer Vision Algorithm Engineer Intern, Chief Technology Office, Computer Vision Algorithm

Robust 3D Localization & Detection : Developed computer vision system with the capability to capture 3D containers, classify different container types, and accurately measure their dimensions/locations. The system is able to perform high-precision localization in high-level 3D sensor noise with low computational cost (e.g., embedded platform). [Patent-1] [Patent-2]

Vision-based Human & Pose Detection : Deployed human/face detection and pose estimation algorithms in a warehouse environment. These algorithms effectively tackle challenges such as low illumination, occlusion, and various interruptions.