Most of my work (both in industry and academia) was automotive/mobile device related. I have knowledge and hands-on experiences on vehicle/human localization, simulation, machine learning, computer vision, platform/SDK development (ADAS/AD, mobile), data analytics, wireless/sensor network and system automation/profiling.

Here below are some of the projects I worked on.

  • AD/ADAS Platform

      • Sensor Logging: Played a key role in the design/development of sensor data logging systems, which could log all the data from cameras, lidars, radars, IMU, GPS, at a rate of 20+ Gbps without loss.

      • Lidar Simulation: Designed & developed end-to-end pipeline for realistic Lidar simulation, including creating static background meshes, processing and organizing dynamic actor assets, implementing the pipeline for constructing simulation scene, ray-casting and organizing simulation output for evaluating autonomy modules. Developed DL models for processing Lidar data, including background segmentation model for filtering dynamic objects in point cloud and ray-dropping model for producing realistic simulation.

      • Camera Visualization: Designed and developed modules for processing raw camera data and visualization.

* Techniques: C/C++, Deep learning, Qt, Python, Shell, DPDK, Linux, Image Processing, Git/Gerrit, Jira, AWS, Spark


  • Driver Assistance Systems

      • V2V Safety Applications: Designed and oversaw the development of SAE V2V use cases (EEBL, FCW, BSW, IMA, DPNW, LTA).

      • Surrounding Vehicle Monitoring: A computer vision and wireless based collaborative system allows the participants to exchange what they see with each other and helps drivers learn about the positions of their surrounding vehicles.

      • Driver Communication: A computer vision and wireless system allows neighboring drivers to directly talk to convey driving related messages to each other.

      • Blind Spot Monitoring: A computer vision-based system for monitoring the blind spot of a vehicle. Schemes based on intensity variation, contour matching and optical flow are developed to analyze the region of interest for detecting the presence of vehicles in blind spot.

* Techniques: Android, Sensors (Accelerometer, Gyroscope, Magnetometer, GPS, Camera), Java, C++, Matlab, ROS, Sensor Fusion (Kalman Filter, Complementary Filter), Vehicle Detection/Tracking (Haar Cascade classifier, Chamfer matching, Optical flow, Intensity analysis), Image Processing (OpenCV), NDK, Vehicular Network Simulation (SUMO, NS2), DSRC


  • Driving Analytics

      • Driving Behavior Analysis: A motion-sensor and GPS based model for analyzing the driving behavior and identifying the aggressive patterns, such as hard brake, sharp turn, harsh acceleration.

      • In-Vehicle Phone Usage: A motion-sensor based model for classifying the positions of a phone inside a vehicle and identifying how a driver is using his phone during driving.

      • Driver Attention Detection: A vision based model for analyzing the attention of drivers. It detects and analyzes the eye blinking and head pose to detect the fatigue/drowsiness of the driver; identify whether the driver is looking ahead/focusing on the road.

      • Vehicle Speed Estimation: A motion-sensor based system (with map knowledge) for estimating the vehicle speed in an overwhelming majority of road/traffic situations.

* Techniques: Android, Sensors (Accelerometer, Gyroscope, Magnetometer, GPS, Camera), Java, C++, Matlab, PHP, MySQL, Sensor Fusion (Kalman Filter, Complementary Filter), Image Processing (OpenCV), NDK, Signal Processing/Feature Engineering, Pattern Recognition, Machine Learning (SVM, kNN, Decision Tree, Random Forest, Neural Network, etc.)


  • Localization

      • Vehicle Localization: A sensor based (IMU, barometer, WiFi, GPS, Cellular network) system for vehicle localization. The system first created a map for sensor-based landmarks corresponding to the road characteristics (e.g. slope, turn, stop sign, rugged road surface, bump, pothole) and wireless environment (e.g. WiFi, Cellular signal), and then did vehicle localization through sensor-fusion based dead reckoning, and continuously corrected the errors accumulated from dead reckoning by referring to the landmarks.

      • Robot Navigation: A robot navigation solution based on range sensor and occupancy grid map. The robot is equipped with a range sensor (LiDAR, which ranges the obstacles in a space with point cloud), IMU and odometer. Particle Filter is used to estimate the optimal pose of the robot by correlating the depth measurements and the occupancy grid map.

      • Ticketless System for Passenger: A ticketless system based on the motion sensors and barometer of smart devices for identifying the details (e.g. the route/time/stop) of passenger bus trip and also enabling traffic analytics. It realizes the function by correlating the sensor traces recorded by passengers’ smart devices with the reference traces recorded by the devices in buses.

      • Semantic Localization: Develop a semantic localization solution, which allows customers’ smartphones to recognize the semantic names of the stores they are in or nearby through WiFi scanning. The system bridges the gap between the WiFi AP vectors and store names by correlating the text seen in the stores with the text shown on their websites. The solution also contains an audio based classifier to identify the categories of the stores, which improves the localization based on text and WiFi. It goes through information retrieval (extracting in-store text from crowdsourced in-store images with OCR; retrieving web text through search engine and web analysis), language processing (extracting noun/proper name, weight assignment and text matching), acoustic processing and classification.

      • Athlete Tracking System: Designed/Developed a sprinter tracking system with motion sensors and wireless sensor network, including drivers, protocols and algorithms. The system tracks the positions, detailed motion (acceleration, rotation, etc.) of athletes and provides an insight of the athletes’ performance through data analysis.

* Techniques: Android, Sensors (Accelerometer, Gyroscope, Magnetometer, Barometer, GPS, Camera, WiFi, Cellular network), Java, C++, C, Matlab, PHP, MySQL, Sensor Fusion (Kalman Filter, Complementary Filter), Localization Algorithms (Particle Filter, Kalman Filter, Iterative Closest Point), OCR, Sensor and Acoustic Signal Processing/Feature Engineering, Machine Learning (SVM, kNN, K-Means, Decision Tree, Random Forest, Neural Network, PCA, HMM, DTW, etc.), Language Processing, Wireless Sensor Network


  • Mobile SDK (MOAP(L))

      • Application Framework/Middleware

        • Develop GUI framework with customized widgets to support application development

        • Customize open-sourced font system for mobile platform

        • Implement SIP protocol stack (SDP parsing, negotiation of media capability) and audio/video transmission protocol stack of media framework for an IP Multimedia Subsystem (IMS)

      • Driver

        • Develop linux software driver for the communication between mobile’s dual CPUs

      • API Evaluation System

        • Design/Develop an API evaluation system, which provides a user-friendly IDE for SDK developers/testers to create/run/manage test cases/applications easily and efficiently.

      • Optimization/Maintenance

        • Troubleshoot & profile the platform and optimize the system

        • Maintain SDK document

* Techniques: Linux (Red Hat, MontaVista), C, Tcl/Tk, Gtk, SIP/SDP, RTP/RTCP, Profiling