Research & Tech Transfer (selected)
My research has been transferred into numerous products with millions of users
Super Quality AI Portrait (2019)
Lenovo S5 Pro: AI 3D Face Reshaping (2018)
We shipped this brand-new face-transforming feature to Lenovo mobile phones announced on March 20, 2018. We trained a network that analyzes a huge collection of "beautiful faces" and generates a 3D face model that transforms the input image. We made a lot of optimization so that the algorithm runs at 60 fps on a mobile phone!
Face-aware Wide-angle Lens Correction (2017)
We developed a technique that corrects the distortion in wide-angle lens while preserving the faces. Note the faces near the image edges. The algorithm is super efficient and runs in real-time.
Adobe After Effects: Camera Shake Deblur (2017)
The new video filter, Camera-Shake Deblur, reduces motion blur artifacts resulting from camera shake. This effect can save shots that might not otherwise be usable due to slight camera bumps or camera motion blur that remains after stabilization.
Adobe Dimension CC: Auto IBL (2017)
This is a key feature in Dimension, a 3D compositing software that makes 3D easy for 2D graphics designers. It completes background images to 360-degree panoramas and approximates lighting and tone-mapping simultaneously.
Lightroom & Photoshop CC: Boundary Warp (2016)
Premiere Elements 15: Fast Video Dehaze (2016)
Front page feature! This new Haze Removal enables you to easily remove haze to make the background of your scenes as crisp as what’s up front. We have done tremendous optimization to the algorithm. The filter processes 1080p videos in real-time!
After Effects CC: Face Tracking (2015)
After Effects CC: Key Cleaner (2014)
The Key Cleaner effect recovers detail from a scene keyed by a typical keying effect, including recovering detail from a keyed scene with compression artifacts.
The Advanced Spill Suppressor effect removes color spill on a foreground subject from a colored background used for color keying.
AE & Premiere Pro: Detail-preserving Upscale (2013)
The Detail-preserving Upscale effect is capable of scaling up images by large amounts while preserving details in the image. The sharpness of sharp lines and curves is preserved. For example, you can scale up from SD frame sizes to HD frame sizes, or from HD frame sizes to digital cinema frame sizes.
After Effects CS6: Refine Edge Tool (2012)
Internal code name: "Hair Brush". It is a video matting tool that keeps the details when separating complicated foreground elements like frizzy hair or motion-blurred edges from complex backgrounds. This means you can create natural-looking composites without keying specially shot footage. When combined with Roto Brush we developed earlier, they give you a powerful, brand new workflow for rotoscoping.
After Effects CS5: Roto Brush (2010)
The all-new Roto Brush tool in After Effects CS5 is largely based on our research on Video SnapCut. This is the KILLER feature of this AE version. it provides an alternative, faster workflow for this segmentation and creation of a matte, saving professional roto artist hours of work on a single footage.
Other Projects/Tech Transfers
Camera, Low-level vision
- Multi-frame Alignment, Denoising, HDR, Super-res
- Video Denoising, HDR
- Night Scene RAW Denoising
- Multi-lens Fusion, Lens Distortion Removal
- Single Camera Depth Estimation using Focus Stacking
- 2PD Depth Estimation
- Motion Capture and Transfer to Cartoons
- Animating Still Images
- Object Tracker for Character Animator