Hailin Jin, Ph.D.
Senior Principal Scientist
Adobe Research
Adobe Research site: https://research.adobe.com/person/hailin-jin/
Google Scholar: https://scholar.google.com/citations?user=DQOT0OMAAAAJ
LinkedIn: https://www.linkedin.com/in/hailin-jin-61613a8/
Email: hljin@adobe.com
Internship opening: I am looking for motivated students to work at Adobe Research on Computer Vision, Deep Learning, and related projects.
I am a Senior Principal Scientist at Adobe Research. I received my Master of Science and Doctor of Science degrees in Electrical Engineering from Washington University in Saint Louis in 2000 and 2003, respectively. My advisor was Professor Stefano Soatto. Between fall 2003 and fall 2004, I was a postdoctoral researcher at the Computer Science Department, University of California at Los Angeles. My current research interests include: deep learning, natural language processing, computer vision, video, image search, 3D reconstruction, structure and motion estimation, optical flow, stereo, and image-based modeling and rendering. My work can be found in many Adobe products including Photoshop, After Effects, Premiere Pro, and Photoshop Lightroom.
Publications
2020
Video Question Answering on Screencast Tutorials.
Wentian Zhao, Seokhwan Kim, Ning, Xu, and Hailin Jin
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, January 2021.
Visual Font Pairing.
Shuhui Jiang, Zhaowen Wang, Aaron Hertzmann, Hailin Jin, and Yun Fu.
IEEE Transactions on Multimedia, Pages 2086-2097, Volume 22, Issue 8, August 2020.
https://doi.org/10.1109/TMM.2019.2952266
2019
An Internal Learning Approach to Video Inpainting.
Haotian Zhang, Long Mai, Hailin Jin, Zhaowen Wang, Ning Xu, and John Collomosse.
Proceedings of International Conference on Computer Vision (ICCV), October 2019.
Large-scale Tag-based Font Retrieval with Generative Feature Learning.
Tianlang Chen, Zhaowen Wang, Ning Xu, Hailin Jin, and Jiebo Luo.
Proceedings of International Conference on Computer Vision (ICCV), October 2019.
Learning Video Representations from Correspondence Proposals.
Xingyu Liu, Joon-Young Lee, and Hailin Jin.
Proceedings of IEEE/CVF Conference on Computer Vision and Patter Recognition (CVPR), June 2019.
LiveSketch: Query Perturbations for Guided Sketch-based Visual Search.
John Collomosse, Tu Bui, and Hailin Jin.
Proceedings of IEEE/CVF Conference on Computer Vision and Patter Recognition (CVPR), June 2019.
Learning from Multi-domain Artistic Images for Arbitrary Style Transfer.
Zheng Xu, Michael Wilber, Chen Fang, Aaron Hertzmann, and Hailin Jin.
The 8th ACM/EG Expressive Symposium (Expressive), May 2019.
2018
Learning from Photoshop Operation Videos: the PSOV Dataset.
Jingchun Cheng, Han-Kai Hsu, Chen Fang, Hailin Jin, Shengjin Wang, and Ming-Hsuan Yang.
Proceedings of the Asian Conference on Computer Vision (ACCV), December 2018.
Characterizing User Skills from Application Usage Traces with Hierarchical Attention Recurrent Networks.
Longqi Yang, Chen Fang, Hailin Jin, Matthew D. Hoffman, and Deborah Estrin.
ACM Transactions on Intelligent Systems and Technology (TIST), 9(6), October 2018.
https://dl.acm.org/citation.cfm?id=3232231
Product Quantization Network for Fast Image Retrieval.
Tan Yu, Junsong Yuan, Chen Fang, and Hailin Jin.
Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
Synthetically Supervised Feature Learning for Scene Text Recognition.
Yang Liu, Zhaowen Wang, Hailin Jin, and Ian Wassell.
Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
“Factual” or “Emotional”: Stylized Image Captioning with Adaptive Learning and Attention.
Tianlang Chen, Zhongping Zhang, Quanzeng You, Chen Fang, Zhaowen Wang, Hailin Jin, and Jiebo Luo.
Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
What do I Annotate Next? An Empirical Study of Active Learning for Action Localization.
Fabian Caba, Joon-Young Lee, Hailin Jin, and Bernard Ghanem.
Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
http://openaccess.thecvf.com/content_ECCV_2018/papers/Fabian_Caba_What_do_I_ECCV_2018_paper.pdf
Interactive Boundary Prediction for Object Selection.
Hoang Le, Long Mai, Brian Price, Adobe; Scott Cohen, Hailin Jin, and Feng Liu.
Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
Speeding up Context-based Sentence Representation Learning with Non-autoregressive Convolutional Decoding.
Shuai Tang, Hailin Jin, Chen Fang, Zhaowen Wang, and Virginia de Sa.
Proceedings of the 3rd Workshop on Representation Learning for NLP (RepL4NLP), July 2018.
http://aclweb.org/anthology/W18-3009
Disentangling Structure and Aesthetics for Style-Aware Image Completion.
Andrew Gilbert, John Collomosse, Hailin Jin, and Brian Price.
IEEE Conference on Computer Vision and Patter Recognition (CVPR), June 2018.
Multi-Task Adversarial Network for Disentangled Feature Learning.
Yang Liu, Zhaowen Wang, Hailin Jin, and Ian Wassell.
IEEE Conference on Computer Vision and Patter Recognition (CVPR), June 2018.
Deep Learning for Font Recognition and Retrieval.
Zhangyang Wang, Jianchao Yang, Hailin Jin, Zhaowen Wang, Eli Shechtman, Aseem Agarwala, Jonathan Brandt, and Thomas S. Huang.
Applied Cloud Deep Semantic Recognition: Advanced Anomaly Detection, Pages 109-130, March 2018.
https://www.taylorfrancis.com/books/e/9781351119016/chapters/10.1201%2F9781351119023-6
2017
M. Wilber, Chen Fang, Hailin Jin, Aaron Hertzmann, John Collomosse, and S. Belongie.
BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography.
In Proc. IEEE International Conference Computer Vision, October 2017.
John Collomosse, Tui Bui, Michael Wilber, Chen Fang, and Hailin Jin.
Sketching with Style: Visual Search with Sketches and Aesthetic Context.
In Proc. IEEE International Conference Computer Vision, October 2017.
Shuai Tang, Hailin Jin, Chen Fang, Zhaowen Wang, and Virginia de Sa.
Rethinking Skip-thought: A Neighborhood based Approach.
In Proc. 2nd Workshop on Representation Learning for NLP, August 2017.
Zhou Ren, Hailin Jin, Zhe Lin, Chen Fang, and Alan Yuille.
Multiple Instance Visual-Semantic Embedding.
In Proc. British Machine Vision Conference, September 2017.
Yinda Zhang, Shuran Song, Ersin Yumer, Manolis Savva, Joon-Young Lee, Hailin Jin, Thomas Funkhouser.
Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks.
In Proc. IEEE Computer Vision and Pattern Recognition, July 2017
Long Mai, Hailin Jin, Zhe Lin, Chen Fang, Jonathan Brandt, and Feng Liu.
Spatial-Semantic Image Search by Visual Feature Synthesis.
In Proc. IEEE Computer Vision and Pattern Recognition, July 2017.
Longqi Yang, Chen Fang, Hailin Jin, Matthew Hoffman, and Deborah Estrin.
Personalizing Software and Web Services by Integrating Unstructured Application Usage Traces.
In Proc. International Conference on World Wide Web (WWW), April 2017. https://dl.acm.org/citation.cfm?doid=3041021.3054183
Jingwei Huang, Zhili Chen, Duygu Ceylan, and Hailin Jin.
6-DOF VR Videos with a Single 360-Camera.
In Proc. IEEE Virtual Reality, March 2017.
https://ieeexplore.ieee.org/document/7892229
2016
Quanzeng You, Hailin Jin, and Jiebo Luo.
Visual Sentiment Analysis by Attending on Local Image Regions.
In Proc. Thirty-First AAAI Conference on Artificial Intelligence, February 2017.
Zhuoyuan Chen, Hailin Jin, Zhe Lin, Scott Cohen, and Ying Wu.
Integrated Variational And Nearest Neighbor field (IVANN) for Optical Flow.
In Proc. ECCV Workshop on Brave new ideas for motion representations in videos, October 2016.
Zhou Ren, Hailin Jin, Zhe Lin, Chen Fang, and Alan Yuille.
Joint Image-Text Representation by Gaussian Visual Semantic Embedding.
In Proc. ACM Multimedia Conference, 2016
Quanzeng You, Liangliang Cao, Hailin Jin, and Jiebo Luo.
Robust Visual-Textual Sentiment Analysis: When Attention meets Tree-structured Recursive Neural Networks.
In Proc. ACM Multimedia Conference, October 2016. https://dl.acm.org/citation.cfm?id=2964284.2964288
Long Mai, Hailin Jin, and Feng Liu.
Composition-Preserving Deep Photo Aesthetics Assessment.
In Proc. IEEE Computer Vision and Pattern Recognition, June 2016.
Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo.
Image Captioning With Semantic Attention.
In Proc. IEEE Computer Vision and Pattern Recognition, June 2016.
Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang.
Cross-modality Consistent Regression for Joint Visual-Textual Sentiment Analysis of Social Multimedia.
In Proc. ACM International Conference on Web Search and Data Mining, February 2016.
Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang.
Building a Large Scale Dataset for Image Emotion Recognition: the Fine Print and the Benchmark.
In Proc. Thirtieth AAAI Conference on Artificial Intelligence, February 2016.
2015
Xin Lu, Zhe Lin, Hailin Jin, and Jianchao Yang.
Image-Specific Prior Adaptation for Denoising.
IEEE Transactions on Image Processing, 24(12): 5469-5478, December 2015.
Xin Lu, Zhe Lin, Hailin Jin, Jianchao Yang, and James Z. Wang.
Rating Image Aesthetics Using Deep Learning.
IEEE Transactions on Multimedia, 17(11): 2021-2034, November 2015.
Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang.
Joint Visual-Textual Sentiment Analysis with Deep Neural Networks.
In Proc. ACM Multimedia Conference, October 2015.
Zhangyang Wang, Jianchao Yang, Hailin Jin, Eli Shechtman, Aseem Agarwala, Jonathan Brandt, and Thomas S. Huang.
DeepFont: Identify Your Font from An Image.
In Proc. ACM Multimedia Conference, October 2015. https://dl.acm.org/citation.cfm?doid=2733373.2806219
Jonathan Krause, Hailin Jin, Jianchao Yang, and Li Fei-Fei.
Fine-Grained Recognition without Part Annotations.
In Proc IEEE Conference on Computer Vision and Pattern Recognition, June 2015.
https://ieeexplore.ieee.org/document/7299194
Chen Fang, Hailin Jin, Jianchao Yang, and Zhe Lin.
Collaborative Feature Learning from Social Media.
In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2015.
Zhangyang Wang, Jianchao Yang, Hailin Jin, Eli Shechtman, Aseem Agarwala, Jonathan Brandt, and Thomas S. Huang.
Real-World Font Recognition Using Deep Network and Domain Adaptation.
In Proc. International Conference on Learning Representations, May 2015.
Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang.
Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks.
In Proc. Twenty-Ninth AAAI Conference on Artificial Intelligence, January 2015.
Xin Lu, Zhe Lin and Hailin Jin.
Tree-based Locally Linear Regression for Image Denoising.
In Proc. IEEE Winter Conference on Applications of Computer Vision, January 2015.
Guang Chen, Jianchao Yang, Hailin Jin, Eli Shechtman, Jonathan Brandt, and Tony X. Han.
Selective Pooling Vector for Fine-grained Recognition.
In Proc. IEEE Winter Conference on Applications of Computer Vision, January 2015.
https://ieeexplore.ieee.org/document/7045973
2014
Linchao Bao, Qingxiong Yang, and Hailin Jin.
Fast Edge-Preserving PatchMatch for Large Displacement Optical Flow.
IEEE Transactions on Image Processing, 23(12): 4996-5006, December 2014.
Linchao Bao, Qingxiong Yang, and Hailin Jin.
Fast Edge-Preserving PatchMatch for Large Displacement Optical Flow.
In Proc IEEE Conf. Computer Vision and Pattern Recognition, June 2014.
https://ieeexplore.ieee.org/document/6905797
Guang Chen, Jianchao Yang, Hailin Jin, Jonathan Brandt, Eli Shechtman, Aseem Agarwala, and Tony X. Han.
Large-scale visual font recognition.
In Proc IEEE Conf. Computer Vision and Pattern Recognition, June 2014.
Kevin Karsch, Kalyan Sunkavalli, Sunil Hadap, Nathan Carr, Hailin Jin, Rafael Fonte, Michael Sittig, and David Forsyth.
Automatic Scene Inference for 3D Object Compositing.
ACM Transactions on Graphics, 33(3), May 2014.
https://dl.acm.org/citation.cfm?id=2602146
Yingzhen Yang, Xinqi Chu, Tian Tsong Ng, Alex Yong-Sang Chia, Jianchao Yang, Hailin Jin, and Thomas S. Huang.
Epitomic Image Colorization.
In Proc IEEE Intl. Conf. on Acoustics, Speech and Signal Processing, May 2014.
https://ieeexplore.ieee.org/document/6854044
Thomas Paine, Hailin Jin, Zhe Lin, Jianchao Yang and Thomas S. Huang.
GPU Asynchronous Stochastic Gradient Descent to Speed up Neural Network Training.
In Proc International Conference on Learning Representations (ICLR), April 2014.
2013
Feng Liu, Yuzhen Niu, and Hailin Jin.
Joint Subspace Stabilization for Stereoscopic Video.
In Proc IEEE Intl. Conf. Computer Vision (ICCV), December 2013.
Hyeongwoo Kim, Hailin Jin, Sunil Hadap, and In So Kweon.
Specular Reflection Separation using Dark Channel Priors.
In Proc IEEE Conf. Computer Vision and Pattern Recognition (CVPR), June 2013.
Zihan Zhou, Hailin Jin, and Yi Ma.
Plane-Based Content-Preserving Warps for Video Stabilization.
In Proc IEEE Conf. Computer Vision and Pattern Recognition (CVPR), June 2013.
Zhuoyuan Chen, Hailin Jin, Zhe Lin, Scott Cohen, and Ying Wu.
Large Displacement Optical Flow from Nearest Neighbor Fields.
In Proc IEEE Conf. Computer Vision and Pattern Recognition (CVPR), June 2013.
Feng Liu, Yuzhen Niu, and Hailin Jin.
Causal Stereoscopic Photo Authoring.
IEEE Transactions on Multimedia. 15(1): 129-140, January 2013. https://ieeexplore.ieee.org/document/6331540
2012
Alper Ayvaci, Hailin Jin, Zhe Lin, Scott Cohen, and Stefano Soatto.
Video Upscaling with Spatio-Temporal Self-Similar Examples.
In Proc. International Conference on Pattern Recognition, November 2012.
Shengqi Zhu, Li Zhang and Hailin Jin.
A Locally Linear Regression Model for Boundary Preserving Regularization in Stereo Matching.
In Proc. European Conference on Computer Vision, October 2012.
James Andrews, Hailin Jin, and Carlo Sequin.
Interactive Inverse 3D Modeling.
Computer-Aided Design and Applications, 9(6), 2012.
[Best Student Paper Award]
Yuzhen Niu, Feng Liu, Wu-Chi Feng and Hailin Jin.
Aesthetics-Based Stereoscopic Photo Cropping for Heterogeneous Displays.
IEEE Transactions on Multimedia, 14(3): 783-796, June 2012. https://ieeexplore.ieee.org/document/6144744
Feng Liu, Yuzhen Niu, and Hailin Jin.
Keystone Correction for Stereoscopic Cinematography.
In Proc. International Workshop on 3-D Cinematography, June 2012.
Zihan Zhou, Hailin Jin and Yi Ma.
Robust Plane-based Structure from Motion.
In Proc IEEE Conf. Computer Vision and Pattern Recognition, June 2012.
2011
Feng Liu, Michael Gleicher, Jue Wang, Hailin Jin, and Aseem Agarwala.
Subspace Video Stabilization.
In ACM Transactions on Graphics, Vol. 30, Issue 1, January 2011.
2009
Kai Ni, Hailin Jin, and Frank Dellaert.
GroupSAC: Efficient Consensus in the Presence of Groupings.
In Proc. IEEE Intl. Conf. Computer Vision, September 2009.
Brandon M. Smith, Li Zhang, Hailin Jin, and Aseem Agarwala.
Light-Field Video Stabilization.
In Proc. IEEE Intl. Conf. Computer Vision, September 2009.
Feng Liu, Michael Gleicher, Hailin Jin, and Aseem Agarwala.
Content-Preserving Warps for 3D Video Stabilization.
In ACM Transactions on Graphics (Proceedings of SIGGRAPH 2009), August 2009.
Brandon M. Smith, Li Zhang and Hailin Jin.
Stereo Matching with Nonparametric Smoothness Priors in Feature Space.
In Proc. IEEE Intl. Conf. on Computer Vision and Pattern Recognition, June 2009.
Li Zhang, Sundeep Vaddadi, Hailin Jin, and Shree. K. Nayar.
Multiple View Image Denoising.
In Proc. IEEE Intl. Conf. on Computer Vision and Pattern Recognition, June 2009.
2008
Liang Wang, Hailin Jin and Ruigang Yang.
Search Space Reduction for MRF Stereo.
In Proc. European Conf. on Computer Vision, October 2008.
Hailin Jin.
A Three-Point Minimal Solution for Panoramic Stitching with Lens Distortion.
In Proc. IEEE Intl. Conf. On Computer Vision and Pattern Recognition, June 2008.
Liang Wang, Hailin Jin, Ruigang Wang and Minglun Gong.
Stereoscopic Inpainting: Joint Color and Depth Completion from Stereo Images.
In Proc. IEEE Intl. Conf. On Computer Vision and Pattern Recognition, June 2008.
Hailin Jin, Daniel Cremers, Dejun Wang, Emmanuel Prados, Anthony Yezzi, and Stefano Soatto.
3-D Reconstruction of Shaded Objects from Multiple Images Under Unknown Illumination.
International Journal on Computer Vision, 76(3): 245-256, March 2008.
https://link.springer.com/article/10.1007/s11263-007-0055-y
2007
Hailin Jin, Anthony Yezzi, and Stefano Soatto.
Mumford-Shah on the Move: Region-based Segmentation on Deforming Manifolds with Application to 3-D Reconstruction of Shape and Appearance from Multi-view Images.
Journal of Mathematical Imaging and Vision, 29(2-3): 219-234, November 2007. https://link.springer.com/article/10.1007/s10851-007-0030-9
2005
Andrea Vedaldi, Hailin Jin, Paolo Favaro and Stefano Soatto.
KALMANSAC: Causal Inference of Dynamic Processes Drowned in Outliers.
In Proceedings of the 10th IEEE International Conference on Computer Vision, volume 1, pages 633-640, October 2005.
Hailin Jin, Stefano Soatto, and Anthony Yezzi.
Multi-view Stereo Reconstruction of Dense Shape and Complex Appearance.
International Journal on Computer Vision, 63(3): 175-189, July 2005.
Hailin Jin and Paolo Favaro.
Visual Tracking in the Presence of Motion Blur.
In Proceedings of the IEEE Computer Society International Conference on Computer Vision and Pattern Recognition, volume 2, pages 18–25, June 2005.
2004
Hailin Jin, Daniel Cremers, Anthony Yezzi, and Stefano Soatto.
Shedding Light on Stereoscopic Segmentation.
In Proceedings of the IEEE Computer Society International Conference on Computer Vision and Pattern Recognition, volume 1, pages 36–42, June 2004.
Hailin Jin, Anthony Yezzi and Stefano Soatto.
Region-based Segmentation on Evolving Surfaces with Application to 3D Reconstruction of Shape and Piecewise Constant Radiance.
In Proceedings of the Eighth European Conference on Computer Vision, volume 2, pages 114–125, May 2004.
2003
Hailin Jin, Anthony Yezzi, Yen-Hsi Tsai, Li-Tien Cheng and Stefano Soatto.
Estimation of 3D Surface Shape and Smooth Radiance from 2D Images: a level set approach.
Journal of Scientific Computing, 19(1-3): 267–292, December 2003.
Hailin Jin, Paolo Favaro, and Stefano Soatto.
A Semi-direct Approach to Structure From Motion.
The Visual Computer, 19(6): 377–394, October 2003.
Stefano Soatto, Anthony Yezzi, and Hailin Jin.
Tales of Shape and Radiance in Multi-view Stereo.
In Proceedings of the 9th IEEE International Conference on Computer Vision, volume 1, pages 947–981, October 2003.
Hailin Jin.
Variational Methods for Shape Reconstruction in Computer Vision.
Ph.D. thesis, Electrical Engineering, Washington University, August 2003.
Hailin Jin, Stefano Soatto, and Anthony Yezzi.
Multi-view Stereo Beyond Lambert.
In Proceedings of the IEEE Computer Society International Conference on Computer Vision and Pattern Recognition, volume 1, pages 171–178, June 2003.
Anthony Yezzi, Stefano Soatto, Hailin Jin, Andy Tsai, and Alan Willsky.
The Mumford-Shah Functional from Segmentation to Stereo.
Geometric Level Set Methods in Imaging, Vision and Graphics. Springer Verlag, 2003.
https://link.springer.com/chapter/10.1007/0-387-21810-6_12
2002
Hailin Jin, Anthony Yezzi and Stefano Soatto.
Multi-frame Stereo in the Presence of Specular Reflections.
In Proceedings of the First International. Symposium on 3D Data Processing Visualization and Transmission, pages 626–630, June 2002.
Hailin Jin and Paolo Favaro.
A Variational Approach to Shape from Defocus.
In Proceedings of the Seventh European Conference on Computer Vision, pages 18–30, May 2002.
Alessandro Chiuso, Paolo Favaro, Hailin Jin, and Stefano Soatto.
Structure From Motion Causally Integrated over Time.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(4): 523–535, April 2002.
2001
Paolo Favaro, Hailin Jin and Stefano Soatto.
A Semi-direct Approach to Structure from Motion.
In Proceedings of the International Conference on Image Analysis and Processing, pages 250–255, September 2001.
Hailin Jin, Paolo Favaro and Stefano Soatto.
Real-time Feature Tracking and Outlier Rejection with Changes in Illumination.
In Proceedings of the 8th IEEE International Conference on Computer Vision, pages 684–689, July 2001.
Paolo Favaro, Hailin Jin and Stefano Soatto.
Real-time Virtual Object Insertion.
In Proceedings of the 8th IEEE International Conference on Computer Vision, July 2001.
2000
Alessandro Chiuso, Paolo Favaro, Hailin Jin and Stefano Soatto.
3-D Motion and Structure Causally Integrated over Time: Implementation.
In Proceedings of the European Conference on Computer Vision, pages 778–779, June 2000.
Hailin Jin, Anthony Yezzi and Stefano Soatto.
Stereoscopic Shading: Integrating Shape Cues in a Variational Framework.
In Proceedings of the IEEE Computer Society International Conference on Computer Vision and Pattern Recognition, pages 169–176, June 2000
Hailin Jin, Paolo Favaro and Stefano Soatto.
Real-time 3-D Motion and Structure from Point Features: a front-end system for vision-based control and interaction.
In Proceedings of the IEEE Computer Society International Conference on Computer Vision and Pattern Recognition, June 2000.
Hailin Jin.
Dense Shape Reconstruction using Visual Methods.
Master thesis, Electrical Engineering, Washington University, May 2000.