Zhicheng Yan (严志程）
Senior Staff Research Scientist, Meta Reality Labs
Email: zhicheng.yan at live.com
Biography
Zhicheng is a Senior Staff Research Scientist at Meta Reality Labs. He has been building and delivering cutting-edge on-device solutions to power the perception stack for Meta MR/VR products. He also conducts foundational research on 3D generative AI for Metaverse.
Before, Zhicheng was a Senior Manager to support the Object and Scene Understanding (OSU) team. The team's mission is to develop a deep and personalized understanding of the objects and scene in the egocentric data for next-generation Meta AR products. In the early stage of his career at Facebook, he worked on large-scale image and video understanding platform. His research interests mainly include computer vision and machine learning.
Zhicheng received his Ph.D from Department of Computer Science, University of Illinois at Urbana-Champaign in 2016. His supervisor is Prof. Yizhou Yu. Before that, he was a master student in the College of Computer Science and Technology, Zhejiang University and a research assistant of State Key Lab of CAD&CG. His supervisor is Prof. Wei Chen. Zhicheng completed his Bachelor's degree at Zhejiang University majoring in Software Engineering in July, 2007. He is an Award Recipient of Chinese Government Award for Outstanding Self-Financed Students Abroad.
Updates
24' 2: [HIRING] Our team CoreAI at Meta Reality Labs has a small number of Research Scientist opening positions. We are hiring exceptional researchers to push our Mixed Reality experience to the next level! Check the JD (https://www.metacareers.com/jobs/3713104949009511/). Please email me if you are interested.
23' 09: EgoObjects Github repo is alive! We release the v1.0 dataset and API.
23' 07: 3 ICCV papers accepts, incl EgoObjects, VL-Part, and FreeSeg. More details will come soon.
23' 05: Can we build a segmentation model recognizing open vocabulary objects and parts? Our new work VL-Part proposes a new model for such tasks. Check it out on Github [link]
23' 03: [Meta AI blog] We introduce EgoObjects Pilot version consisting of over 9K videos and more than 14K unique object instances where each is captured in multiple settings.
Industrial Experience
2016.6 ~ Present, Senior Staff Research Scientist
Meta Reality Labs, Menlo Park, CA
2015.5 ~ 2015.8, Research intern
Google Brain, Mountain View, CA
2014.5 ~ 2014.8, Research Intern
eBay Research, San Jose, CA
2013.6 ~ 2013.8, Research intern
IBM Watson Research Center, Yorktown Heights, NY
2012.5 ~ 2012.8, Research intern
Google Research, NYC
2009.6 ~ 2010.6, 3D graphics intern
Bosch Research Center, Palo Alto, CA.
Talks
Invited talk, "AI for Augmented Reality", CS 190I Deep Learning, University of California San Diego, 2023 March
Invited talk, "Decoupling Representation and Classifier for Long-Tailed Recognition", Imperfect Data (LID) Workshop, CVPR 2020.
Facebook Computer Vision AI Talk Series. "Computer Vision at Scale". UIUC, Champaign, 2018.
Oral presentation, "Automatic Photo Adjustment Using Deep Neural Networks". ACM SIGGRAPH, Anaheim, CA, 2016
Oral presentation, "HD-CNN: Hierarchical Deep Convolutional Neural Network for Large Scale Visual Recognition", UIUC Coordinated Science Lab (CSL) Student Conference, 2016
Invited talk, "Image Recognition,Semantic Segmentation and Photo Adjustment Using Deep Neural Networks", Nanjing University, China, 2015
Oral presentation, "Semantic Segmentation Using RNN Models", Google Brain, Mountain View, CA, 2015
Poster. Sparse Similarity Matrix Learning for Object Retrieval. International Joint Conference On Neural Networks, IJCNN 2013, Dallas, TX
Oral presentation, "Nonrigid 3D Object Retrieval Using Modal Space Transform", ICMR 2013. Dallas, TX
Oral presentation, "Volume Illustration of Muscles from Diffusion Tensor Images", IEEE Visualization 2009, Atlantic City, New Jersey, USA
Oral presentation, "Context-awareVolume Modeling of SkeletalMuscles", EuroVis 2009, Berlin, Germany
Professional Activities
Organizer
Challenge chair of "CLVISION CVPR Workshop" at CVPR '22.
"Workshop on Multi-modal Video Analysis and Moments in Time Challenge" at ICCV '19.
Conference Area Chair: BMVC '21-'23
Program committee: AAAI '20-'21, CVPR Holistic Video Understanding '21, ACM MM '17, CVM '17, LSCVS '17,
Conference Reviewer: CVPR '17-'22, ECCV '20, ICCV '19-'21, NeurIPS '20-'23, ICLR '22, ICML '21-'22, SIGGRAPH '16, '17, '20, SIGGRAPH Asia '15, Eurographics '16, ACCV '20, WACV '20-'22
Journal Reviewer: TPAMI, TVCG, TIP, IJCV, TNNLS, TCSVT, GRSL, MMSJ, IEEE Access
I enjoy reviewing papers. If you need my help, please send me an email.
Mentopships
Interns I worked with: Liunian Li (UCLA), Jun Chen (KAUST), Peize Sun (HKU), Kyungmin Kim (UC Irvine), Chengyue Gong (UT Austin), Yifan Jiang (UT Austin), Xinyu Gong (UT Austin), Fan Ma (UTS), Linchao Zhu (ZJU), Bingyi Kang (Sea AI), Yunpeng Chen (Meitu), Mike Z. Shou (NUS), Hang Zhao (Tsinghua)
Publications & Open Source Projects
2023
Exploring Open-Vocabulary Semantic Segmentation without Human Labels
[paper] [code]
Jun Chen, Deyao Zhu, Guocheng Qian, Bernard Ghanem, Zhicheng Yan, Chenchen Zhu, Fanyi Xiao, Mohamed Elhoseiny, Sean Culatana
ICCV 2023: International Conference on Computer Vision
2022
3rd Continual Learning Workshop Challenge on Egocentric Category and Instance Level Object Understanding
[paper]
Lorenzo Pellegrini, Chenchen Zhu, Fanyi Xiao, Zhicheng Yan, Antonio Carta, Matthias De Lange, Vincenzo Lomonaco, Roshan Sumbaly, Pau Rodriguez, David Vazquez
Arxiv, preprint
NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training
[paper] [code]
Chengyue Gong, Dilin Wang, Meng Li, Xinlei Chen, Zhicheng Yan, Yuandong Tian, Qiang Liu, Vikas Chandra
ICLR 2022, The International Conference on Learning Representations
Auto-X3D: Ultra-Efficient Video Understanding via Finer-Grained Neural Architecture Search
[paper] [code]
Yifan Jiang, Xinyu Gong, Junru Wu, Humphrey Shi, Zhicheng Yan, Zhangyang Wang
WACV 2022, Winter Conference on Applications of Computer Vision
2021
Searching for Two-Stream Models in Multivariate Space for Video Recognition
[paper] [code]
Xinyu Gong, Heng Wang, Mike Zheng Shou , Matt Feiszli, Zhangyang Wang, Zhicheng Yan
ICCV 2021: International Conference on Computer Vision (25.9% acceptance rate)
Visual Transformers: Token-based Image Representation and Processing for Computer Vision
[Paper] [code]
Bichen Wu, Chenfeng Xu, Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Zhicheng Yan, Masayoshi Tomizuka, Joseph Gonzalez, Kurt Keutzer, Peter Vajdai
ICCV 2021: International Conference on Computer Vision (25.9% acceptance rate)
FP-NAS: Fast Probabilistic Neural Architecture Search
[Paper] [Supplement] [Presentation] [code]
Zhicheng Yan, Xiaoliang Dai, Peizhao Zhang, Yuandong Tian, Bichen Wu, Matt Feiszli
CVPR 2021: IEEE International Conference on Computer Vision and Pattern Recognition (23.7% acceptance rate)
2020
2019
Classy Vision: An end-to-end framework for image and video classification
Adcock, A. , Reis, V. , Singh, M. , Yan, Z. , van der Maate, L., Zhang, K. , Motwani, S. , Guerin, J. , Goyal, N. , Misra, I. , Gustafson, L. , Changhan, C. , Goyal, P.
NeurIPS Expo 2019
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
Yunpeng Chen, Haoqi Fan, Bing Xu, Zhicheng Yan, Yannis Kalantidis, Marcus Rohrbach, Shuicheng Yan, Jiashi Feng
ICCV 2019: International Conference on Computer Vision (25% acceptance rate)
HACS: Human Action Clips and Segments Dataset for Recognition and Temporal Localization
[Paper] [Project] [Github] [ICCV 2019 Challenge Results , Submission Portal]
Hang Zhao, Antonio Torralba, Lorenzo Torresani, Zhicheng Yan
ICCV 2019: International Conference on Computer Vision (25% acceptance rate)
DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition
Zheng Shou, Xudong Lin, Yannis Kalantidis, Laura Sevilla-Lara, Marcus Rohrbach, Shih-Fu Chang, Zhicheng Yan
CVPR 2019: IEEE International Conference on Computer Vision and Pattern Recognition (25.2% acceptance rate)
Temporal Segment Convolutional Kernel Networks for Sequence Modeling of Videos
[Paper]
Fei Pan, Yanwen Guo, Zhicheng Yan, Jie Guo
ICME 2019: IEEE International Conference on Multimedia and Expo (30% acceptance rate)
2017
Exemplar-Based Image and Video Stylization Using Fully Convolutional Semantic Features
[Paper]
Feida Zhu, Zhicheng Yan, Jiajun Bu, Yizhou Yu
TIP 2017: IEEE Transactions on Image Processing (Impact factor: 5.07)
2016
Learning Concept Taxonomies from Multi-modal Data
[Paper]
Hao Zhang, Zhiting Hu, Yuntian Deng, Mrinmaya Sachan, Zhicheng Yan and Eric Xing
ACL 2016: Association for Computational Linguistics (25.4% acceptance rate)
2015
HD-CNN: Hierarchical Deep Convolutional Neural Network for Large Scale Visual Recognition.
[Paper] [Project] [Github] [Poster] [Video]
Zhicheng Yan, Hao Zhang, Robinson Piramuthu, Vignesh Jagadeesh, Dennis DeCoste, Wei Di, Yizhou Yu
ICCV 2015: International Conference on Computer Vision (~20% acceptance rate)
2013
Sparse Similarity Matrix Learning for Object Retrieval
[Paper]
Zhicheng Yan. Yizhou Yu.
IJCNN 2013: International Joint Conference On Neural Networks.
2009
Volume Illustration of Muscle from Diffusion Tensor Images
Wei Chen, Zhicheng Yan, Song Zhang, John Allen Crow, David S. Ebert, R. McLaughlin, K. Mullins, R. Cooper, Zi’ang Ding, Jun Liao.
IEEE VIS/TVCG 2009: IEEE Transactions on Visualization and Computer Graphics. (Proceedings Visualization / Information Visualization) (~27% acceptance rate)
A Novel Interface for Interactive Exploration of DTI Fibers
Wei Chen, Zi’ang Ding, Song Zhang, Anna MacKay-Brandt, Stephen Correia, Huamin Qu, John Allen Crow, David F. Tate, Zhicheng Yan, Qunsheng Peng.
IEEE VIS/TVCG 2009: IEEE Transactions on Visualization and Computer Graphics (Proceedings Visualization / Information Visualization), (~27% acceptance rate).
Last update on 9/29/2023