Welcome to Xiangyu Xu's Homepage!

“We are trying to prove ourselves wrong as quickly as possible, because only in that way can we find progress.” ― Richard P. Feynman


Contact me: xuxiangyu2014@gmail.com

I am currently a research fellow at Nanyang Technological University.

My research interest lies at the intersection of Image Processing, Computer Vision, and Machine Learning. In particular, my work focuses on understanding different aspects of visual information in images and videos, including texture, resolution, color, motion, depth, geometry, illumination, etc., and developing algorithms to reconstruct and enhance them. My work includes BlindSR, RawSR, QVI, RSC-Net, and Texformer.

Short bio: I did my postdoc at Carnegie Mellon University and Massachusetts Institute of Technology from 2019 to 2020. I worked as a research scientist at SenseTime from 2018 to 2019. I got the Ph.D. degree in the Department of Electronic Engineering at Tsinghua University in 2018. Before this, I received my B.Eng degree in the Department of Electronic Engineering at Tsinghua University, and my B.Ec degree in the National School of Development at Peking University. I was a visiting scholar at University of California, Merced and Harvard University. I have been fortunate to work with Yu-Jin Zhang at Tsinghua, Ming-Hsuan Yang at UC Merced, Deqing Sun and Hanspeter Pfister at Harvard, Fernando De la Torre, Francesc Moreno-Noguer, and Laszlo A. Jeni at CMU, George Barbastathis at MIT, Chen Change Loy at NTU.

I am open to collaborations in a broad area of computer vision. Please drop me an email if you are interested.

Publications

Zhihao Shi*, Xiangyu Xu*#, Xiaohong Liu, Jun Chen, Ming-Hsuan Yang, “Video Frame Interpolation Transformer”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

[Paper, Project]

Kelvin C.K. Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy, "Investigating Tradeoffs in Real-World Video Super-Resolution", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

[Paper]

Kelvin C.K. Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy, "BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

[Paper]

Xiangyu Xu, Chen Change Loy, "3D Human Texture Estimation from a Single Image with Transformers", IEEE International Conference on Computer Vision (ICCV), 2021. (Oral)

[Paper, Project, ICCV Daily]

Xiangyu Xu, Hao Chen, Francesc Moreno-Noguer, Laszlo A. Jeni, and Fernando De la Torre, “3D Human Pose, Shape and Texture from Low-Resolution Images and Videos”, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021.

Extension of RSC-Net (ECCV'20): reconstruct human textures; temporally-coherent motion from videos.

[Paper, Project]

Kelvin C.K. Chan, XintaoWang, Xiangyu Xu, Jinwei Gu, and Chen Change Loy, "GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (Oral)

[Paper, Project]

Sven Mayer, Xiangyu Xu, and Chris Harrison, "Super-Resolution Capacitive Touchscreens", ACM Conference on Human Factors in Computing Systems (CHI), 2021.

[Paper, Video]

Xiangyu Xu, Yongrui Ma, Wenxiu Sun, and Ming-Hsuan Yang, "Exploiting Raw Images for Real-Scene Super-Resolution", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020.

Extension of RawSR (CVPR'19): dense channel attention; learned guided filtering; applications in image dehazing and joint depth upsampling

[Paper, Project]

Xiangyu Xu, Hao Chen, Francesc Moreno-Noguer, Laszlo A. Jeni, and Fernando De la Torre, "3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning", European Conference on Computer Vision (ECCV), 2020.

Highlight: RSC-Net -- the first 3D human estimation method for low-resolution images; self-supervised learning and contrastive learning are used to deal with weak supervision.

[Paper, Project]

Xiangyu Xu, Muchen Li, Wenxiu Sun, and Ming-Hsuan Yang, "Learning Spatial and Spatio-Temporal Pixel Aggregations for Image and Video Denoising", IEEE Transactions on Image Processing (TIP), 2020.

Early version: "Learning Deformable Kernels for Image and Video Denoising", axXiv 1904.06903.

Highlight: deformable convolutional neural network (DCN) for image and video denoising; the first time DCN is extended to the spatio-temporal space

[Paper, Project]

Xiangyu Xu, Yongrui Ma, Wenxiu Sun, "Learning Factorized Weight Matrix for Joint Filtering", International Conference on Machine Learning (ICML), 2020.

Highlight: deep learning based Robust PCA for image filtering

[Paper, Project]

Xiangyu Xu*, Siyao Li*, Wenxiu Sun, Qian Yin, and Ming-Hsuan Yang, "Quadratic Video Interpolation", Advances in Neural Information Processing Systems (NeurIPS), 2019. (Spotlight, Winner of the AIM video interpolation challenge ICCV 2019)

Highlight: the first nonlinear video frame interpolation method

[Paper, Project]

Xiangyu Xu, Yongrui Ma, Wenxiu Sun, "Towards Real Scene Super-Resolution with Raw Images", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Highlight: RawSR -- the first work on raw image based super-resolution

[Paper, Project]

Wenqi Ren, Sifei Liu, Lin Ma, Qianqian Xu, Xiangyu Xu, Xiaochun Cao, Junping Du, and Ming-Hsuan Yang, "Low-Light Image Enhancement via a Deep Hybrid Network", IEEE Transactions on Image Processing (TIP), 2019.

[Paper, Project]


Xiangyu Xu, Deqing Sun, Sifei Liu, Wenqi Ren, Yu-Jin Zhang, Ming-Hsuan Yang, and Jian Sun, "Rendering Portraitures from Monocular Camera and Beyond", European Conference on Computer Vision (ECCV), 2018.

Highlight: the first deep learning method for monocular bokeh/defocus effect rendering

[Paper]

Yukang Gan*, Xiangyu Xu*, Wenxiu Sun, and Liang Lin, "Monocular Depth Estimation with Affinity, Vertical Pooling and Label Enhancement", European Conference on Computer Vision (ECCV), 2018.

(* indicates equal contribution)

[Paper]

Xiangyu Xu, Jinshan Pan, Yu-Jin Zhang, and Ming-Hsuan Yang, "Motion Blur Kernel Estimation via Deep Learning", IEEE Transactions on Image Processing (TIP), 2018.

Highlight: deep filters to extract salient edges for image deblurring

[Paper, Project]

Wenqi Ren*, Jingang Zhang*, Xiangyu Xu*, Lin Ma, Xiaochun Cao, Gaofeng Meng, and Wei Liu, "Deep Video Dehazing with Semantic Segmentation'', IEEE Transactions on Image Processing (TIP), 2018.

(* indicates equal contribution)

[Paper]

Xiangyu Xu, Deqing Sun, Jinshan Pan, Yu-Jin Zhang, Hanspeter Pfister, and Ming-Hsuan Yang, "Learning to Super-resolve Blurry Face and Text Images", IEEE International Conference on Computer Vision (ICCV), 2017.

Highlight: the first deep neural network for blind image super-resolution

[Paper, Project]

Professional service

  • Conference reviewer: ICCV, CVPR, ECCV, BMVC, ACCV, WACV, NeurIPS, ICML, ICLR, AAAI, IJCAI

  • Journal reviewer: IEEE TPAMI, IEEE TIP, IEEE TVCG, IJCV, ACM TOG

Useful links