Publications

  1. Xiaohua Zhai*, Xiao Wang*, Basil Mustafa*, Andreas Steiner*, Daniel Keysers, Alexander Kolesnikov, Lucas Beyer*, LiT🔥: Zero-Shot Transfer with Locked-image Text Tuning, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, Louisiana, USA, June 21-24, 2022. (*=equal contribution) [PDF]

  2. Xiaohua Zhai*, Alexander Kolesnikov*, Neil Houlsby, Lucas Beyer*, Scaling Vision Transformers, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, Louisiana, USA, June 21-24, 2022. (*=equal contribution) [PDF]

  3. Lucas Beyer*, Xiaohua Zhai*, Amélie Royer*, Larisa Markeeva*, Rohan Anil, Alexander Kolesnikov*, Knowledge distillation: A good teacher is patient and consistent, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, Louisiana, USA, June 21-24, 2022. (*=equal contribution, Oral) [PDF]

  4. Andreas Steiner*, Alexander Kolesnikov*, Xiaohua Zhai*, Ross Wightman, Jakob Uszkoreit, Lucas Beyer*, How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers, Transactions on Machine Learning Research (TMLR), May, 2022. (*=equal contribution) [PDF, JAX, PyTorch]

  5. Wuyang Chen, Xianzhi Du, Fan Yang, Lucas Beyer, Xiaohua Zhai, Tsung-Yi Lin, Huizhong Chen, Jing Li, Xiaodan Song, Zhangyang Wang, Denny Zhou, A simple single-scale vision transformer for object localization and instance segmentation, European Conference on Computer Vision (ECCV), Oct 2022.

  6. Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, Xiao Wang, Xiaohua Zhai, Thomas Kipf, Neil Houlsby, Simple Open-Vocabulary Object Detection with Vision Transformers, European Conference on Computer Vision (ECCV), Oct 2022.

  7. Alexey Dosovitskiy*, Lucas Beyer*, Alexander Kolesnikov*, Dirk Weissenborn*, Xiaohua Zhai*, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby*, An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, International Conference on Learning Representations (ICLR), May 4-7, 2021. (*=equal contribution, Oral) [PDF, GitHub, AI Blog].

  8. Vincent Dumoulin*, Neil Houlsby*, Utku Evci, Xiaohua Zhai, Ross Goroshin, Sylvain Gelly, Hugo Larochelle, Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmark, Advances in Neural Information Processing Systems (NeurIPS), Dec 6th, 2021. (*=equal contribution) [PDF]

  9. Ilya Tolstikhin*, Neil Houlsby*, Alexander Kolesnikov*, Lucas Beyer*, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, Alexey Dosovitskiy, MLP-Mixer: An all-MLP Architecture for Vision, Advances in Neural Information Processing Systems (NeurIPS), Dec 6th, 2021. (*=equal contribution) [PDF]

  10. Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, Mario Lucic, Revisiting the Calibration of Modern Neural Networks, Advances in Neural Information Processing Systems (NeurIPS), Dec 6th, 2021. [PDF]

  11. Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D'Amour, Dan Moldovan, Sylvan Gelly, Neil Houlsby, Xiaohua Zhai, Mario Lucic, On Robustness and Transferability of Convolutional Neural Networks, Computer Vision and Pattern Recognition (CVPR), 2021. [PDF]

  12. Alexander D'Amour, Katherine Heller, Dan Moldovan, ..., Xiaohua Zhai, D. Sculley, Underspecification Presents Challenges for Credibility in Modern Machine Learning, Journal of Machine Learning Research (JMLR), 2021.

  13. Alexander Kolesnikov*, Lucas Beyer*, Xiaohua Zhai*, Joan Puigcerver, Jessica Yung, Sylvain Gelly and Neil Houlsby, Big Transfer (BiT): General Visual Representation Learning, European Conference on Computer Vision (ECCV), Aug. 24-27, 2020. (*=equal contribution, Spotlight) [PDF, GitHub, AI Blog, TF Blog]

  14. Lucas Beyer*, Olivier J Hénaff*, Alexander Kolesnikov*, Xiaohua Zhai* and Aäron van den Oord*, Are we done with ImageNet?, arXiv preprint arXiv:2006.07159, June 12, 2020. (*=equal contribution) [PDF]

  15. Xiaohua Zhai*, Joan Puigcerver*, Alexander Kolesnikov*, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, Lucas Beyer, Olivier Bachem, Michael Tschannen, Marcin Michalski, Olivier Bousquet, Sylvain Gelly and Neil Houlsby, The Visual Task Adaptation Benchmark, arXiv preprint arXiv:1910.04867, Oct 1st, 2019. (*=equal contribution) [PDF, GitHub, Blog]

  16. Xiaohua Zhai*, Avital Oliver*, Alexander Kolesnikov* and Lucas Beyer*, S4L: Self-Supervised Semi-Supervised Learning, International Conference on Computer Vision (ICCV), Oct. 27-Nov. 2, 2019. (*=equal contribution, Oral). [PDF, GitHub]

  17. Alexander Kolesnikov*, Xiaohua Zhai* and Lucas Beyer*, Revisiting Self-Supervised Visual Representation Learning, 32th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, Jun.16-20, 2019. (*=equal contribution) [PDF, GitHub]

  18. Ting Chen, Xiaohua Zhai, Marvin Ritter, Mario Lucic and Neil Houlsby, Self-Supervised GANs via Auxiliary Rotation Loss, 32th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, Jun.16-20, 2019. [PDF, GitHub]

  19. Mario Lucic*, Michael Tschannen*, Marvin Ritter*, Xiaohua Zhai, Olivier Bachem and Sylvain Gelly, “High-Fidelity Image Generation With Fewer Labels”, 36th International Conference on Machine Learning (ICML), Long Beach, CA, USA, Jun.9-15, 2019. (*=equal contribution) [PDF, GitHub, Blog]

  20. Karol Kurach, Mario Lucic, Xiaohua Zhai, Marcin Michalski and Sylvain Gelly, “The GAN Landscape: Losses, Architectures, Regularization, and Normalization”, 36th International Conference on Machine Learning (ICML), Long Beach, CA, USA, Jun.9-15, 2019. [PDF, GitHub]

  21. Ting Chen*, Xiaohua Zhai* and Neil Houlsby, Self-Supervised GAN to Counter Forgetting, NIPS 2018 Workshop on Continual Learning, Montreal, Canada, Dec 3-7, 2018. [PDF, GitHub] (*=equal contribution)

  22. Yuxin Peng, Xiaohua Zhai, Yunzhen Zhao, and Xin Huang, “Semi-Supervised Cross-Media Feature Learning with Unified Patch Graph Regularization”, IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), Vol. 26, No. 3, pp. 583 - 596 , March 2016. (Regular Paper, SCI, IF 2016 = 3.599)

  23. Xiaohua Zhai, Yuxin Peng, and Jianguo Xiao, “Learning Cross-Media Joint Representation with Sparse and Semi-Supervised Regularization”, IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), Vol. 24, No. 6, pp. 965 - 978 , June 2014. (Regular Paper, SCI, IF 2016 = 3.599)

  24. Xiaohua Zhai, Yuxin Peng, and Jianguo Xiao, “Heterogeneous Metric Learning with Joint Graph Regularization for Cross-Media Retrieval”, AAAI Conference on Artificial Intelligence (AAAI), Bellevue, Washington, USA, July 14-18, 2013. (Oral)

  25. Yuxin Peng, Xiaohua Zhai, Jian Zhang, et, al., “PKU-ICST at TRECVID 2012: Instance Search”, TRECVID, Maryland, USA, Nov. 26-28, 2012.