Generative Pretraining of Image and Multimodal Transformers

• Where are my Neighbors? Exploiting Patches Relations in Self-Supervised Vision Transformer

• Is Large-scale Pre-training always Necessary for Vision Transformers?

• Self-Supervised Pre-training of Vision Transformers for Dense Prediction Tasks

• MC-SSL: Towards Multi-Concept Self-Supervised Learning

• Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning

• VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

Zhuowen Tu (UCSD)

Transformers for Robotics and Video Analysis

Long-context Anymodal Generation with Perceivers

Spotlight Talks: 3D and Applications

• NEAT: Neural Attention Fields for End-to-End Autonomous Driving

• BoxeR: Box-Attention for 2D and 3D Transformers

• Depth Estimation with Simplified Transformer

• M2F3D: Mask2Former for 3D Instance Segmentation

• Helix4D: Online Semantic Segmentation of LiDAR Sequences

Poster Session