LiteAT: A Data-Lightweight and User-Adaptive VR Telepresence System for Remote Education

IEEE TVCG (Special Issue on IEEE ISMAR 2025)

Best Paper Awards Honorable Mention

Yuxin Shen1, Wei Liang1,2,3, Jianzhu Ma4,5
1 School of Computer Science, Beijing Institute of Technology, Beijing, China
2 Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing, China
3 State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China
4 Department of Electronic Engineering, Tsinghua University, Beijing, China
5 Institute for AI Industry Research, Tsinghua University, Beijing, China

Paper

Code

Video

🌺 Abstract

In educators' ongoing pursuit of enriching remote education, VR-based telepresence has shown significant promise due to its immersive and interactive nature. Existing approaches often rely on point cloud or NeRF-based techniques to deliver realistic representations of teachers and classrooms to remote students. However, achieving low latency is non-trivial, and maintaining high-fidelity rendering under such constraints poses an even greater challenge. This paper introduces LiteAT, a data-lightweight and user-adaptive VR telepresence system, to enable real-time, immersive learning experiences. LiteAT employs a Gaussian Splatting-based reconstruction pipeline that integrates an SMPL-X-driven dynamic human model with a static classroom, supporting lightweight data transmission and high-quality rendering. To enable efficient and personalized exploration in the virtual classroom, we propose a user-adaptive viewpoint recommendation framework that dynamically suggests high-quality viewpoints tailored to user preferences. Candidate viewpoints are evaluated based on multiple visual quality factors and are continuously optimized based on recent user behavior and scene dynamics. Quantitative experiments and user studies validate the effectiveness of LiteAT across multiple evaluation metrics. LiteAT establishes a versatile and scalable foundation for immersive telepresence, potentially supporting real-time scenarios such as procedural teaching, multimodal instruction, and collaborative learning.

🌷System Overview

LiteAT operates in two stages. In the Pre-Class Preparation Stage, local teachers record the classroom and themselves to reconstruct the static classroom and a canonical human model. In the In-Class Teaching Stage, estimated SMPL-X parameters and audio are transmitted to animate the canonical human, which is then combined with the static classroom and rendered in students’ VR headsets. A viewpoint recommendation module generates high-quality viewpoints that remote students can switch between for flexible exploration.