Fine-Grained Behaviour Analysis in Virtual Reality

Virtual Reality (VR) is an emerging technology that has manifested the potential to change and improve the way students are educated in many fields. There are several VR applications which have been implemented in STEM domain and have shown gains in constructs related to learning such as conceptual understanding, spatial ability, etc. While researchers in VR have demonstrated how VR facilitate learning, there is less focus on which of the features of VR enhance the constructs related to learning. There are a few studies which analyse learners’ behaviour while they interact with the VR environment and they are based on human observation of participants, interviews, and questionnaires. Exploring VR at a fine-grained level (student actions such as read, look, etc.) will provide deeper insights regarding how students are interacting with the features of VR, and which of the features is enhancing student learning at different levels. 

In computer-based (non-VR) learning environments, log-files generated using learner’s interaction with the environment is used to analyse learner’s behaviour to facilitate adaptation and personalization of the learning environment. A similar methodology to understand how VR features facilitate learning can be implemented by using log-files. However, the existing VR application doesn‘t generate log data similar to click-stream data or equivalent to analyze learner’s behaviour. In this manuscript, we collect the learner’s interaction with VR using screen recordings, which can be further annotated manually to signify various actions of a learner in the VR environment. The log files can be used to study the various features of VR and their contribution towards learning a complex system. 

To summarize, we have designed and applied a mechanism to capture learner’s interaction in a virtual reality-based learning environment, specifically:

• Designed a screen-recording-based manual mechanism to capture detailed learner interaction in virtual reality

• Evaluated learner interactions in a virtual reality environment to understand their interaction with various features of VR while learning.

An extension of this work is being carried out by Antony Prakash from the SMILE Lab, at Edtech, IIT Bombay.