A digital twin is a digital representation of an intended or actual real-world physical product, system, or process (a physical twin) that serves as the effectively indistinguishable digital counterpart of it for practical purposes, such as simulation, integration, testing, monitoring, and maintenance. In our work, the physical robot, Dobot Magician, and simulated robot in Unity/VR, build up a pair of digital twins to increase system intuitiveness.
Mixed reality is a term used to describe the merging of a real-world environment and a computer-generated one. Physical and virtual objects may co-exist in mixed reality environments and interact in real time. In our work, operators could observe and interact with the remote robot through a virtual environment and haptic rendering.
Haptic of humans contains two types of receptors, the first is the tactile receptors located in the dermis and epidermis of the skin, and the corresponding evoked sensations are called tactile feedback. The second is the kinesthetic receptors located in the joints and ligaments, and the corresponding evoked sensations are called force feedback or kinesthetic feedback.
In order to simulate the haptic and to apply them to the human body through equipment, the corresponding generation methods of the upper two evoked sensations are called force/kinesthetic rendering and tactile rendering, respectively.
In our paper, haptic rendering consists of two parts:
(1) Kinaesthetic rendering (Through Geomagic Touch)
(2) Tactile rendering (Through SPTA device)
Compared to traditional visual and auditory interaction, haptic interaction can produce a more realistic sense of immersion and has an irreplaceable role. Haptic rendering can generate mechanical signals to give users feedback through human kinesthetic and tactile channels, significantly broadening the interaction bandwidth and enhancing realism and immersion.