Research

Vision–Language–Action (VLA) models generalize well from web-scale data but lack awareness of physical contact, limiting their performance in contact-rich manipulation. We introduce DreamTacVLA, a touch-aware VLA framework that grounds action in contact physics by combining high-resolution tactile images with wrist and third-person visual inputs through a hierarchical perception scheme. A Hierarchical Spatial Alignment loss aligns multi-scale sensory tokens, while a tactile world model enables the system to predict future tactile signals and reason about fine-grained contact dynamics, trained on a hybrid digital-twin and real-world dataset. By conditioning actions on both observed and imagined tactile states, DreamTacVLA significantly outperforms prior VLA baselines, achieving up to 95% success on contact-rich manipulation tasks.

We participated in the Life Science R&D - Experiment Track, focusing on precise laboratory operations, including column chromatography setup and sample handling.

Recent work in dexterous in-hand manipulation achieves object reorientation but often lacks truly stable grasps, relying on palm support or fingertip lifting. Inspired by human fingertip force-torque sensing, we integrate force–torque measurements into the observation space of reinforcement learning policies and introduce a force-closure-based reward to explicitly encourage grasp stability. We evaluate this approach on a simulated cube reorientation task with a multi-fingered robotic hand, comparing policies trained with and without force-torque inputs and stability rewards. Results indicate improved grasp stability, smoother in-hand rotations, and faster early training, highlighting the promise of embedding physical grasp constraints into policy learning.

The contribution of different robotic touch modalities to the extrinsic contact perception, particularly in assisting contact-rich manipulation tasks, remains unclear. We explore a Learning from Demonstration (LfD) framework by collecting human demonstration data for a set of pick-place-insert tasks with observations from vision and diverse touch modalities, including joint torque, contact force, tactile image, and acoustic response, and evaluate the effects of the different touch modalities on the task performance. The findings will serve as a reference for the choice of touch modalities for robotic skill learning in contact-rich manipulation tasks and offer empirical perspectives for the design of multimodal learning algorithms.

We develop an active acoustic sensing method for robot manipulation to address the limitations of visual and tactile sensing. Active acoustic sensing relies on the resonant properties of the object, which are related to its material, shape, internal structure, and contact interactions with the gripper and environment. The sensor consists of a vibration actuator paired with a piezo-electric microphone. The actuator generates a waveform, and the microphone tracks the waveform's propagation and distortion as it travels through the object.

We present a language-guided system for multimodal haptic texture authoring that converts natural-language prompts into coordinated haptic signals (sliding vibrations and tapping transients) and a text-conditioned visual preview via a shared, language-aligned latent space. A user study shows that this latent captures perceptually meaningful attributes, including roughness, slipperiness, and hardness, with interpretable trends across modalities. These results demonstrate that language can serve as an effective control interface for texture authoring, enabling a prompt-first workflow that replaces manual parameter tuning with intuitive, text-guided refinement.

This work proposes an interactive texture generation and search framework driven by user input. We design a GAN-based texture model generator, which can create a wide range of texture models using Auto-Regressive processes as the basis. Our interactive texture search method, which we call “preference-driven”, follows an evolutionary strategy given guidance from user's preferred feedback within a set of generated texture models. 

This work presents a large-scale texture classification method using a novel texture feature Projected Spectral Mapping (PSM) based on audio-tactile crossmodal congruence in unconstrained tool-surface interactions. We describe a quick-computable extraction process for PSM from the proposed crossmodal inter-band spectral mapping (IBSM) that relates the frequency components in different bands between the modalities. 

This paper presents a statistical learning-based approach for modeling and rendering touch-produced sounds in real time. We apply a data-driven modeling method, which recreates highly realistic sounds using audio signals recorded from unconstrained tool-surface interactions.

This paper extends the concept of Smart Braids to the broader class of two-fiber-family cylindrical FREEs. A dimensionless model for the inductance of Smart Braid FREEs is presented that can be scaled to specific sensor geometries.

This study aimed to assess the detrimental effects of time delays in teleoperation on operators’ workload and performance, and how a delay compensation aid mitigated such effects. We conducted a human-in-the-loop experiment with 36 participants using a dual-task teleoperation platform.

This study aimed to assess how one delay compensation algorithm, the model-free predictor, affects operators' workload and teleoperation performance. A dual-task driving simulator was utilized in the present study, where participants drove a High Mobility Multipurpose Wheeled Vehicle (HMMWV) while performing a 1-back memory task. Preliminary results revealed that the delay compensation aid can reduce operators' workload while enhancing primary task performance.