Research

Fingertips are one of the most sensitive regions of the human body and provide a means to dexterously interact with the physical world. To recreate this sense of physical touch in a virtual or augmented reality (VR/AR), high-resolution haptic interfaces that can render rich tactile information are needed. In this paper, we present a wearable electrohydraulic haptic interface that can produce high-fidelity multimodal haptic feedback at the fingertips. This novel hardware can generate high intensity fine tactile pressure (up to 34 kPa) as well as a wide range of vibrations (up to 700 Hz) through 16 individually controlled electrohydraulic bubble actuators. To achieve such a high intensity multimodal haptic feedback at such a high density (16 bubbles/cm2) at the fingertip using an electrohydraulic haptic interface, we integrated a stretchable substrate with a novel dielectric film and developed a design architecture wherein the dielectric fluid is stored at the back of the fingertip. We physically characterize the static and dynamic behavior of the device. In addition, we conduct psychophysical characterization of the device through a set of user studies. This electrohydraulic interface demonstrates a new way to design and develop high-resolution multimodal haptic systems at the fingertips for AR/VR environments. 

We present two types of subject-specific assist-as-needed controllers. Learned force-field control is a novel control technique in which a neural-network-based model of the required torques given the joint angles for a specific subject is learned and then used to build a force-field to assist the joint motion of the subject to follow a trajectory designed in the joint-angle space. Adaptive assist-as-needed control,on the other hand, estimates the coupled digit-exoskeleton system torque requirement of a subject using radial basis function (RBF) and on-the-fly adapts the RBF magnitudes to provide a feed-forward assistance for improved trajectory tracking. Experiments on the index finger exoskeleton prototype with a healthy subject showed that while the force-field control is non-adaptive and there is less control on the speed of execution of the task, it is safer as it doesnot apply increased torques if the finger motion is restricted. On the other hand, adaptive assist-as-needed controller adapts to the changing needs of the coupled finger-exoskeleton system and helps in performing the task with a consistent speed, however, applies increased torques in case of restricted motion and therefore, less safe.

A critical question to be answered to improve robotic rehabilitation is what is the optimal rehabilitation environment for a subject that will facilitate maximum recovery during therapy? Studies suggest that task variability, nature and degree of assistance or error-augmentation and type of feedback play a critical role in motor (re)-learning. In this work, we present a framework for robot-assisted motor (re)-learning that provides subject-specific training by allowing for simultaneous adaptation of task, assistance and feedback based on the performance of the subject on the task. We model a continuous and coordinated multi-joint task using a learning from demonstration approach, which allows the task to be modeled in a generative manner such that the challenge-level of the task could be modulated in an online manner. To train the subjects for dexterous manipulation, we present a torque- based task that requires subject to dynamically regulate their joint torques. Finally, we carry out a pilot study with healthy human subjects using our previously developed hand exoskeleton to test a hypothesis and the results suggest that training under simultaneous adaptation of task, assistance and feedback positively affects motor learning.

Effective teleoperation requires real-time control of a remote robotic system. In this work, we develop a controller for realizing smooth and accurate motion of a robotic head with application to a teleoperation system for the Furhat robot head [1], which we call TeleFurhat. The controller uses the head motion of an operator measured by a Microsoft Kinect 2 sensor as reference and applies a processing framework to condition and render the motion on the robot head. The processing framework includes a pre-filter based on a moving average filter, a neural network-based model for improving the accuracy of the raw pose measurements of Kinect, and a constrained-state Kalman filter that uses a minimum jerk model to smooth motion trajectories and limit the magnitude of changes in position, velocity, and acceleration. Our results demonstrate that the robot can reproduce the human head motion in real time with a latency of approximately 100 to 170 ms while operating within its physical limits. Furthermore, viewers prefer our new method over rendering the raw pose data from Kinect.

Exoskeletons are a new class of articulated mechanical systems whose performance is realized while in intimate contact with the human user. The overall performance depends on many factors including selection of architecture, device, parameters and the nature of the coupling to the human, offering numerous challenges to design evaluation and refinement. In this paper, we discuss merger of techniques from the musculoskeletal analysis and simulation-based design to study and analyze the performance of such exoskeletons. A representative example of a simplified exoskeleton interacting with and assisting the human arm is used to illustrate principal ideas. Overall, four different case-scenarios are developed and examined with quantitative performance measures to evaluate the effectiveness of the design and allow for design refinement. The results show that augmentation by way of the exoskeleton can lead to a significant reduction in muscle loading.

Human pose estimation using monocular vision is a challenging problem in computer vision. Past work has focused on developing efficient inference algorithms and probabilistic prior models based on captured kinematic/dynamic measurements. However, such algorithms face challenges in generalization beyond the learned dataset. In this work, we propose a model-based generative approach for estimating the human pose solely from uncalibrated monocular video in unconstrained environments without any prior learning on motion capture/image annotation data. We propose a novel Product of Heading Experts (PoHE) based generalized heading estimation framework by probabilistically-merging heading outputs (probabilistic/ non-probabilistic) from time varying number of estimators. Our current implementation employs motion cues based human heading estimation framework to bootstrap a synergistically integrated probabilistic-deterministic sequential optimization framework to robustly estimate human pose. Novel pixel-distance based performance measures are developed to penalize false human detections and ensure identity-maintained human tracking. We tested our framework with varied inputs (silhouette and bounding boxes) to evaluate, compare and benchmark it against ground-truth data (collected using our human annotation tool) for 52 video vignettes in the publicly available DARPA Mind’s Eye Year I dataset 1. Results show robust pose estimates on this challenging dataset of highly diverse activities. Read More

Estimating the physical parameters of articulated multibody systems (AMBSs) using an uncalibrated monocular camera poses significant challenges for vision-based robotics. Articulated multibody models, especially ones including dynamics, have shown good performance for pose tracking, but require good estimates of system parameters. In this paper, we first propose a technique for estimating parameters of a dynamically equivalent model (kinematic/geometric lengths as well as mass, inertia, damping coefficients) given only the underlying articulated model topology. The estimated dynamically equivalent model is then employed to help predict/filter/gap-fill the raw pose estimates, using an unscented Kalman filter. The framework is tested initially on videos of a relatively simple AMBS (double pendulum in a structured laboratory environment). The double pendulum not only served as a surrogate model for the human lower limb in flight phase, but also helped evaluate the role of model fidelity. The treatment is then extended to realize physically plausible pose-estimates of human lower-limb motions, in more-complex uncalibrated monocular videos (from the publicly available DARPA Mind's Eye Year 1 corpus). Beyond the immediate problem-at-hand, the presented work has applications in creation of low-order surrogate computational dynamics models for analysis, control, and tracking of many other articulated multibody robotic systems (e.g., manipulators, humanoids) using vision.  Read More