Research

Current Research Areas

My current research interests are principally in the following areas:

• Mobile And Wearable Computing (activity and gesture recognition, IoT-based sensing, wireless energy harvesting, novel wearable interfaces) Less...

This research thread focuses on the judicious use of smartphones and wearable devices, in combination with emerging infrastructure-based IoT devices, to derive an understanding of the “what, where, when and why” people do, in smart spaces. Examples of recent/ongoing projects include:

· Eating and Shopping Analytics: This work explores using wearable devices (such as a wrist-worn smartwatch) to gather fine-grained insights about an individual’s commonplace daily activities, such as shopping and eating. For eating analytics, we have built a fully-automated deployable system called Annapurna [http://is.gd/annapurna] [paper1,paper2] that combines gesture detection using accelerometer and gyroscope sensor data with automatic triggering of the smartwatch-embedded camera to capture a high-quality image of the food being eaten. Similarly, we use both wearables and lightweight IoT devices (specifically BLE beacons) to capture shopper-product interactions in stores. The IRIS platform [paper] used a combination of smartphone and smartwatch sensor data to build a shopper’s profile based on inferring a shopper’s micro-gestural activities, such as “picking up an item” or “placing it in a shopping cart.” More recently, we have built a solution, called I4S, that combines multiple low-energy BLE beacons, mounted on store shelves, with smartwatch sensing to further identify the shelf-level locations from where users pick specific items (see [paper] for preliminary work).

· Robust Context & Activity Recognition: This body of work [paper1, paper2] tackles the problem of robustly identifying the activities of daily living (ADL) and physical activities of individuals. Recently, our focus has been on making such activity detection robust, across variations in individuals, devices and on-body device placements. In [paper], we have introduced such a domain adaptation approach for deep learning based classifiers: the key idea is to rapidly adapt a CNN-based classifier to a different domain, with minimal training data, by preserving the distribution of weights in the hidden layers of a previously trained classification models.

· VR/AR Devices & Gestural Interfaces: This body of work has been looking at the use of augmented reality (AR) & virtual reality (VR) smartglasses, often in tandem with other wearables, for creating new immersive interfaces and applications. In [paper], we have recently shown how to develop very low-latency, accurate tracking and recognition of hand gestures, to enable a new class of interactive, wearable applications. AR devices have also been used [paper] to enable a new form of wearable-based pathfinding interface, where video snippet-based directions are overlaid on the smartglass of an individual. Most recently, we have shown [paper] how to use VR devices, together with wearable sensors and device virtualization, to develop empathetic interfaces that allow users to mimic the real-world smartphone usage application of users with impairments.

· Battery-less Wearable Systems: Very recently, we have been investigating novel energy harvesting techniques to enable the design of wearable devices, which can operate without batteries, using “energy harvesting”. The key idea to date [paper] is to use WiFi transmissions, with appropriate beamforming that increases the efficiency of energy transfer, to recharge wearable devices, which are themselves equipped with novel sensors that enable them to be activated only when the wearer engages in relevant gestural activities. In a related, but separate effort, we are investigating [paper] how simple battery-less RFID wearables can be used to capture details of human activities in smart homes and offices. In ongoing work, we are expanding this concept to develop adaptation techniques, which allow a WiFi AP to be used concurrently for data transfer, sensing and energy harvesting.