Research Themes

  • Individualized modeling of biophysical entities

  • Individualized’ sensing using multi-modal data fusion and robust learning

  • Individualized’ actuation using dexterous robots

Individualized modeling of biophysical entities

Individual plants behave and develop differently due to variable environmental conditions (soil, local nutrient availability, local microclimate, local stress). The use of predictive (temporal) models at the individual plant scale integrated with field-level modeling using a networked, hierarchical approach will allow cyber-physical modeling and control at that plant-scale resolution for large areas. Current capabilities in persistent environmental sensing can provide hyper-local environmental conditions, suggesting that individualized models are possible. However, relying only on data to create such models leaves out contextual constraints and valuable domain knowledge. This calls for principled methods for creating individualized plant models that tightly couple multi-scale data with known biophysical and physiological knowledge. This will additionally ensure that model predictions follow known biophysical rules, thus ensuring generalizability.

Individualized’ sensing using multi-modal data fusion and robust learning

Updating the individualized models require multi-modal measurements to estimate the state variables. These measurements will be performed at different scales of environmental conditions and plant physiology, and they will potentially be quite noisy due to degraded sensing environments. Hence, robust machine learning approaches are needed for feature extraction and fusion of multi-scale, multi-modal data to update the models.

Individualized’ actuation using dexterous robots

Individualized actuation include localized chemical and water distribution (spraying, injection) and mechanical crop management operations. This requires dexterous actuators to enable precise chemical dispersion. Our prior work shows substantial promise in outfitting small, cheap autonomous ground vehicles with dexterous actuators in order to minimize the risk of accidents.


Insect-Pest Classifier

Insect-pest affect crops annually and farmers incur huge economic losses as a consequence. Before coming up with measure to mitigate them, it is essential to identify them correctly, as an incorrect identification of an insect-pest could result in eradicating a pest that could prove beneficial to the crop otherwise .We propose an insect-pest identification mobile app that identifies agriculturally important insect-pests. The classification result would also be integrated with Integrated Pest Management (IPM) strategies.

Insect Detection System Integration

Integrating Terresentia and MyCobot Elephant Arm

Attached Custom Sprayer and Camera 

Incorporated MoveIt Simulation for Control and Motion Planning.

Multimodal Data Fusion for Phenotyping

Sensors collect large volumes of plant canopy reflectance data to monitor, forecast, and perform important tasks. These data are usually heterogeneous making it difficult for usage. We are exploring multimodal data fusion using cross-modal synergy to improve performance of agricultural related tasks.


This research focuses on building a simulation environment for biotic stresses with a user-definable management zone. AgGym, our proposed simulation platform, acts as a Gym environment to design deep reinforcement learning (RL) based mitigation strategies for localized plant biotic stresses without extensive field experimentation.

Distributed Deep Learning for Persistent Monitoring of agricultural Fields

Recent advances in sensor technology have enabled the cheap collection of spatial and temporal high-resolution data for agriculture across a wide geographical area. We deploy distributed deep learning algorithms for a wide variety of decision support tasks in agriculture such as as anomaly detection and image retrieval.

Realistic Simulation Environments For Visual Servoing

Developed simulation environments (“digital twins”) for robot learning tasks in agricultural settings

Created multiple realistic simulations using Blender

Each environment consists of several obstacles and targets created using geometrical meshes

Model-Based Dynamic Position and Orientation Control of a Hybrid Soft Continuum Arm

Dynamic Control for quick coarse control

Visual Servoing for fine tuned manipulation

Closing the Sensing-Actuation Loop

Obtaining stable images from tip camera

Soft arm controls and visual servoing

Path planning through and around obstacle

High Throughput Network for Autonomous Farms

Up and coming applications like teleoperation, edge-controlled robots require high network throughput. However, existing networks are not well suited for farms either due to low throughput (TVWS, LoRa), unavailability (Cellular) or poor range (Wifi, Zigbee).

In this project, we benchmark the performance of Wifi in farms in context of high throughput applications. Our measurements show that throughput degrades quickly with distance even when using latest standard of Wifi 6. We also observe that while 5GHz band give substantially higher throughput compared to 2.4 GHz band, its range is extremely poor. Thus, there is a tradeoff between throughput and range between two bands.

Our findings will help guide us to design a traffic aware Wifi mesh network.