We implemented a reinforcement learning–based framework for robust autonomous drone navigation under wind disturbances, building on the existing NavRL framework developed at CERLab. The system is designed to handle dynamic obstacle avoidance in realistic, cluttered environments where external disturbances such as wind can significantly degrade control performance. All training and evaluation were carried out in Isaac Sim, which provides high-fidelity physics, aerodynamic effects, and large-scale parallel simulation for efficient learning.

To improve temporal awareness and stability under disturbances, we extended the NavRL observation pipeline by stacking a history of drone states and processing this temporal information using an Transformer/LSTM-based policy architecture. Rather than relying solely on instantaneous observations, this enables the policy to infer unobserved dynamics such as wind forces, delayed system responses, and obstacle motion by learning patterns over time. This temporal learning is particularly important for dynamic obstacle avoidance, where safe decisions often depend on how the environment is evolving rather than a single snapshot.

The reinforcement learning policy was trained using thousands of parallel drone instances, allowing the agent to experience a wide range of wind conditions, obstacle configurations, and interaction scenarios. Through this process, the policy learned control strategies that are both reactive and robust, maintaining stable flight while avoiding collisions even in the presence of strong and unpredictable disturbances. Quantitative evaluation showed a significant improvement in performance metrics—including success rate, collision avoidance, and trajectory stability—compared to non-temporal baselines, especially in environments with moving obstacles. Overall, this framework demonstrates how integrating temporal learning into the NavRL pipeline substantially enhances the reliability and safety of autonomous drone navigation in challenging real-world conditions.

Metric after implementing Transformer  learning for obstacle avoidance in Isaac Sim in windy conditions

This project implements Proximal Policy Optimization (PPO),Off policy and Model based policy  for robosuite manipulation (e.g. Lift with a Panda robot): training uses Stable-Baselines3 and Gymnasium with observation and reward normalization, periodic evaluation, checkpointing, and optional Weights & Biases logging; evaluation loads a saved policy and VecNormalize stats and runs deterministic rollouts with configurable episodes and an on-screen or headless viewer. Overall, the Project provides a ready-to-use PPO training and evaluation stack for robosuite.

Designed and implemented a real-time convex MPC controller in ROS 2 for the Unitree Go2 quadruped, casting centroidal balance and motion tracking as a quadratic program that optimizes ground reaction forces over a prediction horizon. The formulation enforces friction-cone, unilateral contact, and minimum/maximum normal force constraints to avoid slipping and ensure physically valid contacts.Integrated Gurobi as the high-performance QP backend and Pinocchio for rigid-body dynamics and kinematics, using foot Jacobians to convert optimized contact forces into joint torques. Connected the full pipeline to the robot’s state-estimation and low-level control interfaces, enabling closed-loop torque control for stable stance.