The paper addresses a continuous-time continuous-space chance-constrained stochastic optimal control (SOC) problem where the probability of failure to satisfy given state constraints is explicitly bounded. We leverage the notion of exit time from continuous-time stochastic calculus to formulate a chance-constrained SOC problem. Without any conservative approximation, the chance constraint is transformed into an expectation of an indicator function which can be incorporated into the cost function by considering a dual formulation. We then express the dual function in terms of the solution to a Hamilton-Jacobi-Bellman partial differential equation parameterized by the dual variable. Under a certain assumption on the system dynamics and cost function, it is shown that a strong duality holds between the primal chance-constrained problem and its dual. The Path integral approach is utilized to numerically solve the dual problem via gradient ascent using open-loop samples of system trajectories. We present simulation studies on chance-constrained motion planning for spatial navigation of mobile robots and the solution of the path integral approach is compared with that of the finite difference method.

In this paper, we deal with the task hierarchical control problem where several subtasks having different levels of importance need to be accomplished by a robotic system. The most frequently applied method to accomplish task hierarchical control is the null space projection technique. In the null-space projection, the output of each task in the hierarchy is projected onto the null space of an immediately higher-priority task. In the state-of-the-art literature, the individual controllers for each task in the hierarchy are designed using simple low-level controllers. Although these individual controllers are easy to design, they do not provide global optimal control policies. In this paper, we combine the null-space projection technique with the path integral control method providing optimal path plans of task behavior. The proposed control synthesis method is validated via simulation studies of a two-vehicle system.

In this paper, we derive a path integral formulation for a discrete-time stochastic LQR problem and conduct its sample complexity analysis. It is well known that the stochastic LQR problem can be solved efficiently using the backward Riccati equation. However, we focus on stochastic LQR with an aim to build a foundation for a theoretical understanding of the sample complexity of path integral when the analytical expressions of optimal control law and the cost are available. An end-to-end bound on the error in the control signals is derived as a function of sample size. Our analysis shows that the sample size of path integral depends only logarithmically on the dimension of the control input; hence does not suffer from the curse of dimensionality. Finally, we provide a mechanism to quantify the worst-case performance loss of the path integral control.

We consider a setting where a supervisor delegates an agent to perform a certain control task, while the agent is incentivized to deviate from the given policy to achieve its own goal. In this work, we synthesize the optimal deceptive policies for an agent who attempts to hide its deviations from the supervisor's policy. We study the deception problem in the continuous-state discrete-time stochastic dynamics setting and, using motivations from hypothesis testing theory, formulate a Kullback-Leibler control problem for the synthesis of deceptive policies. This problem can be solved using backward dynamic programming in principle, which suffers from the curse of dimensionality. However, under the assumption of deterministic state dynamics, we show that the optimal deceptive actions can be generated using path integral control. This allows the agent to numerically compute the deceptive actions via Monte Carlo simulations. Since Monte Carlo simulations can be efficiently parallelized, our approach allows the agent to generate deceptive control actions online. We show that the proposed simulation-driven control approach asymptotically converges to the optimal control distribution.

This paper addresses a continuous-time risk-minimizing two-player zero-sum stochastic differential game (SDG), in which each player aims to minimize its probability of failure (i.e., the event that the state of the game enters into predefined undesirable domains). We derive a sufficient condition for this game to have a saddle-point equilibrium, which leads to a Hamilton-Jacobi-Isaacs (HJI) partial differential equation (PDE) with Dirichlet boundary condition. A path integral framework is developed to numerically solve this Dirichlet boundary value problem via Monte Carlo sampling of system trajectories. We implement our control synthesis framework on two classes of risk-minimizing zero-sum SDGs: a disturbance attenuation problem and a pursuit-evasion game. Simulation studies are presented to validate the proposed control synthesis framework.

The paper addresses a continuous-time continuous-space chance-constrained stochastic optimal control (SOC) problem via a HamiltonJacobi-Bellman (HJB) partial differential equation (PDE) with Dirichlet boundary condition. We leverage the notion of exit time from continuous-time stochastic calculus to formulate a chance-constrained (risk-constrained) SOC problem and convert it to a risk-minimizing SOC problem via Lagrangian relaxation. The resulting Lagrangian possesses the time-additive Bellman structure, allowing the use of dynamic programming to rewrite the risk-minimizing SOC problem as a problem of solving an HJB PDE. We show that the boundary condition of this HJB PDE can be tuned appropriately to achieve a desired level of safety. Furthermore, it is shown that the proposed risk-minimizing control problem can be viewed as a generalization of the problem of estimating the risk associated with a given control policy. Two numerical techniques are explored, namely the path integral and finite difference method (FDM), to solve a class of risk-minimizing SOC problems whose associated HJB equation is linearizable via the Cole-Hopf transformation. We apply our control synthesis framework to a robot navigation problem with 2D and 3D state-space models and compare the solutions obtained using path integral and FDM. 

This paper addresses a continuous-time continuous-space chance-constrained stochastic optimal control (SOC) problem via a Hamilton-Jacobi-Bellman (HJB) partial differential equation (PDE). Through Lagrangian relaxation, we convert the chance-constrained (risk-constrained) SOC problem to a risk-minimizing SOC problem, the cost function of which possesses the time-additive Bellman structure. We show that the risk-minimizing control synthesis is equivalent to solving an HJB PDE whose boundary condition can be tuned appropriately to achieve a desired level of safety. Furthermore, it is shown that the proposed risk-minimizing control problem can be viewed as a generalization of the problem of estimating the risk associated with a given control policy. Two numerical techniques are explored, namely the path integral and the finite difference method (FDM), to solve a class of risk-minimizing SOC problems whose associated HJB equation is linearizable via the Cole-Hopf transformation. Using a 2D robot navigation example, we validate the proposed control synthesis framework and compare the solutions obtained using path integral and FDM. 

We present an analytical method to estimate the continuous-time collision probability of motion plans for autonomous agents with linear controlled Ito dynamics. Motion plans generated by planning algorithms cannot be perfectly executed by autonomous agents in reality due to the inherent uncertainties in the real world. Estimating end-to-end risk is crucial to characterize the safety of trajectories and plan risk optimal trajectories. In this work, we derive upper bounds for the continuous-time risk in stochastic robot navigation using the properties of Brownian motion as well as Boole and Hunter's inequalities from probability theory. We numerically demonstrate that our method is considerably faster than the naive Monte Carlo sampling method and the proposed bounds perform better than the discrete-time risk bounds.

Motion plans generated by planning algorithms cannot be perfectly executed by autonomous agents in reality due to the inherent uncertainties in the real world. Assuming a linear feedback controller is used for trajectory tracking, we present novel upper and lower bounds to estimate the probability of collision with obstacles during the trajectory tracking phase. Our approach is an application of standard results in probability theory including inequalities of Hunter, Kounias, Frechet, and Dawson. Using sample trajectories generated in an example configuration space for a ground robot, we numerically demonstrate that the proposed bounds are significantly less conservative than the Boole’s bound commonly used in the literature.   

We developed a humanoid torso to pick and place objects by obtaining visual and audio information from its surrounding via an input device Microsoft Kinect. The robot arm has 5 DOF and a multi-finger adaptive gripper. We established a real-time control of the arm to attain the required position and orientation of the end-effector.

In this project, we use reinforcement learning to solve the chance-constraint motion planning problem. Chance-constrained motion planning is a method to synthesize an optimal path in the presence of a noisy environment and robot dynamics, that satisfies a user-specified threshold of failure probability. Failure occurs when the robot hits the obstacles in the domain or the boundary of the domain. Through Lagrangian relaxation, we convert the chance-constrained (risk-constrained) motion problem to a risk-minimizing problem. We show that by choosing an appropriate Lagrange multiplier, we can synthesize a policy which has an appropriate level of safety.

We developed an AI chatbot that serves as a tutor for the programming language- Python. It takes the question from the user in the form of voice or text and provides an answer in both voice and text formats. The chatbot is built by fine-tuning the Open AI's GPT-2 model via a dataset that we created using transcripts of several youtube videos on Python. 

We developed a motion planning algorithm in continuous-space by combining formal methods and a sampling-based planning method Probabilistic Roadmaps (PRM) to find infinite paths satisfying Linear Temporal Logic formulae. One of the attractions of the formal methods is its ability to express complex specifications using Boolean, temporal operators, and atomic propositions. Using a PRM-based algorithm we abstract the given continuous-space environment into a finite transition system and use model checkers to generate satisfying trajectories. One of the features of our algorithm is that it is probabilistically complete. 

We developed an interior-point optimization algorithm for efficient collision detection to speed up motion planning in stochastic environments. This algorithm is significantly faster than the off-the-shelf SemiDefinite Programming (SDP) solvers like sdpt3.

We developed a structured light based 3D reconstruction algorithm and tested it on a variety of 3D objects using different binary-coded light patterns. This algorithm can also generate a parametric surface of a 3D object directly from its images. We used this algorithm for the construction of analytical surfaces (e.g. NURBS), texture mapping, and object detection. We also developed an algorithm to fit B-spline surfaces and curves through the reconstructed point cloud of a surface for computing its intrinsic properties such as principal curvatures, principal directions, and normals.

We developed an interface module to use Novint Falcon as an input tool and haptic interface for controlling the system of motors of a needle manipulator (two DC motors-one for needle insertion and the other for needle rotation, and two servo motors for needle-tip bending), developed for minimally invasive surgery. 

We employed an approximate gravity balancing technique using non-zero free length springs, without additional masses or auxiliary links to reduce the actuator torque requirement of a swimming pool lift mechanism developed for physically challenged people. We obtained the optimal spring parameters and spring pivot points of coil springs and gas springs by minimizing the potential energy variance over the configuration space and achieved approximately 80% actuator torque reduction. 

We built a biarticular active ankle prosthesis which not only provides an ankle torque similar to the biological ankle like the current ankle prostheses but also takes into account the musculoskeletal structure of the biological leg; this could help reduce gait asymmetry, joint reaction forces and provide better rehabilitation of the transtibial amputees as compared to the current ankle prostheses. The design mimics the uniarticular Soleus and Tibialis Anterior muscles using a series elastic actuator (SEA) together with a parallel spring and the biarticular Gastrocnemius muscle using another SEA.