Implementation

The Approach

Analyzing terrains present in the environment and navigating was done using the following pipeline:

Take images using the Kinect camera as the robot moves across the environment.
Extract features from a homography transform of the visual input, and update the feature and image map.
Estimate the dynamics of the present location with consideration for both aleatoric and epistemic uncertainty based on robot movement.
Plan the optimal path with Dijkstra’s algorithm considering the dynamics. Less navigable terrains would have higher edge weights.
Execute the plan with a PD controller.

Feature grid creation

We initially wanted to map out our terrain using depth maps and point clouds generated by the Kinect mounted on the Turtlebot. While this produced more accurate results in terms of obstacle information, it was very prone to noise, and reading the data took up to 10 seconds with the computational capability available, an unrealistic amount of time for real-time computation.

We decided to switch to homography and occupancy grids instead. This involves a couple different transformations. We first want to warp the image taken by the onboard camera to an overhead perspective to fit it into the world grid. This is accomplished using a homography transform, and an example is shown below. The calibrated homography matrix, calculated using example images and the position of the camera, makes the image appear as though it was taken by an overhead camera.

Coordinates overlaid on camera image.

Image transformed via homography with recalculated coordinates.

This straightened picture is then positioned in the world grid in the appropriate location. The Turtlebot publishes its position through ROS, allowing us to locate the appropriate box to modify. The map is broken down into grid cells for path planning and featurization. For each region in the image captured, some data for terrain identification, including visual values like intensity and color, are calculated and recorded in a corresponding feature grid. This is constantly updated. The full pipeline is shown below.

Identify floor corners so as to avoid performing homography on walls

Transform to locations on global map based on robot odometry

Project via homography

Update terrain grid with image and featurization

dynamics estimation

We use kernel regression, specifically the Nadaraya-Watson estimator, for learning the dynamics of terrains given visual features associated with them. Kernel regression is a non-parametric technique to fit nonlinear functions to data, and we adapt it to also provide us with uncertainty information. Intuitively, this estimator uses a weighted combination of nearby datapoints (according to some function K) to inform the prediction at a test terrain vector.

Path planning

When executing Dijkstra’s and planning to a particular positions on the grid, the terrains are taken into account. Regions on the grid have edge weights inversely proportional to the speed scaling factor, so areas that are less drivable are planned around. We also sometimes choose to discount further based on aleatoric uncertainty.

Experimental setup

Sim Experiments

There are 2 main methods of our design demonstrated in simulation: the planning algorithm and the resulting state estimation. These work in tandem. The first experimental setup consists of a 10 x 10 grid that includes mock dynamics realized with k and d scaling values randomly sampled between 0.2 and 1. These mock dynamics are a simulation of what the feature extraction from the visual input would give to the planner. Each grid cell has a unique one-hot encoded terrain vector. The planner is then commanded to navigate from (1,1) to (9,9) and back repeatedly until it converges on its conclusion for the optimal path based on the dynamics of the environment. The second simulation experiment instead randomly samples the k and d values from a set number of values (10-20) between 0.2 and 1. The planner is commanded to navigate to the same goal.

Physical Experiments

We used the Turtlebot for physical experiments. It has several useful devices onboard that allow us to collect the data we need to perform TASI. These include data collectors used for mapping, like a depth sensor and an RGB camera. Additionally, it has built-in odometry, allowing us to match an image taken from the local frame to the global frame. We then attempted multiple different forms of manual terrain creation to test our featurization capabilities.

This terrain creation took the form of arranging various materials in the robotics lab to form more difficult paths for the Turtlebot to cross. We started with a smooth board followed by a bumpy set of Lego blocks. The torque in the wheels coupled by the surprisingly strong suspension allowed the robot to cross these obstacles quite easily, so we also ran a trial using our own strength to hold the bot back, representing a less drivable region.

Software used

ROS
Numpy
Turtlebot controller
Turtlebot simulator
Kinect camera

Page updated

Report abuse