My Research, for Poets

This post is inspired by the famous "EC2 for Poets" blog post. In this post, I will explain my research at a level that could be understood by anyone. This is not to say that Poets are not smart, they are very smart, but unlikely to be familiar with robotics and AI. Recently, my mom asked me to explain my research and so this post is also something I can point her to. A finally, it's always a good exercise in scientific communication to explain things in simple terms. Insert that famous Richard Feynman quote here...

Let's start with what I want robots to be able to do.

In broadest sense, I'd like to see robots that are competent at general purpose manipulation. I'm not particularly interested in extreme precision or speed, or robustness. Am I also not particularly interested in dexterous manpiulation, which concerns itself with human-like hands. There are tons of interesting things left for robots to do with their clunky grippers. In fact, I'm ok if robots sometimes drop things or knock things over. I think this may even be a necessary trade off with general competence. This is a very lofty goal though, so during my PhD I'm focusing just on deformable object manipulation. This means things like rope and cloth. Deformable object manipulation is much harder than rigid object manipulation, and so towards general competence I'm starting with deformable objects in the hopes that my techniques can then be extended to cover rigid objects. Maybe rigidity can be treated as only-very-slightly-deformable, we'll see!

The main challenges with deformable object manipulation are

• • 1. Accurate real-time perception

    2. Efficient representation

    3. long-horizon prediction

So first, perception. In the traditional framework of robotics, Sense-Think-Act, we first read our sensors to determine what the state of the world is. In deformable object manipulation, that might be something like "where is the cloth and what is its shape". Unfortunately, that problem is very hard. Even the state of the art in cloth and rope tracking is still not good enough. This problem is hard because the sensors that detect the cloth are not accurate, and very often parts of the cloth are blocked from the view of the sensor by the robots hard, or , other objects, or by other parts of the cloth.

The second challenge is a bit harder to define. In the extreme case, imagine that we chose to represent an object by writing down all the position of every one of its molecules. That would be way more information than we need, and certainly trying to predict the future positions of molecules would be incredibly slow and expensive even for supercomputers. On the other extreme end, we could pretend everthing is a sphere, but this would be too inaccurate. So one of the key challenges in manipulation, especially for deformable objects, is how to represent the objects. As a more reasonable example, one method that works well for rope is to pretend it's a line of about 10 or 20 points connected by springs, but of course this doesn't always work and is specific to just rope.

The third challenge I want to highlight has to do with prediction. Prediction is an essential part of planning. If you can't predict what your action will do, it's hard to make the right decisions. The issue is that no one has figured out how to predict what will happen very far into the future. In a lot of methods, the predictions far into the future tend to either degenerate into meaningless states or explode chaotically. For example, if you're consideing rope represented as a set of points, the prediction of 5 seconds in the future may have the rope shrink to a point or too stretched out.