Planning in Latent Space

One issue with most of the learned dynamics models is that they are deterministic, or one-step, or both. Of course the most successful approaches have used stochastic, probabilistic, and/or multi-step models but in general how to learn these models is an open question. If your model is deterministic, then if your prediction is off, and you then take that to be true next state and do another step of prediction into the future, then your error is likely to be even worse. In this way, small modeling errors can be exacerbated as planning rolls them forward.

This second issue is less obvious and to me this is the most interesting part. Imagine a robot manipulating a piece of cloth, and let's say we (arbitrarily) pick the latent representation of our cloth to have 16 dimensions. The robot grippers probably together have a combined 12 degrees of freedom. See why this might be a problem? If we are at some point in space, and our heuristic tells us we need to move in some direction to decrease cost, then because we have fewer controls than state dimensions we might not be able to move in that direction.

Here's a simple example with a two-link rope. How can we move the end of the rope (green dot) to the goal? Because the rope is soft, you can't actually directly move it left or right at all. The gray shaded region might be the directions in which the green point can move given that you're pulling at the other end of the rope. This means the system is underactuated.

We can also say this the system is non-holonomic, in this case for the same reason that it is underactuated although that's not true in general. The takeaway here is that in general, latent models are likely to be underactuated and may also be non-holonomic.

Non-holonomic motion planning is its own sub-field with plenty of active research, which is why I list it as one of main reasons I have not seen any papers which use traditional motion planning techniques with learned latent space models.

Finally, there is a third issue that. It may not be as deep and theoretical as the others, but in practice is it very annoying. Anyone who is familiar with any kind of planning algorithm will know that you always need to set some parameters like step sizes and time steps. For discrete search, you might choose a step size in position (5cm) or a bound on your environment (10 by 10 meters). However, when you are planning in a latent space it is unclear how one should set these values. The naive approach would be to take two points in the observation space which you believe to be "1 unit" apart, map them to the latent space, and look at the Euclidean distance between the vectors. But does this actually make sense? Why Euclidean? How do you pick which specific observations to use? I don't have an answer to this.