The plan of dexterous Manipulation is a challenge task. There are several philosophies to solve this complex plan task. One of them is to learn dexterous manipulation task plan. This plan can be implemented in symbol level[1] and trajectory level[2].
The philosophy depends on the data sampling and learning during the course of dexterous manipulation. The user which has no/little experience on robot can demonstrate task by his own hand(usually wearing data glove, tactile sensor etc. measurement device), then such measurement data will be postprocessed and identified the intention of user. The intention will serve as the high task to command the robot hand. Usually the robot hand need plan the low level action(trajectory level/joint level) based on the high level task.
Regards the low(trajectory) level learning, its nature is learning a mapping from data glove joint angle feedback to the real robot hand joint feedback. This kind of learning can resort to the robot vision feedback to implement. With this method human hand plan on dexterous manipulation can be
intuitively transfered to the robot hand.
Reference:
[1] Symbolic level generalization of in-hand manipulation tasks from human demonstrations using tactile data information
[2] dexterous skill transfer by externding human body schema