2013.08.21 - Stefan Schaal: From Learning Movement Primitives to Associative Skill Memories

Website: http://www-clmc.usc.edu/~sschaal/

People:

• Auke Ijspeert
• Jan Peters
• Evangelos Theodorou (GT AE) - path integrals http://homes.cs.washington.edu/~etheodor/
• Learning, perception and action
  • Machine learning
  • Neuroscience
  • Robots
• Autonomous robots - how do we make things that work? (Robotic systems, theory/algorithms, neuro studies)
  • Modular planning
  • Teaching and imitation
  • Trial and error learning
  • Interaction with the environment
  • Learning control
  • Multi-modal perception
• Movement primitives
  • Control policy
  • Dynamic movement primitives are solved by producing attractor/repeller dynamics
• Trial error (reinforcement learning)
  • Algorithm
    • Teach with imitation
    • Provide an objective function
    • Perform trial and error to explore space
  • Reinforcement learning
    • Policy
    • Reward function
    • Value function
    • Model
  • What do people do?
    • State-based
    • Trajectory-based
      • Scalable to high-dimensions
  • Path-integral Reinforcement Learning
    • Take affine system dynamics
    • Have stochasitc cost function
    • Try and solve stochastic HJB
      • Use magic log-transform trick
        • Squash reward into weight from zero-to-one, apply weight
      • Apply feynman-kac theorem (allows empirical evaluation)
      • Generate optimal control policy
  • Adapting to dynamically changing world
    • Hoffmann ICRA 2009
    • Pastor IROS 2011
• Learning control with internal models
  • Compliant control
• Associative skill memories
  • Action is associated with all perceptual information
    • Success vs failure detection based on associating sensory info to actions that allow prediction of next action