The Curious Footballer

Can we get a humanoid to kick a ball to a target?

We'll only provide normal regularization rewards and a dense reward based on ball-target distance.
The problem is hard despite the dense reward here, as the dense reward itself is only activated after a sparse event of interacting with the ball.
The agents has to itself figure out through exploration the sequence of actions to start interacting with the ball.

With traditional RL, due to the high number of dimentions, random exploration fails to even find the ball.

We introduce an intrinsice curiosity rewards based on RND (Random Network Distillation).
With this, the agents is able to find and hit the ball, but not towards the target.

The reason for this is that the curiosity rewards's magnitude overshadowed the task reward.
Hence the task reward did not provide any meaningfull feedback even after hitting the ball, and just curiosity dominated.

However after tuning the curiosity factor well and linking it magnitude based on the task-based reward,
we finally get an agent that can smothly kick the ball to the target.