Learning to Rearrange with Physics-Inspired Risk Awareness