Performance of our method on four benchmark environments. Higher reward ratios correspond to better performing policies. Lower JS divergence corresponds to more precise measure reconstruction.