Characterizing what types of exoskeleton gaits are comfortable for users, and understanding the science of walking more generally, require recovering a user's utility landscape. Learning these landscapes is challenging, as walking trajectories are defined by numerous gait parameters, data collection from human trials is expensive, and user safety and comfort must be ensured. This work proposes the Region of Interest Active Learning (ROIAL) framework, which actively learns each user's underlying utility function over a region of interest that ensures safety and comfort. ROIAL learns from ordinal and preference feedback, which are more reliable feedback mechanisms than absolute numerical scores. The algorithm's performance is evaluated both in simulation and experimentally for three non-disabled subjects walking inside of a lower-body exoskeleton. ROIAL learns Bayesian posteriors that predict each exoskeleton user's utility landscape across four exoskeleton gait parameters. The algorithm discovers both commonalities and discrepancies across users' gait preferences and identifies the gait parameters that most influenced user feedback. These results demonstrate the feasibility of recovering gait utility landscapes from limited human trials.

The presented algorithm, termed Region of Interest Active Learning (ROIAL), aims to characterize exoskeleton users' preferences over walking trajectories by learning the users' underlying gait preference landscapes. However, ROIAL does not want to give the user many gaits that make the user feel unsafe or uncomfortable. To this end, ROIAL learns to avoid sampling actions that lie within a "Region of Avoidance" (ROA). This changes the algorithm's goal to only learning the underlying preference landscape across the compliment of the ROA, termed the "Region of Interest" (ROI). This goal can be more formally defined as minimizing the error between the true underlying preference landscape and the learned model posterior, evaluated only over actions within the true ROI. This goal is accomplished by leveraging two forms of online user feedback: pairwise preferences and ordinal labels.

We validate ROIAL's performance on the Hartmann3 (H3) function as well as randomly generated synthetic 3D functions. Through these simulations we evaluated three different aspects of the learning process: the relationship between the number of points included in the posterior update during each iteration (subset size) and the prediction error, the ability of ROIAL to avoid actions in the ROA, and the effect of noisy feedback on the prediction error.

Hartmann3 function prediction error ( leftmost two figures) and synthetic function prediction error (rightmost two figures)