Papers
The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints — IROS 2019
(paper) The CoSTAR Dataset is introduced and the impact of changes in network architecture is demonstrated empirically via real robot data.
Abstract— A robot can now grasp an object more effectively than ever before, but once it has the object what happens next? We show that a mild relaxation of the task and workspace constraints implicit in existing object grasping datasets can cause neural network based grasping algorithms to fail on even a simple block stacking task when executed under more realistic circumstances.
To address this, we introduce the JHU CoSTAR Block Stacking Dataset (BSD), where a robot interacts with 5.1 cm colored blocks to complete an order-fulfillment style block stacking task. It contains dynamic scenes and real time-series data in a less constrained environment than comparable datasets. There are nearly 12,000 stacking attempts and over 2 million frames of real data. We discuss the ways in which this dataset provides a valuable resource for a broad range of other topics of investigation.
We find that hand-designed neural networks that work on prior datasets do not generalize to this task. Thus, to establish a baseline for this dataset, we demonstrate an automated search of neural network based models using a novel multiple-input HyperTree MetaModel, and find a final model which makes reasonable 3D pose predictions for grasping and stacking on our dataset.
The CoSTAR BSD, code, and instructions are available at github.com/jhu-lcsr/costar_plan.
@article{hundt2019costar, title={The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints}, author={Andrew Hundt and Varun Jain and Chia-Hung Lin and Chris Paxton and Gregory D. Hager}, journal = {Intelligent Robots and Systems (IROS), 2019 IEEE International Conference on}, year = 2019, url = {https://arxiv.org/abs/1810.11714}}CoSTAR References
Below are the key papers underlying CoSTAR: the Collaborative System for Task Automation and Recognition, the system used for data collection (github).
User Experience of the CoSTAR System for Instruction of Collaborative Robots
Abstract— How can we enable users to create effective, perception-driven task plans for collaborative robots? We conducted a 35-person user study with the Behavior Tree-based CoSTAR system to determine which strategies for end user creation of generalizable robot task plans are most usable and effective. CoSTAR allows domain experts to author complex, perceptually grounded task plans for collaborative robots. As a part of CoSTAR’s wide range of capabilities, it allows users to specify SmartMoves: abstract goals such as “pick up component A from the right side of the table.” Users were asked to perform pick-and-place assembly tasks with either SmartMoves or one of three simpler baseline versions of CoSTAR. Overall, participants found CoSTAR to be highly usable, with an average System Usability Scale score of 73.4 out of 100. SmartMove also helped users perform tasks faster and more effectively; all SmartMove users completed the first two tasks, while not all users completed the tasks using the other strategies. SmartMove users showed better performance for incorporating perception across all three tasks.
@article{paxton2018evaluating, title={Evaluating Methods for End-User Creation of Robot Task Plans}, author={Chris Paxton and Felix Jonathan and Andrew Hundt and Bilge Mutlu and Gregory D. Hager}, journal={Intelligent Robots and Systems (IROS), 2018 IEEE International Conference on}, year={2018}, url={https://arxiv.org/abs/1811.02690}}CoSTAR: Instructing Collaborative Robots with Behavior Trees and Vision
Abstract—For collaborative robots to become useful, end users who are not robotics experts must be able to instruct them to perform a variety of tasks. With this goal in mind, we developed a system for end-user creation of robust task plans with a broad range of capabilities. CoSTAR: the Collaborative System for Task Automation and Recognition is our winning entry in the 2016 KUKA Innovation Award competition at the Hannover Messe trade show, which this year focused on Flexible Manufacturing. CoSTAR is unique in how it creates natural abstractions that use perception to represent the world in a way users can both understand and utilize to author capable and robust task plans. Our Behavior Tree-based task editor integrates high-level information from known object segmentation and pose estimation with spatial reasoning and robot actions to create robust task plans. We describe the cross- platform design and implementation of this system on multiple industrial robots and evaluate its suitability for a wide variety of use cases.
@article{paxton2017costar, title={Co{STAR}: Instructing Collaborative Robots with Behavior Trees and Vision}, author={Paxton, Chris and Hundt, Andrew and Jonathan, Felix and Guerin, Kelleher and Hager, Gregory D}, journal={Robotics and Automation (ICRA), 2017 IEEE International Conference on}, note={Available as arXiv preprint arXiv:1611.06145}, year={2017}, url={https://arxiv.org/abs/1611.06145}}If you use the CoSTAR Dataset for a paper contact us and we will add your work to this page!