Baxter the Card Playing Bot

Introduction

Goal

We will program Baxter to be able to play card games with a human by using computer vision to locate the cards in 3D space and manipulating the cards with Baxter’s arms.

We'd like to deepen our understanding of computer vision, planning, and actuation by programming the Baxter robot. We seek to identify optimal control strategies for thin, planar objects and the application of modern computer vision techniques in the localization of real-world objects.

Why?

We chose to pursue this project because gameplay is a fun activity to pursue. AI exists for various games on the software level, but playing a physical game with a robot is a fairly novel idea. Interacting with robots in this way provides a unique and amusing experience to players, and is quite fun to work on as well!

Challenges

The main challenges and difficulties in this project are tracking the state of the game through computer vision and manipulation of thin, planar objects. Without a depth camera, we will need to perform calculations to allow the robot to convert the 2D coordinates it receives from the camera frame to 3D coordinates it can plan a path toward.