The entire system is driven by the MAX78000FTHR development board which contains the MAX78000 microcontroller with an on chip CNN accelerator and an integrated camera module. The microcontroller is in charge of the computer vision task for object classification as well as the control logic for sorting (robot arm, lever arm motors, conveyor belt). It has support for the standard peripheral interfaces (UART, SPI, I2C) and PWM and also has support for reading and writing to an SD card.

[Datasheet] [User Guide]

We are collecting our own dataset using the MAX78000FTHR’s camera module. Rather than transmitting the images to a PC over USB, we store them on a 16GB SD card connected to the board. This makes the dataset collection system more portable. In addition, this system is simpler to implement since we don’t need to write a separate program on the PC side to communicate with the MAX78000.

[Product Link]

The LCD is used for dataset collection and debugging purposes by displaying the live camera feed. Its dimensions are 320x240 and it uses the SPI interface at 50 MHz. In general there is a non-negligible latency with displaying images with large resolution (i.e. greater than 100x100). To avoid creating a bottleneck on the system we don’t plan to have it as a component in the software control flow as it is not necessary for the system to function. However, it is convenient to be able to see the live camera feed so we will work to integrate it without impacting the overall sorting latency.

[Datasheet] [Product Link]

We have selected five specific object classes for sorting that we believe will ease the classification task as well as physical sorting. The five classes are paper cup, yellow hexagon, red trapezoid, metal can, and plastic bottle. These objects are also light and easy to physically push (see risk analysis section).

[Shapes] [Bottles] [Cups] [Cans]

A NEMA 23 bipolar stepper motor was chosen as the primary actuator in the sorting mechanism to act as a “flipper”. A lightweight rectangle of plastic or sheet metal will be attached to the shaft of the stepper motor. This will act as both a wall to keep the sorting objects inside the conveyor belt and as a lever arm to guide the items into a classification bin. After an item has been classified, the corresponding stepper motor will swing open to a set angle, around 65°, and wait until the item hits the lever arm. The stepper will then swing back to its resting position, pushing the item with it. Stepper motors were chosen because of their relative simplicity, high torque, and low cost. RC servos were not fast enough at the required torque and a pneumatic system seemed to be over complicated, loud, and costly.

[Product Link] [Manufacturer Page] [Datasheet]

In order to control the selected stepper motors, the Pololu High-Power Stepper Motor Driver 36v4 was selected as the motor driver. It uses the TI DRV8711 with external MOSFETs so that there can be up to 4A of continuous current draw and allows for microstepping for smoother movements. This driver will allow for more than enough power for the stepper motors. This driver can also be configured and stepped through with SPI which means that all five stepper motor drivers can be chained and all controlled while using only one SPI port. This is very advantageous compared to individual PWM inputs for each motor input.

[Datasheet] [Product Page] [Chip Datasheet]

In order to mount the MAX board to the conveyor belt, we have chosen to use a gooseneck mount. We have repurposed a phone mount, which has an adjustable clip on the end which will hold the board in place. The body of the gooseneck mount is sturdy enough to hold the board and maintain its position once it has been set. We have already attached the mount to the conveyor belt and positioned the MAX board at approximately 45°.

[Product Link]

The conveyor belt will be our main working area for placing sortable objects, classifying them, and sorting them into their appropriate bins. The speed of the conveyor belt will be controlled by the MAX78000 microcontroller through a PWM interface. The belt is 59” long by 7.8” wide, which nearly maximizes the working space of the robotic arm, and acts as a consistent background for both the dataset and classification.

[Product Link]

In order to detect when an object has been placed onto the conveyor belt, we will be using IR sensors to detect when the MAX78000 board should take photos of the object. This will help the board only run classification on photos that have the object in full-sight. Once the connection between the IR sensors is broken, the MAX78000 board will take a series of photos, start classification, and choose the category that prevails the most.

[Product Link] [Data Sheet]