OCS Project Conclusion
OCS Project Conclusion
Classification, Rejection, and Sorting
Classification, Rejection, and Sorting
Embedded Files
Why it is needed?
OCS identifies defects that would otherwise be difficult to detect with human eyes. This will save a company money and time. For example, most Food and Beverage companies incur a huge expense from recalls of bad products. However, these defects can be identified during inspection by using the OCS system.
What differentiates this product?
OCS uniquely classifies and sorts known objects at an accuracy over 99%. This confidence level makes the OCS system top of the line and it is cost effective. As it is shown in figure 1-5 below, one of the units under test is classified at 99.97% accuracy. This level of classification accuracy is very crucial for industrial lean manufacturing processes.
Technical Objectives
- Identify trained objects with greater than 99% confidence.
- Establish communication between Matlab and Arduino micro controllers.
- Sort trained objects into designated containers with a robotic arm
- Reject untrained objects into separate container with lever arm.
Functionality and Use Cases
This prototype is a proof of concept for computer vision-based object rejection and sorting systems. The prototype itself does not have any practical use cases. However, it would be a useful tool for people trying to sell companies on the prospect of automating processes.
Key Realistic Constraints (for demonstrated prototype)
Social and Ethical:
· No exposed wires
· Function autonomously
Manufacturability:
· Assembled and disassembled within 30 minutes
· RF, ISO C++, USB, Bluetooth, DC Power Supply
· Arm and table fixed relative orientation
· Items cannot exceed 2 lbs or a width of 5 inches.
· Rejection table requires a 12V battery pack and robotic arm requires a 120V AC outlet.
Environmental:
· Batteries shall properly disposed
Economic:
· Less than $300
· Rechargeable Batteries
Health and Safety:
· High current components properly shielded
Standards Usage (for demonstrated prototype)
OCS background
Deep Learning is a branch of machine learning under the artificial intelligence umbrella that enables computers to learn in a manner that is much similar to humans, by experience. Data is the main driving force behind how a Deep Learning network learns, and it’s directly correlated to how well it performs at its given tasks. Deep Learning’s ability to make significant use of pre-processed data has extended its applications to platforms that vary from handheld devices’ ability to recognize what you look or sound like, to enabling NASA’s Mars Rover to be able to “see” where it’s navigating through with computer vision.
A Deep Learning network’s power lies in its ability to do whatever a mathematical model can do, without requiring one to know which specific signal features to incorporate into the model. The network itself decides which features to focus and learn from in the training process. This allows non-specialists to yield accurate results very quickly. So accurate in fact, that as displayed on figure 1.1, Deep Learning methods are now more accurate than people at classifying images. That, along with MATHWORK's extensive documentations and toolboxes for Deep Learning, have enabled our implementation of an object classification and sorting algorithm (OCS).
Our object classification and sorting system’s universality allows for multiple use cases best tailored for companies as opposed to the general consumer. The modern sorting and rejection process still rely heavily on human labor. Computer vision tasks have historically been very difficult to automate, but with the advent of deep learning, these tasks are becoming easier and easier to automate. The figure below shows how object classification and sorting algorithms can be used to sort and stock items for general stores or be used to identify defective parts in production lines. This product can be used by manufacturing industries to improve the efficiency of all sorting and rejection process. To make it easy to digest the ubiquitous applications of a classification and sorting algorithm. Our network was trained to focus on three specific objects to demonstrate a potential functionality for recyclable waste and other similar object detection, classification, and sorting problems.
OCS systems can replace human sorting at recycling centers. Although there will still be a need for people monitoring and maintaining the OCS system, the need for humans to do the actual sorting of recycled waste can be reduced significantly.
Image courtesy of greenandgrowing.org.
OCS systems can identify defective products. Automatic detection and rejection of defective products can lead to greater customer satisfaction and improved product quality.
Image courtesy of prweb.com.
The modern sorting and rejection process still rely heavily on human labor. Computer vision tasks have historically been very difficult to automate, but with the advent of deep learning these tasks are becoming easier and easier to automate.
Image courtesy of mpcmacs.com.au
1COGNEX VIDI: It is the first deep learning-based software designed to solve complicated applications for factory automation.
Image courtesy of: cognex.com
Our team approached the given problem in two phases. Phase 1 utilizes Matlab deep learning techniques by using a transfer learning to fine-tune the expertly trained network AlexNet for our specific initiative: classify and sort three different recyclable objects with a confidence score of 99% or higher.
In phase 2, our team uses a robotic arm to perform the sorting process and a rejection mechanism to reject unknown objects. The whole OCS System is composed of mechanisms that implement image acquisition, classification, sorting, and rejection.
OCS Process Flow
The image is captured using the web camera and pre-processed to go through our Convolutional Neural Network, titled OCS. Then, the classification process follows. It determines if the object placed on our platform is one of the three objects it was trained for. In the case that the OCS network has 99% confidence or greater, the robotic arm will place the item in its allocated container. In the case that the item isn’t one of our three objects, a lever rejects the item off the platform to set space for another object to be classified.
Project Test Results
- OCS final Transfer Learning is performed using AlexNet on a single GPU and we have obtained a 100% learning validation accuracy
- The OCS deep learning algorithm communicates with two Arduino's to either reject or sort a given object.
- If the object is not known by the OCS CNN, it will be rejected by the rejection mechanism.
- If the object is known, it will be sorted by Brracio robotic arm.
- Communication between the laptop and the Sorting mechanism is achieved by using APC 200 RF communication modules.
- Communication between the laptop and the rejection mechanism is achieved by using the Bluetooth modules.
- The acceptance test is performed by inspection, analysis, demonstration, and measurements. Most of the OCS acceptance test is done by live demonstrating as the project is suitable for this type of test.
- All requirements have been tested and verified.
- Our system satisfies all our system requirements.
Lessons learned
The takeaway from the first phase of this project is, while performing transfer learning from existing CNN, instead of using the lower end CNNs like AlexNet, using the midrange CNN like RasNet-101 or VGG-16 could be a better option. However, there is a huge tradeoff like hard disk space to download these CNNs. Another limitation while trying to perform a transfer learning is prediction time on a CPU is extremely high. OCS team is able to perform the transfer learning using AlexNet on a single GPU and it took about 67 minutes. Therefore, it would have been better if we have multiple GPUs and if we could use a deeper CNN like RasNet-101. However, the selected CNN and the single GPU are satisfactory for demonstration purposes.
For follow-on projects
It will be much better to make sure the computer performing the transfer learning has at least on a built-in single GPU. If this task is performed in a CPU, this will be time consuming and frustrating. Having a fast GPU is a very important aspect of this project. Deep learning requires an exceptional computational power as it involves in operations like matrix multiplication and other complex operations.
Furthermore, the rejection mechanism can be improved to a conveyor belt. This way the system will avoid the human interaction to place the objects in front of the camera. Conveyor belt can be controlled by adding an ultrasonic sensor to stop the motion when the object is in front of the camera. This way the object under classification can be positioned in fixed and accurate position.
Significant changes to concept due to prototype issues, testing and lessons learned
During a functional test phase of the OCS system, we came across to an issue that the Braccio robotic arm has a load capacity limitation. According to the Braccio robotic arm datasheet, the maximum operating weight at 32 cm distance is only 150 gm. However, during the sorting functional test, the robotic arm has struggled to pick and place one of the units under test, the “New York Soda” bottle that weighs 145 gm. The robotic arm can pick and place the bottle, but the servo motors were struggling to provide the required torque. For this reason, we needed to replace the “New York Soda” bottle with a lightweight glass bottle. This requires creating a new datastore of images and re-training the OCS CNN. Therefore, before creating the image "datastore", it will be wise to verify the load limitation of the manipulator. This will save you lots of time.
Many thanks to our project advisor, Dr. Haggerty and the Mathworks company.
Google Sites
Report abuse