Real-time barcode detection in video with Python and OpenCV
In Previous blog of guide of barcode detection we explored how to detect and find barcodes in images. But now we are going to extend our barcode detection algorithm and refactor the code to detect barcodes in video.
So here’s the game plan.
Our barcode detection in video system can be broken into two components:
Component #1: A module that handles detecting barcodes in images (or in this case, frames of a video) Luckily, we already have this (Part II of this guide). We’ll just clean the code up a bit and reformat it to our purposes.
Component #2: A driver program that obtains access to a video feed and runs the barcode detection module. We’ll go ahead and start with the first component, a module to detect barcodes in single frames of a video.
Component 1: Barcode detection in frames of a video
I’m not going to do a complete and exhaustive code review of this component, that was handled in Part I of this guide.
However, I will provide a quick review for the sake of completeness (and review a few minor updates). Open up a new file, name it simple_barcode_detection.py , and let’s get coding:
# import the necessary packages
import numpy as np
import cv2
def detect(image):
# convert the image to grayscale
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# compute the Scharr gradient magnitude representation of the images
# in both the x and y direction
gradX = cv2.Sobel(gray, ddepth = cv2.cv.CV_32F, dx = 1, dy = 0, ksize = -1)
gradY = cv2.Sobel(gray, ddepth = cv2.cv.CV_32F, dx = 0, dy = 1, ksize = -1)
# subtract the y-gradient from the x-gradient
gradient = cv2.subtract(gradX, gradY)
gradient = cv2.convertScaleAbs(gradient)
# blur and threshold the image
blurred = cv2.blur(gradient, (9, 9))
(_, thresh) = cv2.threshold(blurred, 225, 255, cv2.THRESH_BINARY)
# construct a closing kernel and apply it to the thresholded image
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (21, 7))
closed = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)
# perform a series of erosions and dilations
closed = cv2.erode(closed, None, iterations = 4)
closed = cv2.dilate(closed, None, iterations = 4)
# find the contours in the thresholded image
(cnts, _) = cv2.findContours(closed.copy(), cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
# if no contours were found, return None
if len(cnts) == 0:
return None
# otherwise, sort the contours by area and compute the rotated
# bounding box of the largest contour
c = sorted(cnts, key = cv2.contourArea, reverse = True)[0]
rect = cv2.minAreaRect(c)
box = np.int0(cv2.cv.BoxPoints(rect))
# return the bounding box of the barcode
return box
If you read Part I of this guide then this code should look extremely familiar.
The first thing we’ll do is import the packages we’ll need — NumPy for numeric processing and cv2 for our OpenCV bindings.
From there we define our detect function on Line 5. This function takes a single argument, the image (or frame of a video) that we want to detect a barcode in.
Line 7 converts our image to grayscale, while Lines 9-16 find regions of the image that have high horizontal gradients and low vertical gradients.
We then blur and threshold the image on Lines 19 and 20 so we can apply morphological operations to the image on Lines 23-28. These morphological operations are used to reveal the rectangular region of the barcode and ignore the rest of the contents of the image.
Now that we know the rectangular region of the barcode, we find its contour (or simply, its “outline”) on Lines 30 and 31.
If no outline can be found, then we make the assumption that there is no barcode in the image (Lines 35 and 36). However, if we do find contours in the image, then we sort the contours by their area on Line 40 (where the contours with the largest area appear at the front of the list). Again, we are making the assumption that the contour with the largest area is the barcoded region of the frame. Finally, we take the contour and compute its bounding box (Lines 41 and 42). This will give us the (x, y) coordinates of the barcoded region, which is returned to the calling function on Line 45.
Now that our simple barcode detector is finished, let’s move on to Component #2, the driver that glues everything together.
Component #2: Accessing our camera to detect barcodes in video
Let’s move on to building the driver to detect barcodes in video. Open up a new file, name it detect_barcode.py , and let’s create the second component:
# import the necessary packages
import simple_barcode_detection
import argparse
import cv2
# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-v", "--video", help = "path to the (optional) video file")
args = vars(ap.parse_args())
# if the video path was not supplied, grab the reference to the
# camera if not
args.get("video", False):
camera = cv2.VideoCapture(0)
# otherwise, load the video
else: camera = cv2.VideoCapture(args["video"])
Again, we’ll start our by importing the packages we need. I’ve placed our simple_barcode_detection function in the module for organizational purposes. Then, we import argparse for parsing command line arguments and cv2 for our OpenCV bindings.
Lines 6-9 handle parsing our command line arguments. We’ll need a single (optional) switch, -- video , which is the path to the video file on desk that contains the barcodes we want to detect. Note: This switch is useful for running the example videos provided in the source code for this blog post. By omitting this switch you will be able to utilize the webcam of your laptop or desktop. Lines 13-18 handle grabbing a reference to our camera feed. If we did not supply a --video switch, then we grab a handle to our webcam on Lines 13 and 14. However, if we did provide a path to a video file, then the file reference is grabbed on Lines 17 and 18.
Now that the setup is done, we can move on to applying our actual barcode detection module:
# keep looping over the frames
while True:
# grab the current frame
(grabbed, frame) = camera.read()
# check to see if we have reached the end of the
# video
if not grabbed:
break
# detect the barcode in the image
box = simple_barcode_detection.detect(frame)
# if a barcode was found, draw a bounding box on the frame
cv2.drawContours(frame, [box], -1, (0, 255, 0), 2)
# show the frame and record if the user presses a key
cv2.imshow("Frame", frame)
key = cv2.waitKey(1) & 0xFF
# if the 'q' key is pressed, stop the loop
if key == ord("q"):
break
# cleanup the camera and close any open windows
camera.release()
cv2.destroyAllWindows()
On Line 21 we start looping over the frames of our video — this loop will continue to run until (1) the video runs out of frames or (2) we press the q key on our keyboard and break from the loop.
We query our camera on Line 23, which returns a 2-tuple. This 2-tuple contains a boolean, grabbed , which indicates if the frame was successfully grabbed from the video file/webcam, and frame , which is the actual frame from the video.
If the frame was not successfully grabbed (such as when we reach the end of the video file), we break from the loop on Lines 27 and 28.
Now that we have our frame, we can utilize our barcode detection module to detect a barcode in it — this handled on Line 31 and our bounding box is returned to us.
We draw our resulting bounding box around the barcoded region on Line 34 and display our frame to our screen on Lines 37 and 38.
Finally, Lines 41 and 42 handle breaking from our loop if the q key is pressed on our keyboard while Lines 45 and 46 cleanup pointers to our camera object.
So as you can see, there isn’t much to our driver script!
Let’s put this code into action and look at some results
Successful barcode detections in video
Let’s try some some examples.
Open up a terminal and issue the following command:
$ python detect_barcode.py --video video/video_games.mov
The video at the top of this post demonstrates the output of our script. And below is a screenshot for each of the three successful barcode detections on the video games:
Let’s see if we can detect barcodes on a clothing coupon:
$ python detect_barcode.py --video video/coupon.mov
Here’s an example screenshot from the video stream:
Summary
In this blog post we built upon our previous codebase to detect barcodes in images. We extended our code into two components:
A component to detect barcodes in individual frames of a video.
And a “driver” component that accesses the video feed of our camera or video file.
We then applied our simple barcode detector to detect barcodes in video.
However, our approach does make some assumptions:
The first assumption is that we have a static camera view that is “looking down” on the barcode at a 90-degree angle.
And the second assumption is that we have a “close up” view of the barcode without other interfering objects or noise in the view of the frame.
In practice, these assumptions may or may-not be guaranteeable. It all depends on the application you are developing!
At the very least I hope that this article was able to demonstrate to you some of the basics of image processing, and how these image processing techniques can be leveraged to build a simple barcode detector in video using Python and OpenCV.