Introduction

Human sight and vision are the most important senses among all the others, we rely on them to see the outside world and comprehend. It is highly capable and has a lot of components involved, thus making it a complicated system [1]. Mimicking this complex skill of humans by the computer and understanding it is the field of computer vision. Object detection is a task in computer vision and image processing that deals with detecting objects in images or videos. With the advent of technology in all sectors, it has found a prominent place in a variety of applications including video surveillance, self-driving cars, object tracking, etc.[2] Object detection allows us to classify the types of things found in the image while also locating instances of them in the image.

I will be making use of this field of Artificial Intelligence in order to implement my capstone project.

Methodology

Detection of objects involves all the steps of a machine learning pipeline. So will discuss them in detail in the below sections

1. Data Source and collection:

In order to implement the project, we have to train the model to detect these specific logos in the given image or video. As the data is scenario-specific, I will be creating custom images set for each of these companies that have the target logos in them.

• Data collection: Scrapping of images with these logos from Google images using selenium.

2. Data augmentation: Augmenting the real data to produce new fake images that can be used for training. (Neural networks perform better when trained on huge data)

3. Data labeling: Annotations are vital in object detection, so I'm planning of using an open-source tool(labelImg) to label the images.

4. Data configuration: Once the training data is ready it has to be converted in the format how the YOLO framework accepts.

   • Train and test config files have to be modified according to the number of classes we have.

   • File structure alignment as per the YOLO framework

5. YOLO Architecture(Darknet): Use the YOLOv3 network architecture(from the original paper written using Darknet neural network)

6. Train: Training the model with the pre-trained weights on our dataset and save the trained weights for every 100 epochs.

7. Test: Predict bounding boxes on the test images and video files.

2. Data augmentation :

Neural networks, in general, are data-hungry that require large data to learn and perform better.

Data Augmentation is the process of applying different transformation techniques like horizontal flip, cropping, shearing on the available data to synthesize new data.

For this, I have applied 3 kinds of transformations on the available data using the skimage package - Colab Notebook

a. horizontal flip (left to right)

b. vertical flip ( up-down)

c. rotation by 45 degrees

After augmentation, we have 1280 images for five classes that can be split to train and validation.

Before we delve into YOLO, let us look at some of the previously used models for object detection.

• Region-based detectors and Deformable parts model were all two-stage detectors where

  • The first step was to find the region of interest in the input image.

  • The second step was to pass those regions to a classifier for detection.

• DPMv5(Deformable parts model) was like a gold standard in object detection taking 14 seconds to process a single image with an mAP of 33.7. This is extremely far from being real-time.

• Then came R-CNN [4](Region-based Convolutional Neural Networks)

• In Fast R-CNN [5] instead of extracting CNN feature vectors for each region proposal, this model passes the entire image through one conv net and the region proposals share this matrix. Then the same feature matrix is branched out to be used for learning the object classifier and the bounding-box regressor. In conclusion, computation sharing speeds up R-CNN.

• Faster R-CNN [6] in which they construct a single, unified model composed of RPN (region proposal network) and fast R-CNN with shared convolutional feature layers.

• Summary of all the R-CNN family models

This model skips the earlier discussed regional proposal stage and directly runs detection on the image using the deep convnets, hence it is faster and simpler. It is a state-of-the-art model that had a major breakthrough in computer vision mainly because of its speed and accuracy.

Unlike other models like Faster-RCNN, this model uses a single complete neural network to detect the objects in the image and as well locate them on the image. In Yolo, each input image is divided into a S x S grid. For each cell in the grid, some bounding box predictions are generated simultaneously with class probabilities/scores for predicting objects associated with that grid cell. Each score reflects how confident the model is that the box contains a certain class of objects. Another biggest advantage is that it can process 45 frames per second.

• Darknet is the name of the underlying architecture of YOLO

• In total, one image contains SxSxB bounding boxes, each box corresponding to 4 location coordinates, 1 confidence score, C conditional probabilities for object classification.

• The total prediction values for one image is S×S×(5B+C), which is the tensor shape of the final conv layer of the model.

• So for a 7x7 cell with 20 classes with 2 bounding boxes for each cell, the output will be 7x7x30 tensor

Here is the workflow of YOLO from the original paper[9]:

YOLO has 24 convolutional layers followed by 2 fully connected layers (FC). Some convolution layers use 1 × 1 reduction layers alternatively to reduce the depth of the features maps. For the last convolution layer, it outputs a tensor with shape (7, 7, 1024). The tensor is then flattened. Using 2 fully connected layers as a form of linear regression, it outputs 7×7×30 parameters and then reshapes to (7, 7, 30), i.e. 2 boundary box predictions per location.

YOLOv2

A number of enhancements were brought in the version 2 model also known as YOLO9000

• BatchNorm function applied on all the convolutional layers, improvement in convergence.

• Made use of high-resolution images for fine-tuning the base model which improves the detection performance

• brought in the convolutional anchor box detection

YOLOV4 [14]

• improved detection performance and superior speed

• Backbone: CSPDarknet53,

• Neck: Spatial Pyramid Pooling additional module, PANet path-aggregation,

• Head: YOLOv3

• It is a proficient and authoritative object detection model that allows individuals with a 1080 Ti or 2080 Ti GPU to training a very fast and accurate object detector.

• The consequences of state-of-the-art “Bag-of-Freebies” and “Bag-of-Specials” object detection procedures all the while detector training was confirmed.

• The converted state-of-the-art methods covering CBN (Cross-iteration batch normalization), PAN (Path aggregation network), that are greater skilled and applicable for single GPU training.

YOLOv4 tiny [14]

• This is a compressed version of the YOLOv4 model.

• It has only two yolo heads layer

• 29 convolutional layers

• reduced number of anchor boxes predictions

5. Train:

I trained the datasets on YOLOv3, YOLOv4, YOLOv3 tiny, and YOLOv4 tiny models.

Each of the models was trained on 4000 epochs.

1. Yolo_V3 - Colab Notebook

trained model weights file - drive.google.com/file/d/1FBFf_6thDNevuc9qzQS_-LU6mWWgjuln/view?usp=sharing

2. Yolo_V3 tiny - Colab Notebook

trained model weights file - drive.google.com/file/d/1MfVpxBrd0GIAaF0Shf-zufVi-g-_8zEp/view?usp=sharing

3. Yolo_V4 - Colab Notebook

trained model weights file - drive.google.com/file/d/16bQwz1DsTbDlfdJMsWnGwGdnkyyWmdB1/view?usp=sharing

4. Yolo_v4 tiny - Colab Notebook

trained model weights file - drive.google.com/file/d/1ArUkn6WvakV_7mdRzVjCw6FAlv9I1-Jw/view?usp=sharing

References

1. Becoming Blind Is a Top 4 Fear – Smart Vision Labs. “Why Vision Is the Most Important Sense Organ.” Smart Vision Labs, 29 Jan. 2017, www.smartvisionlabs.com/blog/why-vision-is-the-most-important-sense-organ/.

2. Brownlee, Jason. “A Gentle Introduction to Computer Vision.” Machine Learning Mastery, 5 July 2019, machinelearningmastery.com/what-is-computer-vision/.

3. Tzutalin. LabelImg. Git code (2015). https://github.com/tzutalin/labelImg

4. Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. “Rich feature hierarchies for accurate object detection and semantic segmentation.” In Proc. IEEE Conf. on computer vision and pattern recognition (CVPR), pp. 580-587. 2014.

5. Ross Girshick. “Fast R-CNN.” In Proc. IEEE Intl. Conf. on computer vision, pp. 1440-1448. 2015.

6. Shaoqing Ren, Kaiming He, Ross Girshick, & Jian Sun. (2016). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.

7. Weng, L. (2017). Object Detection for Dummies Part 3: R-CNN Family. lilianweng.github.io/lil-log.

8. Redmon, Joseph, and Ali Farhadi. “YOLO9000: Better, Faster, Stronger.” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, doi:10.1109/cvpr.2017.690.

9. Redmon, Joseph, et al. “You Only Look Once: Unified, Real-Time Object Detection.” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, doi:10.1109/cvpr.2016.91.

10. Redmon, J. (n.d.). Darknet: Open Source Neural Networks in C. https://pjreddie.com/darknet/.

11. Christiansen, A. (2021, January 4). Anchor Boxes - The key to quality object detection. Medium. https://towardsdatascience.com/anchor-boxes-the-key-to-quality-object-detection-ddf9d612d4f9.

12. Li, E. Y. (2021, April 21). Dive Really Deep into YOLO v3: A Beginner's Guide. Medium. https://towardsdatascience.com/dive-really-deep-into-yolo-v3-a-beginners-guide-9e3d2666280e.

13. He, & Huang, Chang-Wei & Wei, Liqing & Li, Lingling & Anfu, Guo. (2019). TF-YOLO: An Improved Incremental Network for Real-Time Object Detection. Applied Sciences. 9. 3225. 10.3390/app9163225.

14. ]Alexey Bochkovskiy, Chien-Yao Wang, and HongYuan Mark Liao. YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020.