Results

mean Average Precision (mAP)

The following table shows the average precision of the six classes compared to models with the original dataset as well as the pretrained weights.

The new model has better performance on the Car class, Truck class, and Traffic Light class but worse performance on Person class and Traffic Sign class.

Fig 8. Average Precision of the six classes

Precision-Recall Curve

The precision-recall curve also could verify the results in the table. The model has overall better performance on Bus class, Truck class, Car class, and Traffic Light class.

Fig 9. PR curves of all the six classes

Parameter sensitivity testing

For the sensitivity test, the testing range of confidence threshold is between 0.01 and 0.3. The testing range of IOU threshold is between 0.2 and 0.8.

According to the results in the two figures, the ideal confidence threshold is between 0.01 and 0.06, and the ideal IOU threshold is between 0.4 and 0.6. We are choosing 0.05 for the confidence threshold and 0.5 for IOU threshold for the testing.

Fig 10. mAP vs. IOU threshold

Fig 11. Heatmap of mAP

Testing results of the YOLO model

The YOLO model can almost accurately detect the objects in the images. But errors may also occur. For example, some people in Fig. 11(d) are incorrectly detected as bus class.

Fig 11(a). Image testing example 1

Fig 11(c). Image testing example 3

Fig 11(b). Image testing example 2

Fig 11(d). Image testing example 4

Fig 12(a). Video testing example 1

Fig 12(b). Video testing example 1

Testing results of the Deep SORT model

For the video data, we first use the YOLO model to detect the objects, and then use the results of YOLO as the input of the Deep SORT model. Since the Deep SORT algorithm could predict the future movement of the object, the motions of the bounding boxes are smoother than the bounding boxes generated by the YOLO model.

Fig 13(a). Video testing example 1

Fig 13(b). Video testing example 2

Page updated

Report abuse