Road Pulse 

Traffic Violation Detection and Tracking


ACHIEVEMENT

Our Group Road Pulse secured 3rd Position in FYP 2021, COMSATS Lahore 

Project Abstract

Safety and comfort of road users is a matter of great concern. Due to increase in traffic violation police enforcement is necessary element in road safety. It is essential to build a safer and much more reliable and efficient traffic control and management system to avoid tragic events like accidents etc. We try to propose a system which can detect traffic violations like speed violation, stop line violation, red line violation etc. According to different road conditions our system will be able to provide an estimation of speed limit, vehicles count and accidents prediction. Traffic violation detection system is an effective tool to help traffic administrators so a particular action can be taken in real time .

Rising traffic congestion rules violation and accidents due to lack of proper traffic management and surveillance need to be addressed with smart solutions. Like automated detection of rules violations, vehicles congestion etc. integrated with video analytics that can effectively aid traffic administration. 

 We try to propose a system which can detect traffic violations like speed violation, stop line violation, red line violation etc. According to different road conditions our system will be able to provide an estimation of speed limit, vehicles count and accidents prediction. The accident prediction model is represented by a generalized linear model which, on the basis of the available data, determines the expected number of accidents for individual types of road segments. A critical road segment is defined as a segment where the reported number of accidents significantly exceeds the number of expected accidents on roads with similar geometric and traffic characteristics. This method can be used as an effective tool for road network safety management. This system will also help traffic administrators so a particular action can be taken in real time. 

Introduction 

Violation of traffic rules is a critical and rising issue presently. Fast reaction and prevention of traffic violation plays a key role to ensure safety of vehicles and citizens. For this purpose, in Pakistan surveillance cameras are installed everywhere to prevent traffic violations but they are unable to detect events like red-light violation, wrong way driving etc. due to the faulty architecture. In Pakistan traffic issues are rising day by day which are causing many causalities like accidents, loss of innocent lives and compromise in safety of citizens. Recently a survey related to the accidents causing deaths in Pakistan increases dramatically due to the traffic violations.So to minimize such alarming factors we should provide an automated solution using latest technology which will help to reduce these factors.

Rising traffic congestion,rules violation and accidents due to lack of proper traffic management and surveillance need to be addressed with smart solutions. Like automated detection of rules violations, vehicles congestion etc. integrated with video analytics that can effectively aid traffic administration. 

By implementing such system labour cost can be reduced.  Reduction in labour will also eliminate many other factors like lack of attention while monitoring many cameras at the same time, pin pointing such events when and where are they happening etc.

Automatic traffic violation in Video analytics using can play an important role in getting in-depth insights into traffic conditions [4] and these insight can help by routing this information to traffic administrators which can take an effective action on any type of event. Surveillance cameras are cheap and ubiquitous, but the labour required for monitoring them increases the cost now a day. Because generally it happens that the surveillance cameras are not monitored by proper attention and it is also difficult for a person to pin point a violation between large number of screens. Usually, either no video is monitored at all or infrequent video is observed, alternatively it is used only for reviewing the incident just for once. But they are also helpful as they are capable of detecting events instead of passive recording. They make use of the events as they appear to happen and in accordance to it, take appropriate actions such as alerting the traffic management department. This is the utmost requirement of automatic violation detection system. 

So, our main objective is to derive an optimized and effective solution in traffic violation matter which can detect any type of violation such as red light violation, wrong-way vehicle detection, over speeding and stop line violation etc. Our proposed system will be flexible in nature. Flexibility allows user add new module to system easily. Some other information can also be infer from this system like vehicle count, provide a prediction of an accident using traffic pattern etc. This system will help the Traffic management authorities in accurate violations detection so proper action can be taken in real time and prevent accidents and guarantee citizens safety.

Machine learning algorithms and digital image processing methods are being used for these types of problems [5] still there are many complexities which require greater computation and adaptiveness in algorithms which is possible but are time taking

They are many solutions proposed to detect the abnormal patterns in traffic but they require proper guidance how to interact with application’s UI, costly hardware such as costly sensors to detect any type of violation and heavy GPU for processing video streams and are cost effective. So we try to propose a system based solution by training our model with latest and 3 to 4 types of datasets which will cover all type of traffic violation. The system will be cheaper eliminating the costly hardware like GPU’s, sensors etc. Other factors are discussed below.


Basic System Module  diagram

Basic working functionality of the system for detecting the anomalies in real time using our 3D trained model.

Motivation and Scope 

The main motive to design and develop such system is to reduce the amount of road accidents by ensuring proper traffic regulation and management. Reduction in number of road accidents will result in the reduction of human injuries, death rate and financial loss.

This system will also help and assist law enforcement agencies to minimize the traffic violations which will ultimately reduce the risk factor for pedestrians etc. Accuracy and quick processing will help such departments to take regarding set of action in real time. Interactive and user friendly UI will allow law enforcement agencies to use of the 3D trained model quite effectively. The initial scope of the project involves the detection of traffic rules violation like red light violation, wrong way moving vehicle, stop line violation etc. and vehicle count, but gradually we will try to extend our domain toward new modules for example accident prediction using road patterns which will play an important role in current traffic situations.

Assumptions: Our model will be trained using three different datasets which cover’s almost all types of violation. Also datasets will be taken from quite reliable resources


System Architecture 

Since there are many architectures are introduced to detect traffic violation automatically. Architectures like spatial analysis were previously proposed for traffic accidents and vehicle communication system data [9] and Vehicle detection through image processing for traffic surveillance and control [10] only target spatial domain only. 

So, we will not only manipulate spatial domain for our data but will also work on temporal domain. Our architecture will use 3D deep neural networks based models which extract useful features from videos both spatially and temporally.

Training process will start from converting the input video into frames. These 3D frames will be passed to our 3D architecture for training and learning purpose of our model. After that once this model is trained it will be tested on different datasets to check the accuracy and reliability. Then this architecture will be mounted in an application having an interactive UI for the automatic detection of traffic violation. 

Application design will be quite simple, innovative and user friendly. Application will detect different violations in videos and will pop up a message or alarm to pin point the event. This will make easier for operator to differentiate between the violation event and other activities happening in the video.

Basic Working Flowchart of our Architecture

Pipeline ::

Trained our model on custom classes .Custom classes include Lane Violation and vehicle crash.

The weight files were saved in form iterations around after every 2000 frames .This was done to be on the safe side that if training stops at any point we can resume the training.

The training process was done on Google Collab using GPU environment.

The accuracy of the model on test videos were around 75 to 85 %  as the frames were limited around 10k frames were trained

Problems Faced During Development Phase 

1-Labelling the dataset.

-Manually labelled the dataset by watching every video (200 Videos).

-The labels of dataset were in XML format but later on we discovered that YOLO V3 use labels of .txt format

-So all the labels were converted then to YOLO V3 format using a script written in python

-Average video length was around 12-13 minutes

2-Using YOLO V3  --> ‘You Only Look Once’

Problem: Understanding the YOLO V3.

Solution: Extensive research which led to these results:

-YOLO makes use of only convolutional layers, making it a fully convolutional network (FCN) which uses deeper architecture of a feature extractor called Darknet-53.

-Each layer is followed by a batch normalization layer .No form of pooling is used, and a convolutional layer with stride 2 is used to down sample the feature maps.

-This network applies single Neural network to the Full Image which divides the image into regions and predicts bounding boxes and probabilities for each region.

3-Custom Training Files

Problem: Creating custom training files for our custom classes and extensive training time

Solution:

-Used the already available CFG files .Update the main parameters like batch size ,classes, filter size in the convolutional layers.

-After first training which was done in around 11 hours reduced the convolutional layers to reduce training time.

-Used colab GPU Environment for the training

-Implemented the model on our CPU.

4-Appending Processing Info

Problem: Tried to display the backened processing to the user on web page .Got compatibility issues with the HTML.

Solution:

-Tried to pass the Jason object to the HTML but object was unable to decode.

-So the processing output was displayed in browser console

5-Threshold Estimation

Problem: Defining the threshold for what can be considered an anomalous snippet

Solution:

Static Thresholding:

-Use a static value i.e 0.5 to classify anomalous segment.

Dynamic Thresholding:

-Frames with scores exceeding the mean of smoothened scores are considered as anomalous.

Goals & Objectives 

 


Datasets used in our project 

AI City Challenge 2019

Web Application User Interface & Demo

Login Screen

User with valid credentials can access our web services

Password Reset Screen

If user forgot his password can reset using his email that is registered in our Firebase Authentication

Dashboard Screen

After Login user can access our dashboard where multiple options are displayed

Video Uploading Screen

Here user can choose a video of his own choice to run detections on the video. Video is uploaded on firestore from where it is passed to our backend flask server.

Backend Processing Displayed to user:

As soon the video is uploaded the video processing is shown to the user through the browser console where all the Hyper-parameters are shown to the user to keep track of processing.

Note: Processing time depends upon the video length and resolution.

Crash Event

Crash class problem is detected from the user-uploaded video. Clipped crash event video and key event frames with relevant information are populated at the output screen using Google Firebase.

Lane Violation

Lane Violation is detected from the user-uploaded video. Clipped Lane Violation anomaly video and key event frames with relevant information are populated at the output screen using Google Firebase.

Firebase Database

Here, all the processed frames with relevant information are saved in the cloud firestore. This information is populated in our web output interface and later on same info is sent to our android application for tracking purposes.

Web Application Demo

Web App Demo.mp4

Android Application UI & Demo

Login Screen

Homepage

Track and Info Page

Clipped Anomaly Frames

Location Pinned on Map

Track and Navigate

Android Application Demo

Android Application Crash Demo.mp4

Road Pulse Poster

Road Pulse Poster.pdf

Road Pulse Presentation

Road pulse Presentation.pptx

Tools and Technologies 

Tools and technologies that will be used for this project:

For Web Application Development

Google Colab

Python Language

Jupyter Notebook

TensorFlow

Flask

 Web Application Framework

Google Firebase

A Real-Time Database for Web & Android Applications

For Mobile Interfaces (UI) :

Java Language

Programming Language used to develop App

Google Firebase

Same Real-Time Database For Android App

Android Studio

Tool To Develop Android Apps

Project Supervisor

Dr. Usama Ijaz BajwaCo-PI, Video Analytics lab, National Centre in Big Data and Cloud Computing,Program Chair (FIT 2019),HEC Approved PhD Supervisor,Assistant Professor & Associate Head of DepartmentDepartment of Computer Science,COMSATS University Islamabad, Lahore Campus, Pakistanwww.usamaijaz.comwww.fit.edu.pk

The Team

Hassan Sharjeel

Email: hsjoiya@gmail.com
BS Student(Computer Science, COMSATS University Islamabad, Lahore)
Linkedin Profile

Hamza Latif

Email: hamza.latif773@gmail.com
Github Profile 
BS Student(Computer Science, COMSATS University Islamabad, Lahore)
Linkedin Profile

References

[1]  S. K. H. Kazmi, “Pakistan And Gulf Economist,” news,research, 11 2017. [Online]. Available: http://www.pakistaneconomist.com/2017/11/13/alarming-road-accidents-rate-pakistan-rules-laws-need-overhaul/. [Accessed 14 2 2020].

[2]  C. Wang, S. Ison and M. Quddus, “Impact of traffic congestion on road accidents: A spatial analysis of the M25 motorway in England,” Accident; analysis and prevention, p. 12, 2009. 

[3]  D. Deme and M. Bari, “Traffic Accident Causes and Its Countermeasures on Addis Ababa-Adama Expressway,” Journal of Equity in Science and Sustainable Development, p. 12, 2016. 

[4]  V. .B*, and M. Babu, “Dynamic Traffic- Rule- Violation Monitoring and Detection System,” IJESRT, p. 14, 2014. 

[5]  “Telegra,” Smart Traffic Mangement , [Online]. Available: https://www.telegra-europe.com/products/product_category-1/product-580. [Accessed 12 February 2020].

[6]  M. Bramberger, J. Brunner and B. Rinner, “Real-Time Video Analysis on an Embedded Smart Camera,” 10th IEEE Real-Time and Embedded Technology and Applications Symposium, p. 11, 2004. 

[7]  S. Kamijo, Y. Matsushita, K. Ikeuchi and M. Sakauchi, “Traffic Monitoring and Accident Detection at,” IEEE Transactions on Intelligent Transportation Systems, vol. 1, p. 20, 2000. 

[8]  A. mariya T.P, A. M.J, F. Aishwarya and L. George, “TRAFFIC VIOLATION DETECTION SYSTEM,” Traffic Enforcement System, vol. 4, no. 03, p. 4, 2017.