Road Pulse
Traffic Violation Detection and Tracking
ACHIEVEMENT
Our Group Road Pulse secured 3rd Position in FYP 2021, COMSATS Lahore
Project Abstract
Safety and comfort of road users is a matter of great concern. Due to increase in traffic violation police enforcement is necessary element in road safety. It is essential to build a safer and much more reliable and efficient traffic control and management system to avoid tragic events like accidents etc. We try to propose a system which can detect traffic violations like speed violation, stop line violation, red line violation etc. According to different road conditions our system will be able to provide an estimation of speed limit, vehicles count and accidents prediction. Traffic violation detection system is an effective tool to help traffic administrators so a particular action can be taken in real time .
Rising traffic congestion rules violation and accidents due to lack of proper traffic management and surveillance need to be addressed with smart solutions. Like automated detection of rules violations, vehicles congestion etc. integrated with video analytics that can effectively aid traffic administration.
We try to propose a system which can detect traffic violations like speed violation, stop line violation, red line violation etc. According to different road conditions our system will be able to provide an estimation of speed limit, vehicles count and accidents prediction. The accident prediction model is represented by a generalized linear model which, on the basis of the available data, determines the expected number of accidents for individual types of road segments. A critical road segment is defined as a segment where the reported number of accidents significantly exceeds the number of expected accidents on roads with similar geometric and traffic characteristics. This method can be used as an effective tool for road network safety management. This system will also help traffic administrators so a particular action can be taken in real time.
Introduction
Violation of traffic rules is a critical and rising issue presently. Fast reaction and prevention of traffic violation plays a key role to ensure safety of vehicles and citizens. For this purpose, in Pakistan surveillance cameras are installed everywhere to prevent traffic violations but they are unable to detect events like red-light violation, wrong way driving etc. due to the faulty architecture. In Pakistan traffic issues are rising day by day which are causing many causalities like accidents, loss of innocent lives and compromise in safety of citizens. Recently a survey related to the accidents causing deaths in Pakistan increases dramatically due to the traffic violations.So to minimize such alarming factors we should provide an automated solution using latest technology which will help to reduce these factors.
Rising traffic congestion,rules violation and accidents due to lack of proper traffic management and surveillance need to be addressed with smart solutions. Like automated detection of rules violations, vehicles congestion etc. integrated with video analytics that can effectively aid traffic administration.
By implementing such system labour cost can be reduced. Reduction in labour will also eliminate many other factors like lack of attention while monitoring many cameras at the same time, pin pointing such events when and where are they happening etc.
Automatic traffic violation in Video analytics using can play an important role in getting in-depth insights into traffic conditions [4] and these insight can help by routing this information to traffic administrators which can take an effective action on any type of event. Surveillance cameras are cheap and ubiquitous, but the labour required for monitoring them increases the cost now a day. Because generally it happens that the surveillance cameras are not monitored by proper attention and it is also difficult for a person to pin point a violation between large number of screens. Usually, either no video is monitored at all or infrequent video is observed, alternatively it is used only for reviewing the incident just for once. But they are also helpful as they are capable of detecting events instead of passive recording. They make use of the events as they appear to happen and in accordance to it, take appropriate actions such as alerting the traffic management department. This is the utmost requirement of automatic violation detection system.
So, our main objective is to derive an optimized and effective solution in traffic violation matter which can detect any type of violation such as red light violation, wrong-way vehicle detection, over speeding and stop line violation etc. Our proposed system will be flexible in nature. Flexibility allows user add new module to system easily. Some other information can also be infer from this system like vehicle count, provide a prediction of an accident using traffic pattern etc. This system will help the Traffic management authorities in accurate violations detection so proper action can be taken in real time and prevent accidents and guarantee citizens safety.
Machine learning algorithms and digital image processing methods are being used for these types of problems [5] still there are many complexities which require greater computation and adaptiveness in algorithms which is possible but are time taking
They are many solutions proposed to detect the abnormal patterns in traffic but they require proper guidance how to interact with application’s UI, costly hardware such as costly sensors to detect any type of violation and heavy GPU for processing video streams and are cost effective. So we try to propose a system based solution by training our model with latest and 3 to 4 types of datasets which will cover all type of traffic violation. The system will be cheaper eliminating the costly hardware like GPU’s, sensors etc. Other factors are discussed below.
Basic System Module diagram
Basic working functionality of the system for detecting the anomalies in real time using our 3D trained model.
Motivation and Scope
The main motive to design and develop such system is to reduce the amount of road accidents by ensuring proper traffic regulation and management. Reduction in number of road accidents will result in the reduction of human injuries, death rate and financial loss.
This system will also help and assist law enforcement agencies to minimize the traffic violations which will ultimately reduce the risk factor for pedestrians etc. Accuracy and quick processing will help such departments to take regarding set of action in real time. Interactive and user friendly UI will allow law enforcement agencies to use of the 3D trained model quite effectively. The initial scope of the project involves the detection of traffic rules violation like red light violation, wrong way moving vehicle, stop line violation etc. and vehicle count, but gradually we will try to extend our domain toward new modules for example accident prediction using road patterns which will play an important role in current traffic situations.
Assumptions: Our model will be trained using three different datasets which cover’s almost all types of violation. Also datasets will be taken from quite reliable resources
System Architecture
Since there are many architectures are introduced to detect traffic violation automatically. Architectures like spatial analysis were previously proposed for traffic accidents and vehicle communication system data [9] and Vehicle detection through image processing for traffic surveillance and control [10] only target spatial domain only.
So, we will not only manipulate spatial domain for our data but will also work on temporal domain. Our architecture will use 3D deep neural networks based models which extract useful features from videos both spatially and temporally.
Training process will start from converting the input video into frames. These 3D frames will be passed to our 3D architecture for training and learning purpose of our model. After that once this model is trained it will be tested on different datasets to check the accuracy and reliability. Then this architecture will be mounted in an application having an interactive UI for the automatic detection of traffic violation.
Application design will be quite simple, innovative and user friendly. Application will detect different violations in videos and will pop up a message or alarm to pin point the event. This will make easier for operator to differentiate between the violation event and other activities happening in the video.
Basic Working Flowchart of our Architecture
Pipeline ::
Trained our model on custom classes .Custom classes include Lane Violation and vehicle crash.
The weight files were saved in form iterations around after every 2000 frames .This was done to be on the safe side that if training stops at any point we can resume the training.
The training process was done on Google Collab using GPU environment.
The accuracy of the model on test videos were around 75 to 85 % as the frames were limited around 10k frames were trained
Problems Faced During Development Phase
1-Labelling the dataset.
-Manually labelled the dataset by watching every video (200 Videos).
-The labels of dataset were in XML format but later on we discovered that YOLO V3 use labels of .txt format
-So all the labels were converted then to YOLO V3 format using a script written in python
-Average video length was around 12-13 minutes
2-Using YOLO V3 --> ‘You Only Look Once’
Problem: Understanding the YOLO V3.
Solution: Extensive research which led to these results:
-YOLO makes use of only convolutional layers, making it a fully convolutional network (FCN) which uses deeper architecture of a feature extractor called Darknet-53.
-Each layer is followed by a batch normalization layer .No form of pooling is used, and a convolutional layer with stride 2 is used to down sample the feature maps.
-This network applies single Neural network to the Full Image which divides the image into regions and predicts bounding boxes and probabilities for each region.
3-Custom Training Files
Problem: Creating custom training files for our custom classes and extensive training time
Solution:
-Used the already available CFG files .Update the main parameters like batch size ,classes, filter size in the convolutional layers.
-After first training which was done in around 11 hours reduced the convolutional layers to reduce training time.
-Used colab GPU Environment for the training
-Implemented the model on our CPU.
4-Appending Processing Info
Problem: Tried to display the backened processing to the user on web page .Got compatibility issues with the HTML.
Solution:
-Tried to pass the Jason object to the HTML but object was unable to decode.
-So the processing output was displayed in browser console
5-Threshold Estimation
Problem: Defining the threshold for what can be considered an anomalous snippet
Solution:
Static Thresholding:
-Use a static value i.e 0.5 to classify anomalous segment.
Dynamic Thresholding:
-Frames with scores exceeding the mean of smoothened scores are considered as anomalous.
Goals & Objectives
The project aims to automate the traffic rules violation detection system and make it convenient for the traffic police department to monitor the traffic and take action against the violated vehicle owner in a fast and efficient way. Detecting and recognizing the vehicle and their activities accurately is the main priority of the system.
We want to design a generic model that would cover many separate related projects like Sensor less red light violation detection system and Vehicle detection and counting system based on vision
Also we want to overcome the human errors in video surveillance
Automatic traffic violation in Video analytics using can play an important role in getting in-depth insights into traffic conditions and these insights can help by routing this information to traffic administrators which can take an effective action on any type of event.
By implementing such system labor cost can be reduced. Reduction in labor will also eliminate many other factors like lack of attention while monitoring many cameras at the same time, pin pointing such events when and where are they happening etc.
Recently working traffic architecture are not so precise and results aren’t that much clear to investigate the driver. It only gathers vehicle info, so we want to cover all the flaws present in previous system
Datasets used in our project
AI City Challenge 2019
No of classes = 2
Classes Names
Plain Dataset which means no annotations were provided with the dataset.
Lane Violation & Vehicle Crash
100 Train + 100 Test Videos
Web Application User Interface & Demo
Login Screen
User with valid credentials can access our web services
Password Reset Screen
If user forgot his password can reset using his email that is registered in our Firebase Authentication
Dashboard Screen
After Login user can access our dashboard where multiple options are displayed
Video Uploading Screen
Here user can choose a video of his own choice to run detections on the video. Video is uploaded on firestore from where it is passed to our backend flask server.
Backend Processing Displayed to user:
As soon the video is uploaded the video processing is shown to the user through the browser console where all the Hyper-parameters are shown to the user to keep track of processing.
Note: Processing time depends upon the video length and resolution.
Crash Event
Lane Violation
Firebase Database
Here, all the processed frames with relevant information are saved in the cloud firestore. This information is populated in our web output interface and later on same info is sent to our android application for tracking purposes.
Web Application Demo
Android Application UI & Demo
Login Screen
Homepage
Track and Info Page
Clipped Anomaly Frames
Location Pinned on Map
Track and Navigate
Android Application Demo
Road Pulse Poster
Road Pulse Presentation
Tools and Technologies
Tools and technologies that will be used for this project:
For Web Application Development
Google Colab
Python Language
Jupyter Notebook
TensorFlow
Flask
Google Firebase
For Mobile Interfaces (UI) :
Java Language
Programming Language used to develop App
Google Firebase
Same Real-Time Database For Android App
Android Studio
Tool To Develop Android Apps
Project Repo & Related Links
Github Link:
https://github.com/FYP-Road-Pulse
Web + Android Demo:
https://drive.google.com/drive/folders/1k0tpwEiaks7xueJwV7-um_wfYIet4OCV?usp=sharing
Custom Trained Model YOLO + Custom Files
https://drive.google.com/drive/folders/1LvB4cocywQEMWkJFqPkWBklP1dulnqkq?usp=sharing
Project Supervisor
The Team
Hassan Sharjeel
BS Student(Computer Science, COMSATS University Islamabad, Lahore)
Linkedin Profile
Hamza Latif
Github Profile
BS Student(Computer Science, COMSATS University Islamabad, Lahore)
Linkedin Profile
References
[1] S. K. H. Kazmi, “Pakistan And Gulf Economist,” news,research, 11 2017. [Online]. Available: http://www.pakistaneconomist.com/2017/11/13/alarming-road-accidents-rate-pakistan-rules-laws-need-overhaul/. [Accessed 14 2 2020].
[2] C. Wang, S. Ison and M. Quddus, “Impact of traffic congestion on road accidents: A spatial analysis of the M25 motorway in England,” Accident; analysis and prevention, p. 12, 2009.
[3] D. Deme and M. Bari, “Traffic Accident Causes and Its Countermeasures on Addis Ababa-Adama Expressway,” Journal of Equity in Science and Sustainable Development, p. 12, 2016.
[4] V. .B*, and M. Babu, “Dynamic Traffic- Rule- Violation Monitoring and Detection System,” IJESRT, p. 14, 2014.
[5] “Telegra,” Smart Traffic Mangement , [Online]. Available: https://www.telegra-europe.com/products/product_category-1/product-580. [Accessed 12 February 2020].
[6] M. Bramberger, J. Brunner and B. Rinner, “Real-Time Video Analysis on an Embedded Smart Camera,” 10th IEEE Real-Time and Embedded Technology and Applications Symposium, p. 11, 2004.
[7] S. Kamijo, Y. Matsushita, K. Ikeuchi and M. Sakauchi, “Traffic Monitoring and Accident Detection at,” IEEE Transactions on Intelligent Transportation Systems, vol. 1, p. 20, 2000.
[8] A. mariya T.P, A. M.J, F. Aishwarya and L. George, “TRAFFIC VIOLATION DETECTION SYSTEM,” Traffic Enforcement System, vol. 4, no. 03, p. 4, 2017.