Anomaly Recognition in surveillance videos using 3D convolutional network
Publication :
Here is the link of our published paper on this project
https://link.springer.com/article/10.1007/s11042-021-10570-3
GitHub Code Link (Implementation details of Proposed work )
https://github.com/ramnamaqsood/Activity_Recognition-3DCNN
Demo of videos (testing on proposed anomaly recognition algorithm)
https://drive.google.com/file/d/12YJy_ut4fP6ax_c3QZbx_QkczuT2WlQj/view?usp=sharing
Annotations of UCF Crime dataset:
You can find our frame level annotations of complete UCF Crime dataset after filling this form: https://docs.google.com/forms/d/e/1FAIpQLSf-qboNRohi7ZV6hdo0p8DH_mliGPwtERqo8OXlIM_Pd7y1Mg/viewform
Anomaly refers to some event which deviates from the normal pattern that does not fit the expected behavior or expected pattern. These non-conforming patterns are considered as abnormal, unexpected behavior, surprises in many application domains. Anomaly detection begins at the start of the 21st century and has been researched within diverse research areas and in application domains.
Many anomaly detection techniques have been specifically developed for certain application domains, while others are more generic. Anomaly refers to the pattern that does not similar to the exact definition of normal behavior and it might be induced for a variety of reasons such as a sudden fight in surveillance videos, abnormal crowd behavior, abnormal pattern in traffic, fraud detection in credit cards, presence of malignant in MRI image and event detection in sensor networks, insurance or health care, intrusion detection for cyber-security, fault detection in safety-critical systems, military surveillance for enemy activities and detection of different events in videos.
Another application of anomaly detection is detecting anomaly events in surveillance videos, as surveillance cameras can capture a variety of realistic anomalies so, through surveillance videos, recognition of different activities is a difficult task to achieve. The anomaly detection in surveillance videos is important due to the fact that surveillance cameras are increasingly being used in public places that capture a variety of realistic anomalous events, but the monitoring capability of law enforcement agencies has not kept pace e.g. traffic accidents, crimes or illegal activities, wrong road crossing and traffic rules. The occurrence of anomaly events are rare than normal events therefore, designing a computer-aided system for anomaly detection is important to alleviate the effort and time of labor.
Outline/Abstract :
Anomalous activity recognition deals with identifying the patterns and events that vary from the normal stream. In a surveillance paradigm, these events range from abuse to fighting and road accidents to snatching, etc. Due to the sparse occurrence of anomalous events, anomalous activity recognition from surveillance videos is a challenging research task. The approaches reported can be generally categorized as handcrafted and deep learning-based. Most of the reported studies address binary classification i.e. anomaly detection from surveillance videos. But these reported approaches did not address other anomalous events e.g. abuse, fight, road accidents, shooting, stealing, vandalism, and robbery, etc. from surveillance videos. Therefore, this paper aims to provide an effective framework for the recognition of different real-world anomalies from videos. This study provides a simple, yet effective approach for learning spatiotemporal features using deep 3-dimensional convolutional networks (3D ConvNets) trained on the University of Central Florida (UCF) Crime video dataset. Firstly, the frame-level labels of the UCF Crime dataset are provided, and then to extract anomalous spatiotemporal features more efficiently a fine-tuned 3D ConvNets is proposed. Findings of the proposed study are twofold 1) There exist specific, detectable, and quantifiable features in UCF Crime video feed that associate with each other 2) Multiclass learning can improve generalizing competencies of the 3D ConvNets by effectively learning frame-level information of dataset and can be leveraged in terms of better results by applying spatial augmentation. The proposed study extracted 3D features by providing frame level information and spatial augmentation to a fine-tuned pre-trained model, namely 3DConvNets. Besides, the learned features are compact enough and the proposed approach outperforms significantly from state of art approaches in terms of accuracy on anomalous activity recognition having 82% AUC.
Challenges
· The difference between normal and anomaly behavior is ambiguous and under realistic conditions same behavior could be normal or anomalous such as a crowed event spotted anywhere may sometimes considered as normal or anomalous.
· As anomaly is arises due to malicious activity, these actions accommodate themselves to appear as normal so the task of tagging normal regions become more difficult.
· It is to indicate an activity that differ from normal pattern in cameras and detect the occurrence time of anomaly.
· Lack of annotated/tagged training data make it hard to identify normal and anomaly patterns for a detection system to learn it better.
· Often data is noisy, and this noise considered as anomaly and most of the time it is difficult to remove noise or differentiate between noise and anomaly.
Motivation
Anomaly detection in surveillance videos is found to be the most challenging and long-standing problem in Computer Vision. For video surveillance applications, there are several attempts made to detect violence or aggression in videos. The most critical tasks in surveillance videos are activity recognition through different methodologies.
Surveillance cameras are increasingly being used in public places that capture a variety of realistic anomalous events, but the monitoring capability of law enforcement agencies has not kept pace. Most of the time there is no police officer to look at those videos so often when an “unusual” event happens, and no one is watching that so there must be an algorithm for attention mechanism, therefore our proposed method will flag those anomalies for officers so that they can take actions whenever anomaly happens.
To lessen the effort of labor and time, designing a computer aided system or algorithm to detect anomalies in videos is becoming a pressing need.
Demo: Explosion Recognition in test Video
System Pipeline
Usefull Reads
[1] "UCF-CRIME[online]," Available at :https://www.kaggle.com/mission-ai/crimeucfdataset, 2016."
[2] "Wang, T., & Snoussi, H. (2013, January). Histograms of optical flow orientation for abnormal events detection. In 2013 IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS) (pp. 45-52). IEEE."
[3] "Farooq, M. U., Khan, N. A., & Ali, M. S. (2017). Unsupervised Video Surveillance for Anomaly Detection of Street Traffic. INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 8(12), 270-275."
[4] "Sultani, W., Chen, C., & Shah, M. (2018). Real-world Anomaly Detection in Surveillance Videos. Center for Research in Computer Vision (CRCV), University of Central Florida (UCF)."
[5] "Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015). Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision (pp. 4489-4497)."
[6] "Sabokrou, M., Fathy, M., Hoseini, M., & Klette, R. (2015). Real-time anomaly detection and localization in crowded scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 56-62)."
[7] "Sabokrou, M., Fayyaz, M., Fathy, M., & Klette, R. (2017). Deep-cascade: cascading 3D deep neural networks for fast anomaly detection and localization in crowded scenes. IEEE Transactions on Image Processing, 26(4), 1992-2004."
Dr. Usama Ijaz Bajwa
Co-PI, Video Analytics lab, National Centre in Big Data and Cloud Computing,Program Chair (FIT 2019),HEC Approved PhD Supervisor,Assistant Professor & Associate Head of DepartmentDepartment of Computer Science,COMSATS University Islamabad, Lahore Campus, Pakistanwww.usamaijaz.comwww.fit.edu.pkJob ProfileGoogle Scholar Profile