SURVEILIA
2020-2021
Surveilia is a desktop application that detects anomaly in real-time by consuming low-resources. Anomaly can be classified as theft, burglary, robbery etc. It is a python-based application developed using OpenCV, PyQt5, PyTorch, NumPy, SQLite3, and NVIDIA’s CUDA.
ACHIEVEMENTS
SURVEILIA secured 3rd position among 110+ Final year projects in CS Department, COMSATS Lahore.
SURVEILIA got funding from National Grassroots ICT Research Initiative (NGIRI) at IGNITE.
INTRODUCTION
The present world has evolved in the terms of advanced technology but still one does not feel secure at malls and other public places. Every day, we hear stories of robbery, vandalism, and other street crimes. Though the surveillance cameras are increasingly being installed in almost all public areas, some of the incidents are either unnoticed or are detected when it is too late as the law enforcement monitoring abilities have not kept up with the pace. The eyes of law enforcement miss several anomalous incidents. Unfortunately, the surveillance is done by human operators. It is almost impossible for a human brain to thoroughly focus on multiple CCTV screens when multiple activities are taking place in parallel across each screen. Some have improved their systems by using DVR (digital video recorder) or applications built through deep learning techniques, but they either require an update in system or CCTVs with integrated chips. Hence, there is a need for a deep learning-based application that utilizes fewer resources and provide security by timely detecting abnormal events.
To solve this problem, there is a need for a low resource-efficient automated system that differentiates between normal and abnormal events in multiple streams, simultaneously, and in case of any abnormal event, the security team is alerted to take appropriate actions immediately. With the development of Surveilia, any abnormal activity in a live video stream or a video file is instantly reported. The system clips and stores the part where an abnormal activity (if any) occurs, making the process of a surveillance system reliable, efficient, effective, and reducing the constant human dependency.
PROJECT SCOPE
Security is one of the key concerns for any person, either at home or in public places. Surveillance cameras are now being used in almost every area, be it home or public place. Security guards sit behind those CCTV screens, inspecting the behaviour of people, and monitoring their activities. But unfortunately, the human brain cannot focus on multiple screens at a single time. Several applications are being developed to automatically detect abnormal activities, therefore, the security can be ensured, and people can move without safety risks and concerns. We have developed an application that automatically detects abnormal activity in surveillance CCTV streams and generates an alarm.
Presently, much work is done in the field of activity recognition but most of them require high computational costs, expensive and advanced systems with the best GPUs. Hence, they are computation-intensive, and training them requires a lot of training data. Resultantly, the user must update his/her system to achieve the desired result. To overcome this problem, we have proposed a low resource-efficient system that detects an anomaly in multiple streams in real-time.
Although we have a lot of pros in our application but there are some small but technical cons; detection of abnormal and normal activities as separate is doubtful, especially in crowded areas, unavailability of labelled data makes system to be inefficient as it’s learning is not enough and to balance privacy of crowd along with the detection of anomalies.
OBJECTIVES
The objectives for the proposed project are as follows:
To develop a product that can detect the actions of humans and their interpretation using deep learning knowledge through automated activity analysis.
To provide a resource-efficient system that works on a simple machine such as a CPU with real-time streaming.
To build a system that can run and detect activities on multiple screens, simultaneously.
The product that supports the security workers to enhance the method of handling abnormal activities in real-time.
To observe suspicious/anomalous behaviour and reduce constant human dependence.
To automate the anomaly detection process likely to be as human
Clipping and storing the part where the anomaly has occurred.
To build the application GUI, user-friendly, easy to use and interactive so that a common man such as a security guard can use it, with less efforts.
TO KNOW ABOUT SURVEILLANCE SYSTEMS
TO KNOW ABOUT NVIDIA JETSON NANO
RELATED WORK
Surveillance App: Surveillance App [1] is a real-time wireless surveillance CCTV home system app developed by Reservoir Dev, which detects noise or any movement in the live stream and notifies the viewer. It allows the user to monitor his/her home and speak through the user’s device microphone to scare off the thief/intruder. But it is limited to indoor or specifically, home of the user.
iCentana: iCentana [2] is AI video analytics for automated real-time identification of critical events. It monitors cameras and automatically detects any abnormal activity. It identifies precursor events, learns and adapts automatically, rapidly reviews recorder video, and detects abnormal precursor events. They are costly and lack in localization of anomaly.
Dahua Security: A Chinese CCTV company [3] launched cameras to detect anomalies in public areas. Their mission is “Enabling a safer society and smarter living”. They have cameras with integrated chips to be able to run deep neural networks right on board. Key technologies used are HDCVI Technology, Predictive Focus Algorithm, EPOE, and ANPR. Though, it is much accurate but requires special CCTV cameras with integrated chips, which are very costly.
Hikvision: Hikvision [4] provides digital surveillance and is also Chinese based, CCTV cameras embedded with intelligent video analytics, company. But people report that it is poor efficiency, does not train properly and causes much delay in alarming the security person which is a huge problem.
Mobotix: Mobotix [5] provides a surveillance system for both indoor and outdoor. It is secure to use. But it is very much complex in terms of its user interface. Most of the important features are hidden behind the toolbars.
HOW THE APPLICATION WORKS?
The flow of the application is discussed below and you can see the visual representation next to it.
Camera or Footage: Live video stream or CCTV footage is fed to the system or to NVIDIA Jetson Nano.
Preprocessing: Data preprocessing is applied and frames are fed to inference engine.
Inferencing Model: Frames are fed to the Trained TSM based Inference Engine to obtain probability outputs/results.
Anomaly Detection: Anomaly is detected and key-event anomaly snippets are extracted.
User Alert: Instant Notification is generated.
WHAT DOES THE APPLICATION INCLUDE?
Minimizes the work of security team by automating the surveillance task.
Detects anomalies such as theft, robbery, burglary, etc.
Detects abnormal activities from single to multiple camera streams. (upto 6 simultaneously).
Detects abnormal activities over live CCTV/ IP Camera input feed, stored video or Webcam.
Requires low resources for computation
Embedded AI on the edge by deploying core application on NVIDIA Jetson Nano.
DATASETS USED FOR THIS PROJECT
UCF Crime dataset [6] (Used in Project)
70 Hours of Videos
1120 Trimmed Videos
Training: 840 Videos
Validation: 140 Videos
Test: 140 Videos
The 20BN-Something-something Dataset V1. [7] (Learning Purpose)
The 20BN-Something-something Dataset V2. [8] (Learning Purpose)
USER INTERFACE
Login Screen
Welcome Screen
Supports Multi-language
Three Options to Add Camera
Camera Screen (displaying 6 screens running simultaneously.)
Displaying the history of anomalies
POSTER
PRESENTATION SLIDES
CS28_SURVEILIA_PRESENTATION (1).pptxTOOLS AND TECHNIQUES
Python
Anaconda
Jupyter Notebook
Google Colab
PyTorch
QT Designer
FUTURE WORK
Discussed below are the ideas for expansion of the project in the future:
In the future, this project can be extended by adding multi classes of anomaly detection such as detection of camera tampering.
Another extension would be, providing the user to choose between the classes of the anomaly which he/she wants to detect at the camera instead of all the classes to be detected at the same time. Such as, if the camera is placed in the meeting room; the user may only want to detect any missing object from the room.
Integration with the City-Wide Surveillance network
Web Application and Mobile Application
THE TEAM
PROJECT SUPERVISOR
Ifrah Tehleel
Email: ifrahteh@gmail.comBSCS(Computer Science, COMSATS Lahore)Linkedin ProfileGithub Profile
Nauman Akram
Email: iemnauman.akram@gmail.comBSCS(Computer Science, COMSATS Lahore)Linkedin ProfileGithub Profile
Jan Muhammad Mirza
REFERENCES
[1] R. Dev, “Surveillance App,” [Online]. Available: https://play.google.com/store/apps/details?id=com.reservoirdev.surveillance&hl=en. [Accessed 30 July 2020].
[2] “iCetana,” [Online]. Available: https://icetana.com/. [Accessed 30 July 2020].
[3] D. Security. [Online]. Available: https://www.dahuasecurity.com/. [Accessed 31 July 2020].
[4] “Hikvision,” Hikvision, [Online]. Available: https://www.hikvision.com/. [Accessed 27 July 2020].
[5] “Mobotix,” [Online]. Available: https://www.mobotix.com/en/unique-quality. [Accessed 27 July 2020].
[6] “UCF Crime Dataset,” [Online]. Available: https://www.crcv.ucf.edu/projects/real-world/. [Accessed 29 July 2020].
[7] “Something-Something V1,” [Online]. Available: https://20bn.com/datasets/something-something/v1. [Accessed 31 July 2020].
[8] “Something-Something V2,” [Online]. Available: https://20bn.com/datasets/something-something. [Accessed 30 July 2020].
[9] “Temporal Shift Module,” [Online]. Available: https://github.com/mit-han-lab/temporal-shift-module. [Accessed 30 July 2020].