Publications and Ongoing Projects

1. Towards Human Activity Recognition: A Survey- Published in Neural Computing and Applications (NCAA), Springer Nature and under Minor Revision

2. Efficient Anomaly Recognition Using Surveillance Videos- Published in PeerJ Computer Science Journal and under minor revision

3. SurveillanceNet: Spatio-Temporal Anomaly Identification in Surveillance Videos using Two-Stream CNN and LSTM- Submitted in Multimedia Tools and Applications, Springer

4. Anomaly Identification Using Surveillance Videos-In Submission Process

5. A Robust Deep Networks based Multi-Object Multi-Camera Tracking System for City Scale Traffic-Published in Multimedia Tools and Applications, Springer

Introduction

Surveillance is the monitoring of behavior, activities, or information for the purpose of influencing, managing, or directing. The activity assessment process is complex and cannot be done at once but by performing the necessary breakdown, the task becomes easy. Unintentionally, some researchers confuse the term activity with the terms action, gesture, motion, and interaction, but in reality they can be part of activity but are not equal in definition.

Anomaly detection aims to identify unusual patterns, anomalies, or data points that do not conform to the expected distribution. Applications of anomaly detection include security surveillance, fraud detection in financial transactions, fault detection in manufacturing, intrusion detection in a computer network, monitoring sensor readings in an aircraft, spotting potential risk or medical problems in health data, and predictive maintenance. Today, due to cheap technology, we have large number of videos i.e. surveillance videos which are the main source to capture real time abnormal activities but for automatic detection of different anomalies, we need to design a system that took such surveillance videos as raw input and then produce a useful output.

Challenges

The output means the specific anomaly which needs public and authoritative attention so that such events can be minimized or at least properly dealt with on time. So by using the surveillance videos and incorporating deep learning solutions with the data, we may be able to assist humans to detect anomalies at the spot. Following are the objectives of this work:

• Fully Automation of Surveillance System

• Low quality video problem

• Representataive Dataset

• Reduced false positive rate

Methodology

Code for the methodology is available at: Anomaly Recognition 

Results

• We attempted to address the issue of the high resource requirement of the anomaly recognition method and proposed a lightweight, resource-efficient real-time streaming TAR framework that can be embedded on a simple machine like a central processing unit (CPU).

• We proposed to use temporal learning via partial shift operation to improve spatiotemporal feature based learning. It enables frames to share their learning among adjacent frames and reduce the cost of processing. Moreover, it helps in building feature maps based on high activity areas that support the classification task of anomaly recognition.

• Our framework is capable of performing online anomaly recognition and it allows six simultaneous screens on a CPU-based system while using fewer parameters (2.2M), FLOPs (0.564GFLOPs), model size (0.6Mb) and low latency overhead which proves it to be resource efficient approach.

• Our model achieves 7.87% and 2.47% increased state-of-the-art accuracy with ResNet-50 and MobileNetV2 respectively on UCF-Crime dataset.

Discussion

Automated surveillance is popular area which is continuously improving over the time. Such systems are designed to process huge amount of data which requires a lot of resources which is a challenging requirement. In practical scenarios, time, and cost both are critical to handle and hence a system needs a lot of computation time and resources to serve the purpose. Therefore, this research aimed at addressing these problems through providing a resource-efficient, high-performing system for anomaly recognition. Increased numbers of CCTV generates vast amount of unlabelled video data and its labeling is a difficult task. Moreover, large amount of data requires substantial computational resources to process it. This study provides a lightweight and cost-effective approach for anomaly recognition in terms of memory consumption, processing parameters, and computation time that is essential for a low-resource systems, such as CPU-based systems. The proposed framework is based on 2D convolutional architecture (2D CNN), with a spatiotemporal feature extractor which functions as a partial shift that learns and distributes temporal information among its neighborhood frames. MobileNetV2 baseline performs spatial feature extraction, which is then combined with temporal learning to perform anomaly recognition. Our proposed framework works with low latency rate of 12.01 ms which makes it effective for performing online video recognition and handle up to six streams at once. On the UCF Crime dataset, the proposed framework achieves an accuracy of 88% for binary anomaly recognition problem with time complexity of 0.198 s. On the UCF Crime2Local dataset, the proposed framework achieves accuracy of 52.7 percent for a multi-class problem. The model outperforms previous models in terms of computational parameters requiring 2.2 M parameters and 0.564 GFLOPs with MobileNetV2 as the baseline architecture. Moreover, our proposed framework has achieved an increased accuracy of 2.47% on UCF Crime dataset with reduced computational requirement. Overall, it performs well in a lot of aspects and can be used for realtime recognition but it can be further improved. Some limitations of this study are highlighted in below section to consider in future.

Limitations and Future Directions

We have proposed a resource-efficient anomaly recognition system that effectively performs recognition tasks, but it is not evaluated for object level detection and tracking. Object detection and tracking can significantly improve security surveillance. We believe that with minor modifications to the current model, it could be useful for detection and tracking as well. Our model performs recognition with adequate speed efficiency, but its early response efficiency can be investigated further. It requires validating a model’s ability to perform recognition on a small sample of incoming frames as soon as possible to improve system’s response time. Surveillance systems are designed to perform anomaly recognition as precisely as possible, but there is an additional issue posed by the false positive rate, which can compromise system reliability. To increase the usefulness of our model, we will strive to reduce the false positive rate as much as possible.

Collaborators

Feng Xia

Associate Professor, Data Science,

f.xia@federation.edu.au

Federation University, Australia

Faculty Page 

Google Scholar