Greetings!
I Muhammad Imran Zaman and I am currently doing Masters in Computer Science from COMSATS University Islamabad Lahore Campus Pakistan. Currently I'm doing research on Multi Camera Multi Object Tracking System. A short overview of my research is given below.
Multi-camera multi-object tracking systems are used in visual analyzing and tracking of an object. In this study, multi-camera object tracking with their usage is traced. The purpose of the study is to highlight the importance of tracking in multi-camera. It also analyses the role of computer vision in detection and tracking of an object. This study focuses on the importance of multi-camera tracking to figure out the target object, its direction and position in multiple cameras in automated method. This study also focuses on challenges faced by multi-cameras in object tracking and different contributions made to this study so far. A detailed discussion of state of art in research papers with their methodologies, datasets and evaluation measures is given. This study also observes limitations and results by analyzing existing work done on different datasets of multi-camera object tracking. We aim to develop a deep network based algorithm by using state of the art deep network architectures and also interested to use Resnet101 and Resnet152 trained model for object detection from video streams. Dataset and a tentative timetable of research has been discussed.
Tracking is a technique used to observe the location of moving objects and to provide a timely sequence of their moves for analyzing and processing. Detection of a subject of interest at a place and to find out the passage of that target object, from where it came and in what direction it will further proceeded. The position and texture of the object focuses on the direction in which further it will move where texture gives information about the spatial arrangements of color or selected region of image. Multi-camera system is a setup that is used to observe and secure a place and also find out any legal or illegal activity at crowded places. In these days, this system of multi-camera is pointed at many shopping malls, stadiums, hospitals and at different parts of the cities to make a city safe. But if someone wants to track an object under this multi-camera system, it is a time taking method. In order to track an object under multi-camera, the system is analyzed and checked manually. All the cameras are checked one by one, all the videos are seen manually to track object. This tracking system takes a long time and manpower meanwhile the object moves away from their access. Like in safe city project where multi-camera system is used and each camera covers only a specific area. If any problem occurs then to solve this problem each and every video of all the cameras must be seen manually which is a time taking task and doesn’t define the true meaning of safe city. Target tracking as a branch of computer vision, has been a main field for many years. If an object is seen suspicious in one camera and then it moves, multi-camera managing team will watch all the videos of each camera to trace him/her. The managing team even cannot watch all video streams to find that object while in multi-camera object tracking system, any object can be tracked by using an efficient algorithm of computer vision.
Surveillance Applications
Sports Player tracking
Crowd Behaviour Analysis
Identity Verification
Activity Recognition
Automated CCTV
Automatic Target Recognition
Object and peoples counting
Traffic Applications
Overlapping
Occlusion
Object Presentation
Object Re-identification
To devise a multi-camera object tracking system for overcoming
the challenges of occlusions, illumination and overlapping
by implementing deep neural networks.
Introduction to Motion Estimation with Optical Flow (https://nanonets.com/blog/optical-flow/ ). 12 Min read
Simple object tracking with OpenCV (https://www.pyimagesearch.com/2018/07/23/simple-object-tracking-with-opencv/ ). 10 Min read
Multi-Camera Large-Scale Intelligent Video Analytics with DeepStream SDK(https://devblogs.nvidia.com/multi-camera-large-scale-iva-deepstream-sdk/ ). 10 Min read
Object Tracking in Deep Learning (https://missinglink.ai/guides/computer-vision/object-tracking-deep-learning/ ). 11 Min read
Dataset
Artificial Intelligence has significantly advanced in recent years that lead us towards a “smarter world”. Among much advancement, smart transportation and smart city are some of the most emerging areas. Cameras with computer vision make devices enough intelligent to do analyses and understand the world. To effectively implement Intelligent Transportation Systems (ITS), the “AI City Challenge”. Workshops are organized in order to encourage smart transportation applications developed with artificial intelligence and computer vision. The AI City Challenge 2020 (AIC20) is the fourth sequel of the series of past years.
We selected the Track3 dataset from https://www.aicitychallenge.org/ of 2020 and the details are given below:
46 cameras
215.03 minutes recording of all cameras
Dimensions 1920x1080
FPS 60
Outdoor recorded data on the roads
900 vehicles
Real world data
There are overlapping of vehicles
There are also blind spots
We used the state of art techniques to overcome the existing challenges and for the better results. There are the following steps of our proposed method:
The system first detects and track the multiple vehicles from the each cameras (the annotations are provided by the AI city challenge)
The system save the tracklets of each vehicle within the single camera
After that, the saved tracklets of each camera re-identifies among the other cameras and keep tracklets same for all the unique vehicles across all cameras
Aggregation loss calculated for the re-identification of the vehicles
We set hard positive and hard negative criteria to check the similarity from the images. Hard negative defines the image of a vehicle which is quite similar to the images of other vehicles. Hard positive means the image of a vehicle which is quite different from the other images of this vehicles.
We tried many backend architectures for the object detection and classification like VGG16, VGG19, Resnet50, Resnet101 and currently experiments are running on the Resenet152.
Results
AI City Challenge only concern with the IDFI score and also evaluate the result of test on leaderboard. Suppose IDTP represents number of true positive IDs, IDTN represents number of true negative IDs, IDFP represents number of false positive IDs and IDFN represents number of false negative IDs.
IDF1 can be represented as:
IDF1 = 2IDT P 2IDT P + IDF P + IDF N
We got the 4th position on the AI City Challenge leaderboard with 0.4623 IDF1 score.
here is the screenshot of our results on the AI City Challenge leaderboard.
Paul Voigtlaender, Michael Krause, Aljosa Osep, Jonathon Luiten, Berin Balachandar Gnana Sekar, Andreas Geiger, Bastian Leibe "MOTS: Multi-Object Tracking and Segmentation," CVPR, 2019.
Ergys Ristani Carlo Tomasi, “Features for Multi-Target Multi-Camera Tracking and Re Identification,” cs.CV, no. 28 March 2018, p. 11, 2018.
Hang Qiu, Xiaochen Liu, Swati Rallapalli, Archith J. Bency, Kevin Chan, “Kestrel: Video Analytics for Augmented Multi-Camera Vehicle Tracking,” 2018 IEEE/ACM Third International Conference on Internet-of-Things Design and Implementation, vol. 10.
Yonatan Tariku Tesfaye, Eyasu Zemene, Andrea Prati, , Marcello Pelillo, Mubarak Shah, “Multi-Target Tracking in Multiple Non-Overlapping Cameras using Constrained Dominant Sets,” Cs.CV, 2017.
Solera, Francesco, Simone Calderara, Ergys Ristani, Carlo Tomasi, and Rita Cucchiara. "Tracking social groups within and across cameras." IEEE Transactions on Circuits and Systems for Video Technology 27, no. 3 (2016): 441-453.
Kim, Yookyung, and Kee-Seong Cho. "Robust multi-object tracking to acquire object oriented videos in indoor sports." In 2016 International Conference on Information and Communication Technology Convergence (ICTC), pp. 1104-1107. IEEE, 2016.
N. Saunier and T. Sayed, “Automated analysis of road safety with video data,” Transp. Res. Rec., vol. 2019, no. 1, pp. 57–64, 2007.
[71] M. Naphade et al., “The 4th AI City Challenge,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020, pp. 626–627.
P. Köhl, A. Specker, A. Schumann, and J. Beyerer, “The MTA Dataset for Multi Target Multi Camera Pedestrian Tracking by Weighted Distance Aggregation,” 2020.
Co-PI, Video Analytics lab, National Centre in Big Data and Cloud Computing, Program Chair (FIT 2019), HEC Approved PhD Supervisor.
Assistant Professor & Associate Head of Department, Department of Computer Science, COMSATS University Islamabad, Lahore Campus, Pakistan
MS (Scholar), Department of Computer Science, COMSATS University Islamabad, Lahore Campus, Pakistan.
