Embedded Files
People
Organisations
Maritime Situational Awareness
Australian maritime trade
Australian maritime trade
Our simulation of Sydney Harbour
Our simulation of Sydney Harbour
Civil maritime pressures
Civil maritime pressures
Definition of maritime situation awareness
Definition of maritime situation awareness
Understanding and anticipating the meaning and future status of maritime elements and events with respect to time or space
Project's goal
Maximising situational awareness via bridging the domain gap of
different environmental conditions
simulated and real images
visible and infrared images
To this end, we propose to employ domain adaptation as our primary strategy.
Research question
We recognise there are two fundamental research questions: "How to adapt?" and "When to adapt?".
For the question "How to adapt?", we propose "Human-in-the-loop test-time domain adaptation" method
Regarding the question "When to adapt?", we suggest employing "Assessing the domain gap" approach.
Both methodologies will be detailed in the following sections.
Human-in-the-loop test-time domain adaptation
Novelty
(a) Previous works have focused on developing fully autonomous solutions, primarily for self-driving vehicles. (b) Our approach, however, is proposed for visual surveillance, which are typically monitored by an operator. Therefore, our method will take advantage of the operator's involvement in the adaptation process. Also, In data-driven approaches, data can be viewed as a vital asset of businesses, thus our online adaptation will be source free to avoid data to be stolen. Furthermore, to significantly reduce labour cost of the human operator, our method only requires weak labels for online adaptation; see (c) for the definitions of full and weak labels in object detection
(a) Previous works have focused on developing fully autonomous solutions, primarily for self-driving vehicles. (b) Our approach, however, is proposed for visual surveillance, which are typically monitored by an operator. Therefore, our method will take advantage of the operator's involvement in the adaptation process. Also, In data-driven approaches, data can be viewed as a vital asset of businesses, thus our online adaptation will be source free to avoid data to be stolen. Furthermore, to significantly reduce labour cost of the human operator, our method only requires weak labels for online adaptation; see (c) for the definitions of full and weak labels in object detection
Methodology
Methodology
Faster-RCNN is used as the object detector. In particular, given each image, the weak labels of human operator is used to compute the image-level loss and instance-level loss. These losses are then used to update the batch normalisation layers of the network.
Faster-RCNN is used as the object detector. In particular, given each image, the weak labels of human operator is used to compute the image-level loss and instance-level loss. These losses are then used to update the batch normalisation layers of the network.
Results
Results
Assessing domain gap for continual domain adaptation
Overview of our approach
RetinaNet is pre-trained on source images with full supervision
The outputs from different blocks C3 , C4 , and C5 are used to evaluate the domain gap between the source and target images
Correlation between the domain gap and object detection accuracy
Correlation between the domain gap and object detection accuracy
1st row is results of DGTA dataset , where source domain is clear conditions from 9h to 15h (i.e., daytime images) and target domains are overcast conditions in 24 hours (i.e. nighttime, sunrise, daytime, and sunset)
2nd row is results of BDD dataset, where source domain is clear-daytime, and target domains are clear-night, cloudy-daytime, overcast-daytime, rainy-daytime, rainy-night, snowy-daytime, and snowy-night.
"AP discrepancy" represents the drop of object detection accuracy
"MMD", "DSS", and "SWD" are three different metrics to measure the domain gap
The x-axis represents different target domains.
We can see there is a strong correlation between the domain gap and detection accuracy. In particular, if the drop of detection accuracy is significant then the domain gap is large, and vice versa.
Therefore, we apply "domain gap" as a criterion to decide when to adapt.
Application of domain gap evaluation in domain adaptation
Application of domain gap evaluation in domain adaptation
For each target domain, the current RetinaNet is used to evaluate the domain gap between the current training data and target domain (without the need of labels). If the domain gap is found to be small, the target domain is discarded, and no adaptation is needed. However, if the domain gap is found to be large, target domain is added to the current training database to refine the current RetinaNet, resulting in a new RetinaNet model which replaces the current RetinaNet in the operating environment.
Benefits of domain gap evaluation (DGE) in domain adaptation (DA)
Benefits of domain gap evaluation (DGE) in domain adaptation (DA)
DGTA dataset: Target domain is sequentially changed 0h-1h ➜ 2h-3h ➜… ➜ 22h-23h
KITTI-Fog dataset: Target domain is sequentially changed fog-750m ➜ fog-375m ➜ fog-150m ➜ fog-75m ➜ fog-30m
BDD dataset: Target domain is sequentially changed clear-night ➜ cloudy-daytime ➜ overcast-daytime ➜ rainy-daytime ➜ rainy-night ➜ snowy-daytime ➜ snowy-night
Using domain gap as a criterion to decide when to adapt will sacrifices 1% accuracy but saves 50%-90% energy usage for continual domain adaptation
Relevant publications
Relevant publications
A.-D. Doan, B. L. Nguyen, S. Gupta, I. Reid, M. Wagner, and T.-J. Chin. "Assessing domain gap for continual domain adaptation in object detection". Computer Vision and Image Understanding (CVIU 2024). [arxiv] [code]
A.-D. Doan, B. L. Nguyen, T. Lim, M. Jayawardhana, I. Reid, M. Wagner, T.-J. Chin. "Human-in-the-Loop Test-Time Domain Adaptation for Object Detection". Under review
Page updated
Google Sites
Report abuse