Overcoming Label Noise for Source-free Unsupervised Video Domain Adaptation


Avijit Dasgupta
C. V. Jawahar
Karteek Alahari


[Conference Paper]
[Journal Paper]
[GitHub]


Plot of average cross-entropy loss per epoch of training with pseudo-labeled target domain videos for clean vs. noisy samples with (a) RGB modality, and (b) Flow modality. We term the target domain samples with correct pseudo-labels as clean samples and with incorrect pseudo-labels as noisy samples. Note that, the groundtruth labels are only used to identify the clean vs. the noisy samples for visualization purpose and not used for training the model. Deep neural networks learn the clean samples first before memorizing the noisy samples according to the deep memorization effect. In our proposed approach CleanAdapt, we exploit this connection to select the clean samples for fine-tuning the model to adapt to the target domain.

Abstract

Despite the progress seen in classification methods, current approaches for handling videos with distribution-shift in source and target domains remain source-dependent as they require access to the source data during the adaptation stage. In this paper, we present a self-training based source-free video domain adaptation approach (without bells and whistles) to address this challenge by bridging the gap between the source and the target domains. We use the source pre-trained model to generate pseudo-labels for the target domain samples, which are inevitably noisy. We treat the problem of source-free video domain adaptation as learning from noisy labels and argue that the samples with correct pseudo-labels can help us in adaptation. To this end, we leverage the cross-entropy loss as an indicator of the correctness of the pseudo-labels, and use the resulting small-loss samples from the target domain for fine-tuning the model. Extensive experimental evaluations show that our method, termed as CleanAdapt, achieves ∼ 7% gain over the source-only model and outperforms the state-of-the-art approaches on various open datasets.


Methodology


Overview of our CleanAdapt framework

Clean sample selection module





Results and Analysis





Retrieval Results on UCF - HMDB Dataset


Paper

Avijit Dasgupta, C. V. Jawahar, Karteek Alahari
Overcoming Label Noise for Source-free Unsupervised Video Domain Adaptation
ICVGIP 2022 (Best paper runner-up award)

conference paper   /   journal paper   /   code



Acknowledgements

Avijit Dasgupta is supported by a Google Ph.D. India Fellowship. Karteek Alahari is supported in part by the ANR grant AVENUE (ANR-18-CE23-0011).

This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.