Multimodal self-supervised learning is getting more and more attention as it allows not only
to train large networks without human supervision but also to search and retrieve data
across various modalities. In this context, this paper proposes a framework that, starting from
a pre-trained backbone, learns a common multimodal embedding space that, in addition to
sharing representations across different modalities, enforces a grouping of semantically
similar instances. To this end, we extend the concept of instance-level contrastive learning …

Multimodal self-supervised learning is getting more and more attention as it allows not only
to train large networks without human supervision but also to search and retrieve data
across various modalities. In this context, this paper proposes a self-supervised training
framework that learns a common multimodal embedding space that, in addition to sharing
representations across different modalities, enforces a grouping of semantically similar
instances. To this end, we extend the concept of instance-level contrastive learning with a …