Awesome Self-Supervised learning

This repository contains the Self-Supervised learning paper lists

Overview

• introduction
• Contrastive Learning
• Self-Prediction
• Distillation
• Clustering

Introduction

The paper lists of Self-Supervised Learning

Contrastive Learning

• SimCLR: "A Simple Framework for Contrastive Learning of Visual Representations", ICML, 2020 [paper] [code] [summary][review]

• SimCLRv2: "Big Self-Supervised models are Strong Semi-Supervised Learners", NeurIPS, 2020 [paper] [code] [summary]

• MoCo: "Momentum Contrast for Unsupervised Visual Representation Learning", CVPR, 2020 [paper] [code] [summary]

• BYOL: "Bootstrap Your Own Latent : A New Approach to Self-Supervised Learning", arXiv, 2020 [paper] [code] [summary]

• MoCo v3: "An Empirical Study of Training Self-Supervised Vision Transformers", ICCV, 2021 [paper] [code] [summary]

• DenseCL: "Dense Contrastive Learning for Self-Supervised Visual Pre-Training", CVPR, 2021 [paper] [code] [summary]

• SEER: "Self-supervised Pretraining of Visual Features in the Wild", arXiv, 2021 [paper] [code] [summary]

Self-Prediction

• Context Prediction: "Unsupervised Visual Representation Learning by Context Prediction", ICCV, 2015 [paper] [code] [summary]

• Predicting Image Rotations: "Unsupervised Visual Representation Learning by Context Prediction", CVPR, 2020 [paper] [code] [summary]

Distillation

• Knowledge Distillation: "Distilling the Knowledge in a Neural Network", NIPS, 2014 [paper] [code] [summary]

• DistilBERT: "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter", NIPS, 2019 [paper] [code] [summary]

• Noisy Student: "Self-training with Noisy Student", CVPR, 2020 [paper] [code] [summary]

Clustering

• DeepCluster: "Deep Clustering for Unsupervised Learning of Visual Features", ECCV, 2018 [paper] [code] [summary]
• SeLA: "Self-labelling via simultaneous clustering and representation learning", ICLR, 2020 [paper] [code] [summary]
• SwAV: "Unsupervised Learning of Visual Features by Contrasting Cluster Assignments", NeurIPS, 2020 [paper] [code] [summary]
• SLIC: "SLIC: Self-Supervised Learning with Iterative Clustering for Human Action Videos", CVPR, 2022 [paper] [code] [summary]