Self-Supervised Learning

2 articles

Artificial Intelligence 09 Sep 2026 10 min read

Detect Representation Collapse in Self-Supervised Learning

Self-supervised learning can train an encoder without manually assigning a class label to every example. But removing labels also removes an obvious force that tells different examples to occupy meaningfully different parts of representation space. A badly designed objective can therefore admit a trivial solution: the encoder maps many or all inputs to essentially the same representation. This failure is called representation collapse. The training loss may even look good, because a model that emits the same vector for two augmented views of every input has achieved perfect agreement without learning useful distinctions.

Artificial Intelligence Machine Learning

Artificial Intelligence 08 Sep 2026 9 min read

Masked Autoencoders for Visual Representation Learning

Labeled image datasets are expensive to build, but unlabeled images are often plentiful. A useful pretraining strategy is therefore to create a learning signal from each image itself instead of asking a human to annotate it. A masked autoencoder (MAE) does this by hiding part of an image and training a model to reconstruct the missing content. The reconstruction task is not usually the final product. Its purpose is to make the encoder learn visual representations that can later support tasks such as classification or detection.

Artificial Intelligence Computer Vision