Tag: contrastive-learning
All the articles with the tag "contrastive-learning".
Embeddings and Contrastive Learning: Making Similar Mean Close
Updated:An embedding is only as meaningful as the loss that made it, and the data it saw. Word2vec trained on this blog, worked and measured, including the famous analogy test and why it fails here; cosine vs dot product; InfoNCE as cross-entropy over one positive and many negatives, with the temperature and batch size explained; a contrastive head on frozen word2vec vectors, which adds about six and a half points of recall on this little data; and pretrained sentence encoders, one of which matches every English section to its Vietnamese translation.
Seeing with Transformers: ViT, CLIP and Vision-Language Models
Updated:Cut an image into patches and it becomes a sequence a Transformer can read; train images against their captions and a picture and its description land near each other in one space. Patches and the patch embedding worked for 224×224 images; the class token, pooling and the inductive bias ViT gives up; CLIP's symmetric contrastive loss and SigLIP's sigmoid loss computed on a 3×3 batch; vision-language models built from an image encoder, a projector and an LLM, and what their image tokens cost in KV cache; and CLIP tested on this blog's own diagrams, where it matches by reading their labels, and a rendering mistake first made it look much better.