Word embeddings, from classic to modern retrieval

Embeddings are dense vectors representing items (words, sentences, users, products). Classical options: count-based: bag-of-words, TF-IDF (sparse, high-dim, no semantic info). Distributed/static: Word2Vec (CBOW/Skip-gram), GloVe (global co-occurrence matrix factorization): 2013-2014 era, captures semantic relations ("king - man + woman = queen"). Contextual: ELMo, BERT-family, OpenAI text-embedding-3: same word gets different vector by context. These power retrieval-augmented generation, recommendation, classification with linear probes, clustering. Senior nuance: modern dense embedders (BGE, E5, Cohere embed-v3, OpenAI text-embedding-3-large) are almost always the right answer for production retrieval. Sparse lexical (BM25) and dense are usually complementary (hybrid wins). Embedding quality degrades under domain shift, so finetune on your data with hard negatives for production.