Tag: retrieval
All the articles with the tag "retrieval".
Agent Memory: Write, Read, Update, Forget
Updated:An agent's context is its working memory; everything else is storage with a retrieval policy. A taxonomy of agent memory, the four operations and the rule each needs, Generative Agents' recency, importance and relevance score worked, MemGPT's tiers, forgetting, memory as files, and privacy. Then retrieval policies measured on 250 conversations whose facts change, with ablations: how often each shows the model an outdated fact, and what fixing that costs.
Building RAG: Chunking, HyDE and Caching
Updated:Most RAG failures happen before generation: the chunk that holds the answer is never retrieved. Eight ways to cut this blog into chunks, plus a control, measured on 150 questions by whether the answer's sentence comes back; HyDE, measured; an embedding cache keyed by model and text; a semantic cache's hits and wrong answers at six thresholds; and when RAG is the wrong tool.
Embeddings and Contrastive Learning: Making Similar Mean Close
Updated:An embedding is only as meaningful as the loss that made it, and the data it saw. Word2vec trained on this blog, worked and measured, including the famous analogy test and why it fails here; cosine vs dot product; InfoNCE as cross-entropy over one positive and many negatives, with the temperature and batch size explained; a contrastive head on frozen word2vec vectors, which adds about six and a half points of recall on this little data; and pretrained sentence encoders, one of which matches every English section to its Vietnamese translation.
Semantic and Hybrid Search: BM25, Dense Retrieval and Rank Fusion
Updated:Keyword search matches the words a query uses; dense retrieval matches what it means. BM25 derived and worked on this blog's sections, dense retrieval and its truncation, reciprocal rank fusion, and the metrics. Then a benchmark built from the blog itself, English and Vietnamese: fusion helps in English and hurts in Vietnamese, diacritics matter, and technical Vietnamese can be searched with English keywords, with bootstrap intervals.