Papers
Browse the full research library.
Ten papers per page, with summaries, diagrams, formulas, concepts and citations kept close to the source.
Precise Zero-Shot Dense Retrieval without Relevance Labels
Introduces Hypothetical Document Embeddings (HyDE), a query-transformation technique for zero-shot dense retrieval without relevance labels.
HyDE is a practical example of using an LLM to transform a query into a retrieval-friendly representation before dense search.
Luyu Gao, Xueguang Ma, Jimmy Lin
Read paper
Atlas
Shows that retrieval-augmented language models can learn knowledge-intensive tasks effectively in few-shot settings.
Atlas connects retrieval augmentation with few-shot learning and demonstrates that external memory can reduce the need to encode all task knowledge in model parameters.
Gautier Izacard, Patrick Lewis, Maria Lomeli
Read paper
Improving Language Models by Retrieving from Trillions of Tokens
Shows that retrieval from a 2-trillion-token database can substantially improve language modeling while using far fewer model parameters than similarly capable dense models.
RETRO demonstrates retrieval as a scalable alternative to simply increasing model parameter count and introduces a practical architecture for integrating massive external memory into Transformers.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann
Read paper
Fusion-in-Decoder
Introduces FiD, which scales multi-passage QA by encoding retrieved passages independently and fusing them inside the decoder.
FiD showed that a generative reader can effectively combine many retrieved passages and became a major architecture for open-domain QA and retrieval-augmented generation.
Gautier Izacard, Edouard Grave
Read paper
Dense Passage Retrieval for Open-Domain Question Answering
Introduces a practical dual-encoder dense retriever that replaces sparse lexical matching with learned semantic passage representations.
DPR became the standard retriever initialization for early RAG systems and demonstrated that dense retrieval can outperform strong BM25 baselines on open-domain QA.
Vladimir Karpukhin, Barlas Oğuz, Sewon Min
Read paper
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Defines the canonical RAG architecture that combines a dense retriever, an external document index, and a seq2seq generator trained end-to-end.
This paper is the foundational reference for modern retrieval-augmented generation systems and the RAG-Sequence/RAG-Token formulation.
Patrick Lewis, Ethan Perez, Aleksandra Piktus
Read paper
REALM
Introduces end-to-end retrieval during language-model pre-training so factual knowledge can live in an external, replaceable corpus rather than only in model parameters.
REALM established an important early design pattern for retrieval-augmented models: learn retrieval jointly with language modeling and use external documents as explicit memory.
Kelvin Guu, Kenton Lee, Zora Tung
Read paper
OpenForgeRL
OpenForgeRL connects real agent harnesses, remote environments, and standard RL infrastructure so agents can be trained in deployment-like conditions.
Read this if you are building production agents and want to understand how the training stack can match the actual inference harness instead of training against a simplified approximation.
Xiao Yu, Baolin Peng, Ruize Xu
Read paper
Gemini 4 Argon
Gemini 4 Argon targets long-horizon, multimodal professional workflows with a 1M-token output ceiling and a staged safety-first rollout.
A practical breakdown of Google's latest frontier-model announcement, its reported evaluations, safety design, and engineering implications.
Koray Kavukcuoglu
Read paper
Transformer
The original Transformer paper: architecture, attention mathematics, training recipe, benchmark evidence, ablations, and engineering trade-offs.
Read it before modern decoder-only LLM papers, RoPE, KV cache, GQA, FlashAttention, MoE, and modern normalization variants.
Ashish Vaswani, Noam Shazeer, Niki Parmar
Read paper