Your RAG system combines data from APIs updated every minute and PDFs updated monthly. How would you enforce appropriate freshness guarantees?

Production incident (illustrative): A support assistant answers a customer's question about inventory availability and warranty eligibility. Inventory changes every minute through an API; warranty policy PDFs are revised monthly. At 10:07 the assistant says an item has 40 units available, citing an inventory snapshot captured at 09:50, although the API now reports 2 units. It also quotes a warranty policy published three weeks ago, which remains authoritative. Applying the same 24-hour cache TTL to both sources produces a dangerously outdated inventory answer even though the PDF answer is acceptable.

Core principle: Freshness is a per-source, per-field, and per-use-case contract, not a universal re-embedding interval. Distinguish *how old a value is*, *how quickly the source changes*, *whether it is still authoritative*, and *what the user needs at answer time*.

1. Contain and debug the production incident

• For transactional or safety-critical answers, stop relying on a stale vectorized API snapshot. Fetch from the authoritative live API, or abstain with a clear "current data unavailable" message when it fails.
• Inspect ingestion logs, API polling/webhook events, queue lag, dead-letter queues, index acknowledgment times, search read visibility, and cache hit metadata. Trace tenant_id, source_id, record_version, source_updated_at, observed_at, indexed_at, last_verified_at, and response_as_of.
• Establish whether the lag came from an API poll scheduled too slowly, missed change events, throttling, a stuck job, retries, eventually consistent search writes, stale CDN/application caches, or an LLM response cache that ignored source versions.
• Check source-authoritative timestamps instead of assuming indexed_at equals data freshness. An index can be rebuilt today from yesterday's API extract.
• Root cause: a single TTL and a shared ingestion cadence incorrectly treated minute-level API state and slow-changing reference documents as having identical freshness requirements.

2. Define explicit freshness contracts

• Live API facts (inventory, pricing, availability): example freshness SLO ≤ 2 minutes at the point of answering, with a ≤ 30-second target for critical order confirmation. These are illustrative business requirements, not universal defaults.
• PDF policy documents: sync on version/change notifications plus a scheduled reconciliation (for example daily), with a ≤ 24-hour discovery lag goal for revisions, despite normal monthly revision frequency. A PDF being "monthly" does not make every PDF safe to serve for 30 days.
• For each source and field, define max_age, preferred refresh mechanism, allowed stale behavior, error budget, source ownership, revocation priority, and escalation.
• Distinguish freshness (age since the source's last confirmed state) from completeness (all required updates incorporated), authority (which version governs), and latency (retrieval response time). Track both actual event propagation and periodic successful source verification; unchanged sources should not appear permanently stale solely because no update event arrived.
• Time-sensitive requests ("right now", "can I order?") should force live verification. Historical questions may explicitly use an as_of timestamp and versioned snapshot.

3. Engineer two complementary data paths

• Fast-changing API path: ingest authenticated change events/webhooks or CDC where supported; use short interval polling with conditional requests/version tokens as a fallback. Queue events durably, deduplicate by record ID and version, handle out-of-order messages, and retry with exponential backoff + dead-letter queues. Maintain structured current-state storage for numeric attributes; use RAG embeddings only for descriptive/searchable context.
• Slow-changing PDF path: watch source revision IDs, hashes, effective dates, ownership and ACL changes; parse/embed only changed document versions; stage and validate a complete revision, then atomically promote the active version. Run reconciliation to catch missed notifications.
• At query time: classify each retrieved claim by required freshness. For inventory/price, call the live service (or sufficiently fresh authoritative snapshot) *after* retrieval identifies the entity, then generate a cited answer. For policy, retrieve only the current effective document version. Carry source_updated_at, verified_at, version and as_of timestamps through citations.
• Enforce freshness in code: check age before passing data to the LLM. If the source is too old, refresh synchronously within an explicit timeout; if unsuccessful, fail closed for critical fields. An unrestricted stale-while-revalidate response is appropriate only for low-risk content with a clearly communicated age.
• Bound answer caches by the shortest relevant source contract, and use source version/etag plus tenant and authorization scope in cache keys. Invalidate proactively on source changes, deletions and permission revocations.

4. Numerical example: does the result meet its SLO?

• At 10:07:00, the inventory record used by retrieval has verified_at = 09:50:00: 17 minutes old. Its 2-minute max age is exceeded by 15 minutes, so the retrieval middleware rejects it.
• A live API call returns 2 units with a successful check at 10:07:01; the generated answer says "2 units as of 10:07:01". No model-generated extrapolation from the previous 40 units is permitted.
• A warranty PDF was published 21 days ago but the connector last verified its current revision and effective status at 09:00. Its verification age is 67 minutes, inside an illustrative 24-hour source-verification SLO. It remains eligible, provided no revocation/effective-date signal contradicts it.
• In a sample of 10,000 API-answer requests, suppose 9,850 used source-confirmed values no older than 2 minutes. The freshness compliance rate is 9,850 / 10,000 × 100 = 98.5%. If the target is 99.9%, the system misses by 1.4 percentage points, or 140 additional compliant requests for that sample.
• Illustrative propagation budget: 20 seconds source event delivery + 25 seconds processing + 10 seconds read visibility + 15 seconds cache propagation = 70 seconds, below 120 seconds. However, delays compound and percentiles must be measured end to end; component averages do not prove the freshness SLO.

5. Trade-offs and tricky cases

• Live API calls: strongest currency for critical facts but add latency, per-request cost, rate-limit dependence, and downstream outage exposure.
• Event-driven ingestion: low normal lag but missed events and out-of-order delivery require replay, sequence/version fencing and periodic reconciliation. Event-time watermarks help reason about lateness; they do not guarantee that an external API has delivered all updates.
• Frequent polling: simpler when webhooks are absent, but wasteful and vulnerable to quotas; interval alone cannot guarantee a strict maximum age without an explicit lag allowance.
• Aggressive TTL: cheaper and faster but can serve stale results. Short TTL: fresher but creates load and thundering-herd risk; use request coalescing and bounded jitter.
• PDF updated monthly: authoring cadence is not a contractual freshness window; security revocations, access changes, emergency bulletins and effective-date rules must propagate promptly.
• Clock skew and unchanged records: prefer trusted source versions and successful last_verified_at; monitor event-to-observation lag rather than treating old source_updated_at as proof of pipeline failure.
• Cross-source contradictions: show individual timestamps, select authoritative owners by field, and avoid merging values collected at incompatible points in time.
• Critical failures: if source verification times out, say availability is unverified; never silently substitute stale data for order-critical numbers.

6. Prevent recurrence

• Publish a freshness policy registry by source, field and risk class, reviewed with product and data owners.
• Instrument event age, verification age, ingestion lag, index visibility lag, cache staleness, query-time rejection rate, and freshness SLO compliance with per-source percentile dashboards.
• Alert on missing heartbeats, dead-letter queue growth, poll failures, missed source-version changes, and any unauthorized/expired source offered to the model.
• Exercise chaos tests for dropped webhook events, delayed queues, clock skew, API 429/5xx, PDF emergency replacement, ACL revocation and cached answers spanning a change.
• Store source IDs, versions, timestamps and source-of-truth references in an auditable answer trace; periodically compare answered values with direct source reads.

Interview takeaway: Don't choose a single TTL for every RAG source. Use freshness budgets based on the business meaning of each fact: event-driven ingestion plus live verification for high-churn API state, version-aware incremental indexing for PDFs, and a retrieval-time freshness gate that abstains when a critical source cannot be verified.

Further reading

• Confluent, Time and Watermarks in Flink: https://docs.confluent.io/cloud/current/flink/concepts/timely-stream-processing.html
• Confluent, Handle Late-Arriving Data: https://docs.confluent.io/cloud/current/flink/how-to-guides/handle-late-arriving-data.html
• AWS, EventBridge Scheduler retry and dead-letter queues: https://docs.aws.amazon.com/scheduler/latest/UserGuide/managing-schedule.html