How do you update or delete indexed documents in a RAG vector database?

When a source document changes or is deleted, its indexed chunks, embeddings, and metadata must be synchronized so the RAG system does not retrieve outdated or removed information.

Updating a document

A reliable update workflow is:

1. Identify the source document using a stable document_id, along with its tenant or namespace.
2. Detect the change using a version number, content hash, file event, or updated_at timestamp.
3. Parse and re-chunk the latest document. If the content changed, regenerate embeddings for affected chunks using the currently configured embedding model.
4. Upsert updated records into the vector database using stable chunk IDs where possible. An *upsert* inserts missing records or updates existing ones.
5. Remove obsolete chunks that no longer exist, especially if editing changed the number or boundaries of chunks.
6. Verify retrieval returns the new content and no longer surfaces the old version.

If chunk boundaries change significantly, replacing all indexed chunks belonging to the document may be simpler and safer than guessing which old chunks correspond to new ones.

Example: updating a policy PDF

Suppose leave_policy.pdf originally produces five chunks. After revision, it produces four.

• Reprocess the updated PDF and create four new embeddings.
• Upsert the four current records, preserving document_id and adding a new version.
• Delete the fifth old record, plus any other obsolete chunk IDs.
• Query a changed policy statement to confirm that only the latest version is available.

Important: Updating four chunks does not automatically remove the old fifth chunk. That needs explicit cleanup or a document-version replacement strategy.

Deleting a document

Deleting a source file does not necessarily delete vectors that were created from it.

To remove a document from retrieval:

1. Identify all chunk IDs associated with its document_id and the correct tenant or collection.
2. Delete those vectors by ID, or use a supported metadata-filter delete operation.
3. Remove or invalidate separately stored chunk text, cached retrieval results, and related references as required.
4. Verify that queries no longer return the deleted content.

Different databases expose different update/delete APIs. For example, some support deletion by record ID and by metadata filter; others require tracking the affected IDs.

Avoid stale or partially updated results

For production RAG, consider:

• Versioning: Index a new document version before switching a version filter or alias to make it active.
• Consistency: Some vector services apply writes asynchronously, so verify the change after it becomes visible.
• Idempotency: Design ingestion jobs so retrying an upsert or deletion does not produce duplicate records.
• Failure recovery: Track job status and retry partial failures rather than leaving old and new versions mixed.
• Access control: Scope update and delete operations to the verified tenant and authorized documents.
• Auditability: Record which document version was indexed, when, and by which ingestion job.

Interview takeaway

To update an indexed document, reprocess its changed content, regenerate the necessary embeddings, upsert current chunk records, and delete obsolete chunks. To delete a document, remove every associated vector and invalidate any related cached or stored content. Stable IDs, document versions, tenant scoping, and post-update verification prevent stale retrieval results.