The model produces an answer using evidence from two different financial reporting periods. How would you enforce temporal consistency?
Production incident (illustrative): A finance RAG assistant receives “What was Helios Components' FY2026 operating margin?” It retrieves an annual-report passage giving FY2026 revenue = ₹1,200 crore (April 2025–March 2026) and a near-identical comparative table giving FY2025 operating profit = ₹180 crore (April 2024–March 2025). It then calculates 180/1,200 = 15%, cites both genuine passages and confidently labels the result “FY2026 operating margin.” The FY2026 operating profit, from a different page, was ₹144 crore: the correct FY2026 margin is 144/1,200 = 12%. The generated arithmetic is correct but the join between facts is invalid. Both snippets were retrieved, indexed and authorized; the failure is cross-period evidence assembly and answer synthesis.
Principle: A financial fact is not merely (metric, value, document). It must include reporting entity, accounting basis, period start/end or instant, fiscal calendar, unit, currency, segment/dimensions, filing/version and restatement status. Two true source facts are not necessarily composable. Enforce a typed temporal fact contract before any cross-chunk aggregation, arithmetic, comparison or citation. SEC EDGAR's XBRL API and XBRL specifications represent reporting contexts explicitly; revenue and operating profit are duration facts, while many balance-sheet amounts are instant facts.
1. Contain the production incident and define the intended period
- Do not return a mixed-period KPI as a confident financial answer. If an exact period-aligned numerator or denominator is missing, say what is unavailable or ask which period the user wants; never silently borrow a nearby fiscal year.
- Capture the user question; trusted reporting entity ID; financial-reporting calendar and timezone; requested metric; canonical fiscal year/quarter;
period_start/period_end; whether the question requests an *as-of balance*, *duration flow*, trailing-twelve-month amount, growth comparison or restated figure. - Trace retrieval through the final prompt: query, search filters, index generation, each chunk's source/version, document title, table row/column headers, extractable time context, model output, calculated formula and per-number citations.
- Identify authoritative ground truth for FY2026: a canonical accounting data API or filing table with matching 2026 period, unit and consolidated/standalone scope. For a public issuer, SEC's companyfacts XBRL data can supply facts with date ranges and filing metadata; do not treat an XBRL calendar frame as a universal company fiscal quarter without date inspection.
- Respect tenant and document ACLs, retention, embargo and source effective dates. Diagnostic widening to an unauthorized filing is never a valid fix.
2. Separate the clocks that commonly get confused
- Reporting period: the actual interval for a flow fact, e.g.
2025-04-01..2026-03-31for FY2026 revenue; or a single2026-03-31instant for closing cash. - Fiscal period label: an entity-specific
FY2026,Q4 FY2026orH1. The same fiscal label across companies does not guarantee identical dates. Some organizations use calendar years; many Indian companies close 31 March; always inspect dates. - Publication/filing date: when a company published or filed the statement, possibly weeks after the period ended. A 2026 publication can contain FY2025 comparative values; a 2025 publication can reference forward-looking 2026 forecasts.
- Data-valid and system-observed time: a restatement may be issued later but relate to an earlier reporting period. Record
period_end,filed_at,ingested_at,restatement_versionseparately instead of using one ambiguousdatefield. - Comparative column: an FY2026 annual report may have FY2026 and FY2025 side by side. Document-level metadata
fiscal_year=2026is insufficient to label every numeric cell FY2026. - Instant vs duration: balance-sheet values as of a date are not comparable by naive addition to period flows. XBRL describes
instantanddurationas distinct period types; validatestartandendwhen applicable. - Quarter-to-date / year-to-date: FY2026 Q2 YTD revenue covers April–September, while standalone Q2 revenue covers July–September. Do not mix period lengths, even if both are labelled “Q2.”
- Restated versus originally reported: restated prior-period facts can be intentionally used when a user requests restated comparisons. Choose a documented basis consistently and label the version; avoid silently mixing original and restated numbers.
3. Root-cause debugging sequence
- A. Query-time interpretation: confirm the planner parsed “FY2026” into Helios' exact fiscal calendar, accounting basis and requested KPI. Check query rewrite and chat-history carryover, especially “last year” relative to the company's reporting period rather than current wall clock.
- B. Source metadata: inspect whether every candidate numeric passage has a
fact_periodand source version. Missingperiod_start/period_endis a provenance defect, not an invitation to assume document year. - C. Table extraction: compare the original PDF/table with extracted cells and their column headers. OCR and markdown conversion can flatten “FY2026 / FY2025” into text with ambiguous numeric associations. Test header inheritance across pages and repeated table headings.
- D. Index granularity: see whether the chunk contains multiple comparative columns or includes a 2025 number under a 2026 document title. A one-label-per-chunk metadata design cannot express mixed-period facts safely.
- E. Retrieval filters: confirm the intended period filter is applied to a fact-level index or correctly typed records. A parent-document filter alone may admit both 2026 and 2025 figures.
- F. Selection/reranking: trace which candidate carried each of the two arithmetic inputs; check if the FY2026 numerator existed lower in rank but was dropped during rerank/packing. Rerank with matching metric and period, not only name and semantic similarity.
- G. Context assembly: inspect whether table headers, units, footnotes, currency and period labels survived compression. Repeated data blocks may be deduplicated incorrectly across years.
- H. Arithmetic reasoning: log parsed inputs as structured fact IDs. Recompute derived metrics outside free-form generation. Reject a formula if operands have incompatible periods, inconsistent entity/consolidation scopes, mismatched units or currency.
- I. Output provenance: require each number in the answer to carry its own source fact ID and period. A citation to the annual report alone does not show that the cited row and fiscal column match the claim.
- J. Failure classification: source extraction mismatch, fact metadata missing, period filter error, wrong rerank choice, context header loss, unsupported cross-period join, or answer/citation mismatch. Fix the first observed boundary rather than adjusting the LLM prompt blindly.
4. Typed fact model and compatibility guard
- Represent a fact as
{entity_id, metric_id, value, unit, currency, period_type: instant|duration, start_date?, end_date, fiscal_calendar_id, fiscal_year, fiscal_quarter?, period_length_days?, reporting_scope, accounting_standard, segment_dimensions, source_id, accession_id?, filed_at, restatement_version, provenance_span}. - Prefer authoritative structured XBRL facts where available. For PDF-only filings, extract candidate rows with inherited headers and validate against the source. Do not infer source periods from document names, embedding similarity, current dates or timestamps alone.
- Same-period KPI guard: for a margin
profit/revenue, both operands must be duration facts covering the *same start and end* dates and comparable accounting/entity scopes. Ensurerevenue ≠ 0; preserve units, percentage scaling and rounding rules. - As-of guard: for a debt balance or cash balance, require instant facts as of the requested date, with applicable basis. Comparing cash at two dates is valid when expressly labelled as a change, not as a same-period arithmetic input.
- Comparison guard: year-on-year growth intentionally uses two different comparable periods, but only under an explicit comparison plan:
(FY2026 revenue − FY2025 revenue)/FY2025 revenue; validate corresponding fiscal durations and restatement policy. Cross-period mixing is wrong when accidental, not when requested and labelled. - Rolling period guard: for a trailing-12-month calculation, sum non-overlapping, contiguous quarters or obtain a verified TTM fact; never sum YTD and full-year values that overlap.
- Restatement guard: record whether prior-year comparative is restated and whether current-year values have been revised; choose authoritative current restated reporting consistently rather than assuming latest filing date equals latest period.
- Cross-entity guard: an issuer's group report can include parent, subsidiary and consolidated columns. Bind
entity_id,reporting_scopeandsegment_dimensionsbefore joining. This complements Q270's entity-resolution work. - Ambiguity guard: if two trusted sources disagree on period boundaries, definitions or restatements, surface an explicit conflict and request clarification or route to accounting review.
5. Beginner-friendly numerical production example
- The FY2026 Helios annual report provides revenue ₹1,200 crore and operating profit ₹144 crore for 1 April 2025 to 31 March 2026. The adjacent FY2025 comparative column provides revenue ₹1,000 crore and operating profit ₹180 crore for 1 April 2024 to 31 March 2025.
- The broken RAG pipeline extracts FY2026 revenue
1200, FY2025 operating profit180, and computes 180/1200 × 100 = 15%, incorrectly calling it FY2026 operating margin. - A period-consistency gate rejects this join because
profit.period=[2024-04-01,2025-03-31]differs fromrevenue.period=[2025-04-01,2026-03-31]. - The corrected join uses FY2026 facts: 144/1200 × 100 = 12%. For an explicit comparison, FY2025 operating margin was 180/1000 × 100 = 18%; FY2026 margin is 6 percentage points lower, which is a legitimate cross-period comparison because each margin was first calculated within its own consistent period.
- For a hypothetical 200-question evaluation set involving reported numbers, 160 questions have a single requested fiscal period and 40 require deliberate year-over-year comparisons. The baseline produces valid period-aware, correctly supported answers for 126/200 = 63%.
- A fact-level date/column extractor, typed fiscal context, compatibility checks and targeted refetch improve illustrative supported-answer success to 178/200 = 89% on the same frozen, authorized queries, an absolute 26-percentage-point increase. There are still 22 failures needing review of table parsing, restatement scope, source availability or generation.
- Suppose a further 12 requests with ambiguous fiscal/calendar mapping are held for clarification rather than answered; report answer rate, correct supported answer rate, period-mismatch escape rate and abstention separately. The 178/200 figure is a controlled illustration rather than a claim about current MachinoAI traffic.
- A structured fact lookup could increase retrieval-and-validation p95 from 170 ms to 245 ms in this example. Accept the 75 ms overhead only if measured latency, cost, security and correctness gates meet service objectives.
6. Engineering solution and safe deployment
- Fact-level indexing: index numeric facts or table cells with exact source row/column provenance plus normalized temporal metadata, rather than relying on one reporting year per PDF chunk. Keep the original excerpt and page/span for human verification.
- Authoritative fiscal calendar registry: maintain an effective-dated calendar per reporting entity or business unit; store FY labels and actual start/end dates. Handle 52/53-week years, nonstandard reporting quarters and fiscal-calendar changes explicitly.
- Source lineage: version facts by filing accession/document version and
filed_at. Retain restated facts and superseded values with explicit precedence instead of overwriting the reporting period itself. - Typed retrieval planner: interpret query → resolve entity/metric/scope/period intent → form authorized fact search constraints → retrieve exact period-aligned facts → fetch surrounding textual disclosure/explanations where needed.
- Two-tier retrieval: use hybrid semantic/BM25 for descriptive narrative and section discovery, then authoritative structured lookup for numeric operands and their reporting contexts. Do not ask a semantic reranker to guarantee equality of financial periods.
- Validated calculation service: compute margin, growth, average or TTM with domain-specific compatibility rules. Reject invalid period joins with a typed error such as
PERIOD_MISMATCH,INSTANT_DURATION_MIX,OVERLAPPING_INTERVALS,INCOMPARABLE_SCOPEorRESTATEMENT_CONFLICT. - Evidence manifest: attach
fact_id,entity,metric,period_start/end,period_type,currency,unit,scope,source_version, and citation span to each input and calculated output. - Narrative answer: state the fiscal period and context: “For FY2026 (1 Apr 2025–31 Mar 2026), operating margin was 12%, calculated from revenue ₹1,200 crore and operating profit ₹144 crore.” Cite each operand/source and label comparative figures explicitly.
- Uncertain evidence: when facts are missing, mixed, or unauthoritative, perform a bounded refetch with the correct period and source owner. If reconciliation fails, abstain or show the periods separately; never invent or silently backfill a numerator from another year.
- Rollout: shadow-test against audited filings/structured facts, canary by fiscal calendar and document type, and gate on period alignment, correct calculations, version precedence, source citations, access controls, p95/p99 and cost. Preserve rollback that cannot resurrect revoked filings or unauthorized records.
7. Trade-offs and common mistakes
- One document-year metadata field: easy to implement but wrong for comparative tables, quarterly/YTD columns and restated numbers. Use fact-level period attribution.
- Automatic “latest” selection: latest filing can contain a historical value. Ask what period the user means and inspect the XBRL fact's actual start/end or instant, not just
filed_at. - Date filters alone: narrow the candidate corpus but cannot prove the chosen two values are compatible. Add post-retrieval typed join validation.
- XBRL structured facts: high-quality temporal contexts where available, but concepts, dimensions, units, fiscal calendars and restatement semantics still require interpretation; some narrative financial facts lack clean tags.
- PDF tables: broad coverage but OCR/table parsing can detach headers, units and comparative periods. Require source-span verification and quality gates.
- Strict same-period joins: essential for margins, but would incorrectly reject legitimate cross-year growth and cash-balance changes if the operation type is ignored. Model the permitted temporal relation per calculation.
- Period-length equivalence: equal number of days does not guarantee the same fiscal quarter or comparable seasonal basis. Explicitly label and review nonstandard comparisons.
- Currency conversion: same fiscal dates but different currencies or units still need conversion rules and conversion-date provenance; never assume two numbers share a unit.
- Restatement changes: a later filing may correct an older fiscal value; choosing “newest filed” indiscriminately can mix standards or scopes.
- Single tokenized chunk for whole table: can accidentally treat adjacent FY2025 and FY2026 cells as one number sequence. Separate rows/columns while retaining source references.
- LLM arithmetic: model-generated formulas can be correct on wrong inputs. Typed operand validation and deterministic calculations provide a more auditable boundary.
- Latency: structured lookup and validation add I/O/CPU but can prevent expensive policy and investor-facing errors; measure selective activation for factual numeric questions.
8. Prevention, monitoring and regression suite
- Build a temporal golden set across calendar years, FY labels, Q1–Q4, half years, YTD vs quarter standalone, TTM, closing balances, comparative columns, restatements and cross-company fiscal calendars.
- Include explicit negative cases: FY2026 numerator + FY2025 denominator, 2026 publication containing 2025 comparative, Q2 YTD + Q2 standalone, 2026 consolidated profit + 2026 standalone revenue, and FY2025 original versus restated values.
- Alert on
PERIOD_MISMATCH,MISSING_FACT_PERIOD,COLUMN_HEADER_LOST,INSTANT_DURATION_MIX,UNVERIFIED_RESTATEMENTand unsupported derived metrics; monitor by ingestion source and extraction version. - Track period-aligned fact recall, cross-period join rejection rate, period-mismatch escape rate, correctly supported calculation rate, period-label citation correctness, abstention, latency and confidence intervals on labelled audit samples.
- Require every numeric extracted fact to retain a reporting context and exact source citation; reject index writes if a mandatory date or table-column binding cannot be established, or quarantine the fact for manual review.
- Regression-test on each document loader, OCR, chunking, table extractor, financial-calendar mapping, query parser, model and prompt change. Verify both fact-level contexts and full answer arithmetic.
- Preserve an audit record: original request, resolved temporal intent, candidate fact IDs, rejected joins and reasons, chosen operands, equation, output, versions and citation spans.
- Use source-controlled, reviewed changes to fiscal calendars and precedence. A source reindex should not automatically make expired or differently dated facts eligible for an unrelated query.
Interview takeaway: When a RAG answer merges two financial reporting periods, inspect the reporting period of every individual numerical fact, not just the file year or publication date. Make the join a typed operation that checks entity, period start/end, instant/duration, metric, scope, units and restatement context *before* calculation. Cross-period comparisons are valid only when explicitly requested, individually period-consistent and clearly labelled.
Research references
- SEC EDGAR APIs and XBRL Company Facts: https://www.sec.gov/search-filings/edgar-application-programming-interfaces
- XBRL International, *Dates in XBRL* (instants and durations): https://www.xbrl.org/dates-in-xbrl/
- FASB, *Period Type Selection* guidance: https://xbrl.fasb.org/resources/Style_Guides/periodtypeselection.pdf
- Microsoft Azure AI Search, *Text Query Filters*: https://learn.microsoft.com/en-us/azure/search/search-filters
- Microsoft Azure Architecture Center, *RAG Information Retrieval*: https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/rag/rag-information-retrieval
Learning Sequence
Retrieval-Augmented Generation Curriculum
A RAG assistant gives different answers to the same question across repeated requests. How would you separate retrieval nondeterminism from generation nondeterminism?
Explore Related Questions
Deepen your understanding across related Retrieval-Augmented Generation topics.