Persistent Reasoning Engine for Evolving Enterprise Knowledge Models

By Amar Kumar

A proposed persistent reasoning engine that ingests evolving public documents — SEC filings, annual reports, shareholder letters, earnings transcripts, and news — and maintains a continuous enterprise knowledge model over time. Each new document would update existing enterprise state rather than produce an isolated summary. Version 1 would validate the architecture on multi-year Microsoft and Amazon annual reports using Python, FastAPI, PostgreSQL, and LLM-driven structured extraction.

Proposed outcome: A working prototype that extracts doctrine, capabilities, obligations, risks, management decisions, and causal relationships into a versioned enterprise graph — with year-over-year diffs and trajectory views, not stock recommendations.

Scenario

This brief describes a proposed solution — not a delivered engagement. It maps a pattern for builders who need organizational continuity across documents, not per-document RAG answers.

• Platform: Research or intelligence product maintaining evolving enterprise models from public filings
• Owner profile: Technical founder or systems engineer who values architecture over API glue; prototype is validation for a larger multi-domain reasoning platform
• Scope: SEC 10-K, 10-Q, 8-K; shareholder letters; earnings transcripts; PDF and HTML parsing; persistent state; temporal comparison
• V1 milestone: Ingest Microsoft and Amazon annual reports across multiple years; extract structured enterprise objects; demonstrate state evolution

Problem

Document summarization and RAG pipelines fail the continuity requirement:

• Isolated summaries — each filing becomes a standalone blob; no merge semantics when Q3 guidance contradicts Q1 statements
• No enterprise ontology — free-text summaries do not encode doctrine, obligations, or causal links in queryable form
• Lost temporal context — systems cannot answer how stated risk posture changed from 2022 to 2024
• Re-ingestion amnesia — re-summarizing all documents on each update is expensive and non-deterministic; diffs are unexplainable
• Entity drift — the same capability described differently across years is not reconciled to a stable enterprise object ID
• Wrong output shape — buyers need a structured enterprise model and trajectory, not investment advice or bullet-point abstracts

Summarization vs persistent reasoning

Requirements

Functional

• Ingest SEC filings (10-K, 10-Q, 8-K), shareholder letters, earnings transcripts from PDF and HTML
• LLM extraction into typed enterprise objects: doctrine, capabilities, active states, active obligations, risks, management decisions, causal relationships, enterprise trajectory
• Persistent enterprise record per organization with stable object IDs and provenance (source doc, section, span)
• Temporal comparison: current state vs prior years; change log with add, update, supersede, and retire events
• Reconciliation when new documents conflict with or refine prior extractions
• Query API returning structured state and diffs — not recommendations

Non-functional

• Deterministic merge rules where possible; LLM used for extraction, not ad-hoc state mutation
• Full audit trail: every object field links to source evidence
• Idempotent ingestion (re-run same filing does not duplicate state)
• PostgreSQL as system of record; optional graph layer for causal edges
• FastAPI services with OpenAPI contracts for prototype consumers
• Eval harness on extraction quality and merge correctness

Architecture

Four layers: ingestion normalizes source documents; extraction produces candidate objects; reconciliation merges into authoritative enterprise state; temporal API serves state snapshots and diffs.

flowchart TB classDef ingest fill:#dbeafe,stroke:#2563eb,color:#1e3a8a classDef process fill:#ede9fe,stroke:#7c3aed,color:#5b21b6 classDef data fill:#ccfbf1,stroke:#0d9488,color:#115e59 classDef api fill:#f1f5f9,stroke:#64748b,color:#334155 classDef ext fill:#f8fafc,stroke:#475569,color:#334155 EDGAR["SEC EDGAR\n10-K · 10-Q · 8-K"]:::ext PDF["PDF / HTML\nletters · transcripts"]:::ext ING["Ingestion service\nparse · sectionize"]:::ingest EDGAR --> ING PDF --> ING EXT["Extraction agent\nLLM structured output"]:::process REC["Reconciliation engine\nmerge · supersede"]:::process ING --> EXT EXT --> REC PG[("PostgreSQL\nenterprise_objects · versions\nprovenance · snapshots")]:::data GRAPH[("Graph layer\nAGE or Neo4j\ncausal edges")]:::data REC --> PG REC --> GRAPH DIFF["Temporal diff service"]:::process API["FastAPI\nstate · diff · graph"]:::api PG --> DIFF GRAPH --> DIFF DIFF --> API

Architecture — ingestion, extraction, reconciliation into versioned enterprise state, temporal diff API

sequenceDiagram autonumber participant S as Scheduler participant I as Ingestion participant E as Extraction participant R as Reconciliation participant DB as PostgreSQL participant G as Graph participant API as FastAPI S->>I: new 10-K available I->>I: parse PDF sections I->>DB: store document + spans I->>E: section batches + state context E->>E: LLM extract candidates E->>R: validated objects R->>DB: load current snapshot R->>R: merge plan R->>DB: apply version + provenance R->>G: upsert causal edges R->>DB: freeze snapshot API->>DB: diff vs prior year API-->>API: trajectory response

Ingestion sequence — each filing updates authoritative state rather than appending a summary

End-to-end flow

Happy-path flow from new public document to updated enterprise model and year-over-year comparison

Recommended stack

Recommendation: FastAPI and PostgreSQL as the core, LLM structured extraction for candidate objects, Apache AGE or Neo4j for causal graphs, and pgvector only for evidence retrieval during reconciliation.

Why not RAG-only? Retrieval returns chunks; it does not maintain authoritative merged state. Vectors support evidence lookup during reconciliation, not replace the enterprise record.

Why PostgreSQL-first? Enterprise objects are relational — obligations reference capabilities; decisions reference risks. Temporal versioning maps cleanly to snapshot tables. A graph extension adds causal edges without abandoning SQL operations.

Component design

1 — Ingestion service

• Input: CIK or ticker, filing type, date range
• Output: normalized documents rows with parsed sections, tables, metadata
• Tools: EDGAR fetch, PDF/HTML parser, section classifier (Item 1A Risk Factors, MD&A, etc.)

2 — Extraction agent

• Input: document sections + current enterprise state summary (for coreference)
• Output: candidate objects per schema with confidence and evidence spans
• QA gates: schema validation, required fields, citation to source offset

3 — Reconciliation engine

• Input: candidate objects + existing enterprise state
• Output: merge plan: create, update, supersede, retire
• Rules: stable IDs via embedding similarity + label match; explicit supersession chains

4 — Temporal diff service

• Input: organization ID, two snapshot dates
• Output: structured diff per category; optional trajectory narrative grounded in diff JSON only

Core reconciliation event schema — merge actions are explicit and auditable:

from enum import Enum
from pydantic import BaseModel

class MergeAction(str, Enum):
    CREATE = "create"
    UPDATE = "update"
    SUPERSEDE = "supersede"
    RETIRE = "retire"

class ReconciliationEvent(BaseModel):
    object_id: str
    category: str
    action: MergeAction
    prior_version_id: str | None
    evidence_doc_id: str
    evidence_span: tuple[int, int]

Implementation plan

Reporting & ops

Prototype dashboard: extraction failures by section, merge action distribution, obligation and risk count trends per fiscal year.

Proposed deliverables

Following the phased plan above, a V1 build would ship these artifacts:

• FastAPI service with ingestion, enterprise state, and temporal diff endpoints
• PostgreSQL schema for versioned enterprise objects, provenance, and snapshots
• LLM extraction pipelines for SEC 10-K sections (extensible to 10-Q and 8-K)
• Reconciliation engine with supersession chains and stable object IDs
• Microsoft and Amazon multi-year prototype corpus with demonstrated trajectory diffs
• Causal relationship graph export for management decisions and risks
• Eval harness with golden extractions and merge scenario tests

Effort estimate

Indicative engineering effort for the V1 prototype (assumes two companies × three or more years of 10-K as validation corpus; human review of merge rules during the first month):

The first milestone is a working prototype, not a polished application — scope is deliberately bounded to prove persistent state updates across years.

Glossary