Every US insurer has to file its products and prices with state regulators before it can sell. Those filings are public — but they live as multi-document PDF packets on SERFF, a portal that is a workflow, not a database. I built the platform that turns them into a fifty-state answer to one question: who is selling what, where, at what price, and on which terms. No one assigned this. I found the problem and built it.
Before a carrier launches a group dental PPO in Ohio or reprices a vision rider in Texas, product and pricing teams need the competitive set: which carriers are approved, which forms they filed, what the rate pages actually say, and which waiting periods, age bands, and participation rules they locked in. The answers exist — every licensed insurer files them — but the record is unreadable at scale.
SERFF search is a multi-step form that expires sessions and rejects naive crawlers. A filing is not one PDF; it is a packet of policy forms, rate manuals, actuarial memoranda, redlines, and correspondence. Product names do not travel between states, rate factors hide in scanned tables, and the provisions that move price sit in different exhibits depending on the state and the year.
So teams do it by hand: an analyst pulls a handful of packets and pastes fragments into a workbook. Coverage is late and incomplete. Competitive pricing is a sample, not a market.
A production data platform, not a scrape plus a chat window. It collects the packets, splits and classifies the documents, extracts a product, rate, and provision ontology, resolves carriers and products into one entity graph across states, and serves the result as a warehouse, an API, a comparison workbook, and a dashboard.
Every extracted field carries a citation — document, page, verbatim snippet. If the source is silent, the field is empty; nothing is invented to fill a schema. A go-to-market team filters by state, line, carrier, and effective date, then opens a cited field rather than a chat transcript.
Collection is headed Playwright against SERFF and the state portals that never joined it. The worker keeps a real browser session, walks the multi-step search, retries on timeout, and checkpoints after every filing so a job resumes without re-walking the queue. Temporal owns the orchestration — collect, parse, extract, resolve, publish. Raw packets land in S3; filing metadata lives in Postgres.
The document pipeline splits each packet, classifies form types, OCRs the scans, and runs a layout-aware parse so tables keep their structure. Extraction is schema-constrained: Pydantic models define the ontology, and the extraction call has to fill them or fail closed. Agents on the OpenAI Agents SDK run extract, normalize, and enrich under guardrails that refuse any field that cannot point at a page and a snippet. LiteLLM routes the model per stage — cheaper for classification, stronger for actuarial memos. Entity resolution is a second pass: embeddings propose that two carrier names are the same thing; deterministic rules and a graph of known aliases decide.
Quality is measured, not assumed. A labeled gold set of filings scores field-level precision and recall on every pipeline change; Phoenix traces each run; Promptfoo regressions sit in CI so a prompt edit cannot silently drop waiting periods or invent a rate factor.
Python, Playwright, Temporal, AWS (S3, EC2), Postgres, Pydantic, OpenAI Agents SDK, LiteLLM, Phoenix, Promptfoo, pdfplumber, Tesseract, pytest.
Insurers have used the platform to go to market and to price against what competitors actually filed, not against last year’s rumor — state by state, with a citation behind every cell. A few million dollars of projects have run on it: competitive scans ahead of a launch, rate reviews when a peer files a change, and form-by-form comparisons when a state objects to language someone else already got approved.