FHIR + LLM: A Practical Integration Pattern for Clinical Document AI

Status: Satellite draft for Healthcare RAG hub. Distinct from the FHIR vs HL7 standards-comparison post — this one is about *integrating LLMs with FHIR data*, not about choosing the standard. Body to be drafted; outline locked. Target length: 1,500 words. Schema: Article + FAQPage.

If you're building clinical document AI and your data source is an EHR, you'll touch FHIR. The question is not whether — it's how to make FHIR resources useful as both retrieval input and structured output for a RAG system. This post covers the patterns we ship in production.

Why FHIR + LLM is harder than people expect

Three reasons:

1. FHIR resources are not natural-language documents. They're typed, nested JSON. An LLM trained on text doesn't index them well as embeddings without preprocessing.
2. Real-world FHIR has profile drift. Epic's Patient is not Cerner's Patient. The optional fields are populated differently.
3. The output direction is harder than the input direction. Going *from* FHIR to a summary is easy. Going *from* a summary back *into* a FHIR resource (so it can update the EHR) requires structured generation with strict schema validation.

The integration topology

Diagram (writer to add): EHR FHIR API → SMART on FHIR auth layer → FHIR caching/normalization layer → embedding layer (with FHIR-aware preprocessing) → vector store → LLM → structured-output layer → FHIR PUT/POST back to EHR.

Each arrow is a real engineering problem. We'll walk through them.

Pattern 1: FHIR resources as retrieval source

The naive approach: stringify the FHIR JSON, embed it, retrieve. This works badly because:

• Field names dominate the embedding. "code", "system", "valueQuantity" appear in every Observation.
• Nested references (Patient → Encounter → Condition) are lost in flat embedding.
• Coded values (LOINC, SNOMED, ICD-10) are codes, not natural language.

The pattern that works: render FHIR resources as natural-language summaries before embedding. Writer to render a code sample:

// Pseudocode placeholder — writer to fill in real implementation
function renderObservationForEmbedding(obs: Observation): string {
  // "On 2026-03-14, patient's hemoglobin A1C was 7.2 (high). Reported by Dr. Chen at Tampa General."
  return `On ${formatDate(obs.effectiveDateTime)}, patient's ${codeToName(obs.code)} ` +
    `was ${formatValue(obs.valueQuantity)} ${interpretReferenceRange(obs)}. ` +
    `Reported by ${obs.performer?.display} at ${obs.performer?.organization?.display}.`
}

The natural-language rendering preserves clinical meaning, normalizes coded values, and embeds well. Keep both: the rendered string for embedding, and a pointer back to the original FHIR resource for retrieval-time hydration.

Pattern 2: FHIR resources as structured output

When the LLM needs to *write* back to the EHR (e.g., creating a DocumentReference for a generated discharge summary), use structured generation with schema enforcement:

• Constrained generation against the FHIR JSON schema (use response_format: json_schema with Bedrock/Anthropic, or Tool Use with a strict schema).
• Required fields enforced at generation time, not at validation time.
• US Core profile compliance verified post-generation with fhir-validator before any PUT to the EHR.

Writer to render a code sample for generating a DocumentReference resource with all required fields, including subject, author, content.attachment.url, and context.encounter.

Pattern 3: SMART on FHIR auth — the gotchas

• Token storage: server-side only, never in localStorage. Refresh token rotation enforced.
• Scope discipline: patient/Observation.read not patient/*.read. The audit log is what makes this enforceable.
• Launch context: the EHR-launched flow injects patient and encounter IDs into the token; build the assistant assuming they're authoritative.

Pattern 4: Profile drift across EHRs

Epic FHIR ≠ Cerner FHIR ≠ athenahealth FHIR. The patterns we ship:

• A FHIR adapter layer that normalizes incoming resources to a canonical internal shape before the LLM sees them.
• Per-EHR test fixtures captured from the sandbox (open.epic.com, code.cerner.com).
• Profile-specific prompts when the LLM needs to *generate* output that will be consumed by a specific EHR.

Internal link to: /blog/fhir-vs-hl7 for the underlying standard comparison.

Pattern 5: When NOT to use FHIR

Sometimes the right answer is: don't put FHIR on the LLM's critical path.

• For free-text clinical notes (rich text in DocumentReference.content), retrieve via FHIR but treat the content as text, not as FHIR.
• For high-volume HL7 v2 message processing, the integration engine (Mirth, Rhapsody) is still the right place — write to FHIR after enrichment, not before.
• For workflows where the EHR is read-only to the AI system, skip the SMART on FHIR auth complexity and use a service-account read-only flow with logging.

How this lands in practice

A representative prior-authorization workflow: the system pulls Patient, Coverage, MedicationRequest, and recent Observation resources as patient context. Those resources are rendered as natural-language clinical summaries. A separate retrieval pass fetches payer policy text from a vector index. The LLM generates a draft authorization rationale grounded in both. A pharmacist reviews and approves before anything is submitted. The audit log records FHIR resource references — not the resource values — to keep PHI out of unnecessary log copies.

The same FHIR-as-context pattern applies to any retrieval-grounded legal or document AI workflow where structured records need to feed an LLM without raw JSON verbosity. See the legal deposition RAG case study for a close parallel in a different domain.

FAQ

1. Can we just give the LLM the raw FHIR JSON?
2. Which FHIR version should we target — R4 or R5?
3. How do we handle FHIR resources that span millions of records?
4. What's the latency impact of FHIR adapter layers?
5. How do we keep retrieved FHIR data in sync with the source EHR?
6. Can we cache FHIR data in a HIPAA-aligned way?

Where we fit

CTA — Cal.com architecture review, link to HIPAA-aligned RAG cornerstone, link to healthcare AI consulting.