The FDE Playbook: Delivering an Enterprise RAG Assistant in 16 Weeks

A client asks for "a RAG chatbot over our policy documents." A vendor demos a chatbot on sample PDFs. A Forward Deployed Engineer (FDE) does something different: they find the real work, the pain and the money, then own the outcome from the first call to a signed handoff. This is a distilled, end-to-end playbook for running one enterprise policy-RAG engagement in 16 weeks — the exact kind of work we train for in our FDE PRO program.

The worked example below uses an illustrative, fictional North-American insurance carrier. Insurance policy documents carry every hard RAG problem — editions, state rules, endorsements that change a base form, and real regulatory risk — which makes them a perfect teaching case. The method applies equally to HR/corporate policy, legal, healthcare or any versioned-document domain.

"Chatbot over PDFs" is the client's guess. Find the outcome.

In discovery, "a RAG chatbot" became a measurable outcome: correct, cited coverage answers in under a minute, inside the agent desktop, run by the client's own team. Discovery found complex questions took reps 6–9 minutes, 22% were escalated, and QA caught coverage-explanation errors on 12% of audited calls. That's the difference between shipping a system and delivering a result.

The FDE mindset

An FDE is measured by the outcome the client gets, not the system shipped. The principles that drive every decision:

• Problem before product — "RAG" is a guess at a solution; find the work and the money first.
• Go where the work happens — shadow real users answering real questions.
• Define "correct" before you build — the golden set and grading rubric are the real specification.
• Ship a walking skeleton early — ugly end-to-end beats polished parts that don't connect.
• The data is the product — parsing, editions, metadata and access rules decide quality more than the model.
• Trust is a feature — cite every claim, quote the clause, and say "I don't know."
• Own the outcome, not the ticket — chase access, security reviews and training as hard as code.
• Bad news early, in writing — raise a risk the day you see it, with a fix and a clear ask.
• Scope like a surgeon — start narrow, widen on evidence.
• Leave them stronger — the client can run, change and evaluate the system without you.

Vendor reflex vs. FDE move

• First call — vendor demos a chatbot; FDE asks about the last hard question a rep got and what a wrong answer cost.
• Accuracy — vendor says "our model is 95% accurate"; FDE says "on your 400 questions, graded by your SMEs, we're at 93% — here are the 28 misses and why."
• Problems — vendor raises them at the monthly steering meeting; FDE raises them the same day, in writing, with options.
• Handoff — vendor sends a PDF and a farewell call; FDE has the client's engineers run the last release while we watch.

The shape: 8 phases, 6 gates, 16 weeks

Every phase ends at a gate the client signs before the next begins:

1. Prepare — research the business; walk in already knowing their states, products and regulators.
2. Discover (2-week paid sprint) — shadow users, harvest 800+ real questions, profile the corpus, build a throwaway prototype. Gate: a one-page, baselined problem statement.
3. Scope — a solution-options memo (recommend what's right for the client, even if it's less work for you) and a Statement of Work. Gate: signed SOW.
4. Design — architecture + security/responsible-AI pack + evaluation plan. Gate: architect + security + compliance approve.
5. Build — six one-week sprints, each ending in a demo with an eval report. Gate: pilot release gates met on a blind test set.
6. Test & pilot — UAT, then a 3-week pilot measured against a matched control group. Gate: production gates + go/no-go.
7. Deploy & operate — phased rollout, runbooks, hypercare.
8. Handoff — the client's team runs a release and an incident drill without you. Gate: acceptance certificate.

Why policy documents are hard RAG

A naïve pipeline (PDF → embeddings → vector DB → LLM) scored 61% on the prototype — and seven in ten misses were wrong edition or a missing endorsement. That one finding set the architecture's first priority:

• Editions — several editions of a form are in force at once; older policies legitimately use older wording, so you filter by effective date, never delete.
• State & jurisdiction — amendatory endorsements change the base form per state (and per province for Canada).
• Endorsements — an endorsement modifies a base form, but the link between them often isn't recorded anywhere.
• Access — some content (e.g. underwriting authority limits) must never reach a front-line rep.

Design around the failures, not a reference diagram

The architecture's first job is to answer from the right edition, state and endorsements, then prove every claim with a checked citation. Key decisions:

• Resolve edition, state and endorsements before searching — with rules, not the model.
• Hybrid search (lexical + vector) + reranking — form numbers and defined terms need exact match; paraphrases need semantics.
• Structure-aware chunking — split by section/clause; keep definitions and whole tables as their own chunks; prefix each chunk with its breadcrumb.
• Verify every citation before the answer is shown; if support is missing, abstain and route — an abstention is a correct answer.
• Enforce access at retrieval time, never in the prompt.
• No fine-tuning — knowledge changes monthly and must be cited; retrieval and prompts carry the domain logic. Choose the model last, by evaluation.
• Deploy inside the client's cloud tenant — no client data leaves it; no data trains any model.

The evaluation plan is the real specification

The golden set and release gates are agreed with compliance before you have results to defend (a threshold set after you see the numbers is a negotiation, not a standard). The set grows from 100 questions in discovery to 400 before the pilot, SME-graded on a simple rubric (2 = correct and cited, 1 = partial, 0 = wrong; a "critical flag" for anything that could mislead a customer). Example production gates: correctness ≥ 92%, critical-error rate ≤ 0.5%, citation accuracy ≥ 98%, edition/state accuracy ≥ 99%, abstention accuracy ≥ 95%, every adversarial case passed. A gate missed on any one slice blocks the release — even if the average passes.

Security & responsible AI, in week 4 — not week 12

Take the threat model and data-flow diagram to security early; reviewers who see the design early become co-owners, not blockers. The LLM threats are mapped to the OWASP Top 10 for LLM Applications (prompt injection, sensitive-information disclosure, improper output handling, excessive agency, and more), and the engagement is registered in the insurer's AI-governance inventory with its intended use, testing and monitoring. The assistant supports a licensed person; it never makes a coverage, claims or underwriting decision — and the design enforces that.

Build in the open, pilot as an experiment

Demo working software on real data every week, including what broke — a client who watches the numbers climb never needs a status meeting to trust you. Then run the pilot as an experiment, not a launch party: 40 reps against a matched control group turns "reps like it" into "handle time fell 17% against matched reps" — a number a sponsor can defend.

Leave them stronger

Handoff is a process that starts in week 5, not a meeting at the end: see one, do one, teach one. We build while the client's engineer pairs, they lead while we watch, then they run the production release and an incident drill themselves. You're done when you're no longer needed.

Learn to do this — FDE PRO at Cloud Soft Solutions

Every capability in this playbook — RAG, embeddings, hybrid retrieval, agents, MCP, evaluation, guardrails, cloud deployment and the customer-facing engineering mindset — is hands-on curriculum in our programs:

• FDE PRO — AI Forward Deployed Engineer: take AI from a customer's problem to production, with real enterprise projects.
• APEX — AI, ML, Cloud & Cyber Security and AI, GenAI & Agentic AI training in Hyderabad.
• See the method in practice: how we built Aanya AI with RAG and the FDE roadmap.

Frequently asked questions

What is a Forward Deployed Engineer (FDE)?

An engineer who works close to the customer, understands their real problem, builds and deploys the solution into their environment, and owns the business outcome — blending engineering, AI and customer skills.

Why is enterprise RAG harder than a demo?

A demo answers from a few clean PDFs. Enterprise RAG must handle document versions/editions, jurisdiction rules, access control, citations, abstention, security (prompt injection, data residency), evaluation and monitoring — the "last mile" that decides whether it's usable and safe.

Can I learn to deliver engagements like this?

Yes — the FDE PRO and APEX programs teach exactly this, end to end, with real projects and placement support.