Before a Forward Deployed Engineer (FDE) writes much code, they write a solution design document. It is the page the customer's architects, security team and business owner sign off on, and it is what keeps a project from drifting. Good FDEs write short, specific design docs; weak ones skip them and pay for it in rework. This guide explains each section and gives you a template to copy.
The design doc usually follows discovery (see the customer workshop playbook and AI use-case discovery) and comes before the build phase of a POC-to-production engagement.
What a good design doc does
- States the business problem and how success will be measured.
- Shows the architecture clearly enough for the customer's security team to review.
- Lists the decisions made and the options rejected, with reasons.
- Names the risks and what will happen if they occur.
- Fits in 4–8 pages. Longer documents don't get read.
The template, section by section
1. Summary (half a page)
Problem, proposed solution, expected outcome and the decision you need from the reader. Example: "Support agents spend 6 minutes searching policies per ticket. We propose a policy assistant with cited answers inside the ticketing tool, targeting 50% less search time in the pilot group. We need approval to index the policy SharePoint site."
2. Scope
In scope, out of scope, and users. Being explicit about what you won't build in phase 1 prevents most scope creep.
3. Success metrics
| Type | Example |
|---|---|
| Business | Average handling time, tickets resolved without escalation |
| Quality | Groundedness and answer correctness on an agreed evaluation set |
| Operational | p95 latency, availability, cost per request |
| Adoption | Weekly active users in the pilot group |
Agree the evaluation set with the customer's subject experts early. See LLM evaluation.
4. Architecture
One diagram plus a numbered request flow, for example:
- User asks a question in the ticketing tool.
- The API checks the user's identity and groups.
- Retrieval runs hybrid search filtered by the user's permissions.
- The model generates an answer with citations; guardrails check the output.
- The response, sources and trace ID are logged.
Name the actual services (for example Amazon Bedrock and OpenSearch, or Microsoft Foundry and Azure AI Search) and where each one runs.
5. Data
Sources, owners, volume, update frequency, formats, sensitivity and retention. Note how deletions in the source reach the index.
6. Security and compliance
- Authentication and document-level authorisation.
- Data residency and which model endpoints are used.
- PII handling, encryption, network isolation (private endpoints).
- Prompt-injection defences and tool permissions for agents. See AI guardrails and enterprise AI security.
7. Key decisions
| Decision | Chosen | Rejected | Why |
|---|---|---|---|
| Grounding approach | RAG | Fine-tuning | Policies change monthly; citations required |
| Search | Hybrid + reranking | Vector only | Users search exact policy codes |
| Agent autonomy | Suggest-only | Auto-actions | Pilot phase; human approval required |
8. Operations
Monitoring, alerting, on-call, index refresh jobs, model version changes, and the cost estimate (see LLM cost estimation for FDEs). Link the handover runbook you'll deliver.
9. Rollout plan
Phases with entry and exit criteria: internal test, pilot group, wider rollout. Include the rollback plan.
10. Risks and open questions
Each risk with likelihood, impact, mitigation and owner. Open questions with the person who must answer and the date needed.
Copy-paste skeleton
Title / version / owner / reviewers / date
1. Summary: problem, solution, outcome, decision needed
2. Scope: in, out, users
3. Success metrics: business, quality, operational, adoption
4. Architecture: diagram + numbered request flow
5. Data: sources, owners, sensitivity, refresh, deletion
6. Security: authN/authZ, residency, PII, network, guardrails
7. Key decisions: chosen / rejected / why
8. Operations: monitoring, refresh, model changes, cost
9. Rollout: phases, exit criteria, rollback
10. Risks and open questions: owner + date
Common mistakes
- Metrics like "improve productivity" with no baseline or number.
- Architecture diagrams with no data flow or security boundaries.
- No statement of what happens when the model is wrong.
- Ignoring how permissions in source systems carry over to retrieval.
- No owner for open questions, so they stay open.
Practise it
Write a design doc for one of the FDE projects before you build it, and include it in your portfolio. Interviewers rarely see candidates who can do this. Our FDE PRO program includes design reviews on every enterprise project.
📞 Call +91 96660 19191 · WhatsApp · Flat No. 513, 5th Floor, Nilagiri Block, Aditya Enclave, Ameerpet, Hyderabad
Frequently Asked Questions
What is a solution design document?
The document the customer's architects, security team and business owner approve before the build: problem, scope, metrics, architecture, data, security, decisions, operations, rollout and risks.
How long should an AI design document be?
Usually 4 to 8 pages. Longer documents tend not to be read.
Which success metrics belong in it?
One or more each for business outcome, answer quality, operations (latency, availability, cost) and adoption.
Why record rejected options?
So reviewers see the trade-offs, for example RAG chosen over fine-tuning because policies change monthly and citations are required.
What are common design-doc mistakes?
Vague metrics with no baseline, diagrams without data flow or security boundaries, no plan for wrong answers, and open questions without owners.
