A Forward Deployed Engineer's (FDE's) job isn't finished when the AI system goes live. It's finished when the customer's own team can run it without you. A clean handover (runbooks, ownership, monitoring and training) is what separates a successful deployment from a system that quietly degrades after the FDE leaves. This guide gives you the checklist and a runbook template.
Handover is the last phase of an engagement that starts with discovery and the solution design document. See also POC to production and an FDE's first 90 days.
Why AI handovers are harder than normal software
- Quality can drift without errors. New documents, changed policies or a model version update can make answers worse while every health check stays green.
- More moving parts: ingestion jobs, indexes, prompts, model endpoints, guardrails and tools all need owners.
- Prompts are code. Someone must own changes to them, with review and tests.
- Costs scale with usage and need watching.
Handover checklist
Ownership
- A named owner for the application, the data sources, the index, prompts and evaluation, and the cloud account.
- Escalation path and on-call rota agreed with the customer's operations team.
Documentation
- Final architecture and request flow (update the design doc to match what was built).
- Configuration reference: environment variables, model and deployment names, index names. Secrets referenced by vault location only, never written in documents.
- Runbooks for the incidents below.
- Known limitations and the backlog of agreed future work.
Monitoring and alerts
- Availability and p95 latency per endpoint.
- Error rates for model calls, retrieval and tools, including throttling.
- Ingestion job success and index freshness.
- Token usage and cost against budget.
- Quality signals: user feedback rate, "I don't know" rate, scheduled evaluation scores. See AI observability.
Quality process
- The evaluation set lives in the repository and runs in CI on every prompt or configuration change.
- A monthly review of low-rated answers with subject experts.
- A procedure for model version upgrades: run the evaluation set first, compare, then switch. See AI agent evaluation.
Security
- Access reviews for the app, the index and the cloud resources.
- Logging and retention settings confirmed with the customer's compliance team.
- Guardrail configuration documented. See AI guardrails.
Knowledge transfer
- Walkthrough sessions with the support team, recorded.
- The support team handles at least one simulated incident before go-live sign-off.
- A shadow period where they run operations and the FDE only advises.
Runbook template
Write one short runbook per likely incident. Each follows the same shape:
Title: e.g. "Answers are outdated"
Symptoms: what users or alerts report
Checks: 1) ingestion job status 2) index document count / last update
3) source system changes 4) recent prompt or config changes
Fix: step-by-step commands or console actions
Escalate to: owner + contact, and when to escalate
Verify: how to confirm it's fixed (e.g. re-run 10 evaluation questions)
Runbooks every AI system needs
| Incident | First checks |
|---|---|
| Answers outdated or missing documents | Ingestion job logs, index freshness, source permissions changed |
| Wrong or ungrounded answers increase | Recent prompt or model change, evaluation scores, retrieval results for failing questions |
| Slow responses | Model latency and throttling, retrieval latency, number of agent steps |
| Model errors or throttling (429) | Quota and rate limits, retry settings, fallback model |
| Cost spike | Requests per user, tokens per request, runaway agent loops, cache hit rate |
| Suspected data exposure | Disable the feature, preserve logs, check retrieval filters, follow the customer's incident process |
| Tool or API failures in agents | Downstream API health, credentials expiry, schema changes |
Sign-off
Close with a short handover document signed by the customer's owner: what was delivered, metric results against the targets in the design doc, open items with owners, and the support arrangement after handover. This also becomes your best evidence when you describe the project in interviews (see the FDE résumé and portfolio guide).
Learn it in a real project
The capstone in our FDE PRO program ends with a handover pack: runbooks, dashboards and an evaluation pipeline. See all courses and a day in the life of an FDE.
📞 Call +91 96660 19191 · WhatsApp · Flat No. 513, 5th Floor, Nilagiri Block, Aditya Enclave, Ameerpet, Hyderabad
Frequently Asked Questions
Why are AI handovers harder than normal software handovers?
Quality can drift with no errors, there are more moving parts (ingestion, indexes, prompts, models, guardrails, tools), prompts need change control, and costs scale with usage.
What should an AI handover include?
Named owners, final architecture and configuration docs, runbooks, monitoring and alerts, an evaluation pipeline, security reviews and recorded knowledge-transfer sessions.
Which runbooks does every AI system need?
Outdated answers, rising wrong answers, slow responses, model throttling, cost spikes, suspected data exposure and tool or API failures.
How do you handle model version upgrades after handover?
Run the evaluation set against the new version, compare scores, then switch, with a rollback plan.
When is the handover complete?
When the customer's team has run operations through a shadow period and the owner has signed off on delivered results and open items.
