FDE engineer interview questions test whether you can do the job on day one: write working code against messy data and flaky APIs, design a RAG or agent system that respects enterprise identity and security, deploy it on a customer's cloud, prove it works with evaluation, and handle a difficult customer conversation without losing the room. This guide collects 150 high-value forward deployed engineer interview questions with model answers: 80 core questions organised by topic and interview round, 40 advanced real-world scenarios on 30-day delivery, live escalations, secure agents, Kubernetes, Terraform and senior AI architecture, 30 extended questions on production Python, RAG evaluation, LangGraph, Bedrock and Microsoft Foundry, and portfolio projects, plus a 12-week preparation roadmap.
How to use this guide:
- Freshers and early-career engineers: master the fundamentals, coding and GenAI sections, and be ready to defend every decision in your own projects.
- Experienced developers, DevOps and cloud engineers: expect depth in RAG, agents, integration, identity and cloud, and at least one full system design.
- Senior candidates: interviewers weight the troubleshooting, stakeholder and system design scenarios heavily and look for trade-offs, evaluation and operations in every answer.
- Practise answers out loud. The wording here is a model, not a script.
Contents:
- The FDE interview process, round by round
- FDE fundamentals: the role and how it differs
- Customer discovery and requirements
- Practical coding questions (Python, APIs, data handling)
- GenAI and LLM fundamentals
- RAG interview questions for FDEs
- Agentic AI, tools, MCP and A2A
- Enterprise integration and identity
- Cloud (AWS and Azure) for FDE interviews
- Security and governance
- Evaluation, observability and AgentOps
- Production troubleshooting scenarios
- Customer-facing and stakeholder scenarios
- System design scenarios
- Advanced scenarios: delivery, escalations and customer pressure
- Advanced practical coding scenarios
- Advanced cloud, Kubernetes and Terraform scenarios
- Senior AI architecture scenarios
- Extended questions 121–150: Python, RAG depth, LangGraph, cloud and projects
- 12-week FDE interview preparation roadmap
- Key takeaways
- Interview preparation checklist
- FAQ
The FDE interview process, round by round
Loops differ between AI labs, enterprise software vendors, global capability centres in Hyderabad and Bengaluru, and services firms, so treat this as a common shape rather than any one company's process. After a recruiter screen, most FDE loops include some version of these rounds:
| Round | Format | What it really tests | Questions in this guide |
|---|---|---|---|
| Practical coding | Live coding or a short take-home: parse data, call an API, fix a bug | Working code, edge cases, tests, reading unfamiliar code | 14 to 21 |
| AI system design | Whiteboard or doc: RAG assistant, agent, evaluation setup | Requirements first, access control, failure modes, evaluation, cost | 22 to 41, 77 to 80 |
| Cloud, integration and security | Often folded into design: deployment, identity, secrets, networking | Whether the design survives a security review and on-call | 42 to 62 |
| Troubleshooting | A broken system and its logs or traces | Method, hypotheses, evidence, prevention | 63 to 68 |
| Customer scenario / role-play | Interviewer plays a vague, sceptical or demanding customer | Discovery, honesty, scoping, calm under pressure | 8 to 13, 69 to 73 |
| Behavioural | Past experience: ownership, ambiguity, failure | Evidence that you finish things and own mistakes | 74 to 76 |
The customer round is what separates an FDE loop from a standard software engineering loop. Many strong coders prepare for it least and lose the offer there. Across all rounds, interviewers tend to score the same dimensions:
| Dimension | Weak signal | Strong signal |
|---|---|---|
| Requirements | Jumps to a solution | Asks about users, data, systems and success criteria first |
| Coding | Happy-path code, silent on errors | Validation, timeouts, retries, idempotency and a test or two |
| Security | Mentioned only when prompted | Identity, authorisation and data flow raised unprompted |
| Evaluation | "It looked good in testing" | Test set, metrics, release gates and online monitoring |
| Operations | Deployment ends at "push to cloud" | IaC, CI/CD, observability, rollback and runbooks |
| Customer handling | Over-promises or argues | Listens, scopes, says no with an alternative, writes things down |
FDE fundamentals: the role and how it differs
Interviewers open with these to check that you understand what the job actually is. Answer crisply; a rambling definition is an early negative signal.
1. What is a Forward Deployed Engineer, in one minute?
Answer: A Forward Deployed Engineer is a software engineer who works embedded with a customer to ship a production system that solves a specific business problem, and owns it from discovery through deployment to a measured outcome. Palantir popularised the model of engineers being "forward deployed" into customer organisations; AI labs and enterprise software companies now hire FDEs to get GenAI and agent systems running inside real enterprises.
"Forward" means close to the customer's data, identity provider, network, approval processes and people. The work product is running code in the customer's environment, not slides. For the full picture, see what a Forward Deployed Engineer does.
2. How is an FDE different from a software engineer, solutions architect, consultant and ML engineer?
Answer: All five can write or design systems, but they differ in who sets the requirements, who owns the running system and what they are measured on. The FDE is the one accountable for a working, adopted system in one customer's environment.
| Role | Main focus | Typical output | Measured on |
|---|---|---|---|
| Product software engineer | Features for many customers from a roadmap | Code in the vendor's product | Feature quality and velocity |
| Solutions architect | Designs that fit the platform to the customer | Reference architectures, guidance, reviews | Technical win and design quality |
| Consultant | Advice, process, programme management | Assessments, roadmaps, recommendations | Engagement delivery |
| ML engineer | Models, training and serving pipelines | Models and ML infrastructure | Model quality and reliability |
| Forward Deployed Engineer | One customer's problem, end to end | Production code and integrations in the customer's stack | Business outcome in production |
Interview tip: Do not disparage the other roles. Say where they overlap and where the FDE's accountability is different. FDE vs solutions architect vs solutions engineer goes deeper.
3. Isn't an FDE just a solutions architect who codes?
Answer: No. The difference is ownership of the production system. A solutions architect shapes the design and often hands over to a delivery team; an FDE writes the integration service, debugs the identity flow at 11 pm before go-live, sets up evaluation and stays until the system is used.
4. How is an FDE different from an IT consultant or a services delivery engineer?
Answer: A consultant is usually paid for advice and a services engineer for delivering a defined scope; an FDE is expected to find the right problem, build the engineering solution and feed reusable patterns back into a product or platform.
In practice the lines blur, and many FDEs in Indian services firms and GCCs work on customer engagements. The distinguishing habits are engineering depth (you can read and change the customer's code), a bias for shipping small working increments, and turning one-off fixes into reusable components such as a connector, an evaluation harness or a deployment template.
5. How is an FDE different from an ML engineer? Who owns the model?
Answer: An ML engineer optimises models and the pipelines that train and serve them. An FDE usually consumes models through APIs or managed platforms and owns everything around them: data access, retrieval, tools, identity, security, deployment, evaluation and adoption.
With foundation models, most enterprise value comes from the system around the model, so the FDE's job is choosing and constraining the model rather than training it. An FDE should still understand embeddings, fine-tuning trade-offs and evaluation well enough to know when to bring in an ML specialist.
6. Is an FDE a support or customer-success role?
Answer: No. Support resolves issues in an existing product and customer success drives adoption and renewals. An FDE designs and builds the system the customer's business process depends on, frequently makes the architecture decisions, and carries production responsibility for it.
The customer exposure is a multiplier on engineering impact, not a substitute for it. A strong FDE turns a customer's vague goal into a running system, and the patterns from that engagement often become product features.
7. Why are cloud providers and AI companies investing in forward deployed engineering now?
Answer: Because the hard part of enterprise AI has moved from building a demo to running a governed system against real data and systems of record. That gap needs engineers inside the customer's environment.
AWS, for example, has publicly described a Forward Deployed Engineering organisation that embeds engineers with customers to co-develop and deploy agentic AI, and in June 2026 it extended that model to consulting partners in an AWS Partner Network blog post, "Introducing Forward Deployed Engineering for Partners". The post frames the work as moving "from counsel to outcomes" and delivering systems built for customer self-sufficiency.
The FDE path looks like this, and interviewers often probe each stage:
Customer problem -> Discovery -> Requirement
-> Data -> AI architecture -> RAG / Agent
-> Tools / MCP -> APIs -> Security -> Cloud
-> Deploy -> Observe -> Evaluate -> Optimise
-> Business outcome
Customer discovery and requirements
Discovery is where FDE loops differ most from standard engineering loops. The interviewer wants to see you slow down, ask the right questions and narrow scope before designing anything.
8. "We want AI for our operations team." That is all the customer says. What do you do?
Answer: Run discovery before proposing anything. Ask who the users are, which task costs the most time or causes the most errors today, what data and systems are involved, what "better" would look like and who signs off. Then narrow to one measurable use case and propose a small first deliverable with explicit non-goals.
Shadowing two or three users for an hour usually reveals more than a workshop. Confirm data access and security constraints early, because those decide feasibility more often than model capability.
9. What do you ask in the first discovery session for an internal knowledge assistant?
Answer: Questions that decide scope, feasibility and success: who asks what today, where the answers live, who is allowed to see what, and how you will know it works.
- Users, their top ten questions, and how they get answers today
- Source systems (SharePoint, Confluence, file shares, ticket history), owners and update frequency
- The permission model: groups, confidential documents, regional restrictions
- Languages, for example English plus Hindi or Telugu queries
- Where users will ask: Teams, a portal, the ServiceNow service portal
- Compliance needs: retention, logging, personal data under the DPDP Act
- The success metric and the baseline, and who signs off on go-live
The method is covered in AI use case discovery.
10. How do you define success metrics for an AI pilot?
Answer: Agree four kinds of metric in writing before building: a business metric, a quality metric, an operational metric and a safety metric, each with a baseline and a threshold.
- Business: average handling time, first-contact resolution, hours saved per week
- Quality: answer correctness and faithfulness on an agreed test set, task success rate for agents
- Operational: p95 latency, availability, cost per successful task
- Safety: zero unauthorised actions, no cross-permission leakage, escalation rate
Without a baseline you cannot prove value, and without a threshold every review turns into an opinion contest.
11. How do you decide whether a problem needs an LLM, an agent, or neither?
Answer: Use the simplest thing that meets the requirement. Rules, SQL or a normal integration for deterministic work; an LLM call for language tasks such as summarising or extracting; a fixed workflow with LLM steps when the path is known; an agent only when the path genuinely cannot be decided in advance.
Agents add cost, latency, non-determinism and a larger security surface. An interviewer is pleased when you talk a customer out of an agent they do not need.
Real-world example: Consider a retailer that wants an "agent" to answer "where is my order?". A tool call to the order API plus a templated response covers most cases; the LLM is only needed to understand the question and phrase the reply.
12. How do you assess whether the customer's data is ready?
Answer: Run a short data spike in the first week: get real access, sample the data and measure quality against the use case, instead of trusting a description of it.
Check access paths and approvals, ownership, duplicates and conflicting versions, missing permission metadata, scanned versus digital documents, personal data, and how often the data changes. Write the findings down with a feasibility call.
13. What goes into the scope document for a short pilot?
Answer: Enough that anyone can tell what is being built, what is not, and how the pilot will be judged: problem statement, users, in-scope workflows, explicit non-goals, data sources, integrations, security assumptions, success criteria, risks, milestones, the sign-off owner and the change process.
Practical coding questions (Python, APIs, data handling)
FDE coding rounds lean towards realistic integration work rather than puzzles. You are judged on whether the code runs, handles bad input and could be maintained by someone else. Python is the common choice for AI FDE roles.
14. Parse this CSV/JSON export from a customer's system and produce a cleaned summary.
Answer: Inspect the data and ask about it first: encoding, delimiters, duplicates, missing fields, date formats and size. Then write a small parser with explicit validation, collect bad rows into a rejects list with reasons instead of crashing, and only then transform and aggregate.
Use a Pydantic model or typed dataclass per row. Test with a fixture containing a malformed row, an empty file and a duplicate. If the file could be large, stream it rather than loading it all into memory, and report counts of accepted and rejected rows.
15. Call a third-party REST API that is rate-limited and occasionally fails. Make it reliable.
Answer: Put a timeout on every request, retry only retryable failures (timeouts, connection errors, 429 and 5xx) with exponential backoff and jitter, respect Retry-After, handle pagination, and use idempotency keys on writes so a retry cannot create duplicates.
for attempt in range(max_retries):
try:
r = client.get(url, timeout=10)
if r.status_code in RETRYABLE:
raise Retryable(r)
r.raise_for_status()
return r.json()
except (Retryable, Timeout):
sleep(backoff(attempt))
raise GaveUp(url)
16. This integration worked yesterday and fails today. Here are the logs. Debug it.
Answer: Reproduce the failure, read the actual error, then test hypotheses in order of likelihood: expired credential or rotated secret, changed API schema or version, certificate or DNS change, rate limit, a deployment on either side, or bad input data.
Ask "what changed?" across deploys, config and secrets on both sides, and narrow with evidence instead of changing several things at once. After the fix, add a test or alert so the same failure is caught earlier next time.
17. Write a function that maps records from System A's schema to System B's.
Answer: Get both schemas and agree which system is the source of truth. Keep the mapping explicit and testable, as a declarative table of field rules rather than nested conditionals, decide what happens to unmapped fields and invalid values, and record every transformation for audit.
Strong candidates raise ID reconciliation without being asked: how a record in B is matched to its record in A, and what happens on conflicts or partial updates.
18. Wrap a slow internal API behind a clean service the AI layer can call.
Answer: Build a small FastAPI (or similar) service with input validation, typed responses, timeouts, caching only where data is safe to cache, structured errors and health checks. Most importantly, pass the user's identity through so the AI acts with the user's permissions, not a superuser account.
This is the same pattern you use when exposing the API as a tool through MCP (Model Context Protocol): narrow contract, clear errors the model can act on, and authorisation enforced in the service.
19. Write a webhook receiver for ticket events that is safe against duplicates and replays.
Answer: Verify the signature, check a timestamp window, store the event ID and ignore ones already processed, acknowledge quickly, and do the real work asynchronously from a queue.
Most ticketing and CRM platforms retry deliveries, so duplicates are normal, not exceptional. Make the processing idempotent (an upsert keyed on the event or record ID), send failures to a dead-letter queue, and expose a replay tool so you can recover after an outage without hand-editing data.
20. The LLM must return JSON for a downstream system. How do you make that reliable in code?
Answer: Use the provider's structured-output or tool-calling feature with a schema, validate the result with Pydantic anyway, and on failure retry once with the validation error included. If it still fails, route to a fallback rather than passing bad data downstream.
class Claim(BaseModel):
policy_no: str
amount: Decimal
category: Literal["motor", "health"]
try:
claim = Claim.model_validate_json(raw)
except ValidationError as e:
claim = retry_with_feedback(raw, e)Validate business rules too, not just types: an amount above the policy limit is valid JSON and still wrong.
21. You need to call an API for 20,000 records quickly without getting throttled. How?
Answer: Use bounded concurrency, for example asyncio with a semaphore or a worker pool sized to the API's documented limits, combined with backoff on 429, checkpointing so a crash resumes rather than restarts, and a report of failed records for reprocessing.
GenAI and LLM fundamentals
You will not be asked to derive attention, but you will be asked how model behaviour affects cost, reliability and design. Answer in engineering terms.
22. Why do tokens and context windows matter to an FDE?
Answer: Because cost and latency scale with input and output tokens, and the context window caps how much instruction, history and retrieved content fits in one call. Both directly shape architecture and the customer's bill.
Stuffing more context is not free: it costs more, slows responses and can dilute attention so relevant facts in the middle get missed. FDEs budget tokens per request, trim retrieved context and summarise long histories.
23. What does temperature control, and is a model deterministic at temperature zero?
Answer: Temperature controls randomness in token sampling: low values for extraction and classification, higher for drafting. Even at zero, outputs are not reliably identical across runs or model versions, so systems must not depend on exact wording.
24. What causes hallucinations, and how do you reduce them in an enterprise system?
Answer: Models generate plausible text from patterns, not from a verified fact store, so they fill gaps confidently. Reduce it by grounding answers in retrieved sources, requiring citations, allowing "I don't know", constraining outputs to schemas, and checking high-risk answers with a verification step.
25. How does function calling (tool calling) work, and who executes the function?
Answer: You describe tools with names and JSON schemas; the model returns a structured request to call one with arguments; your code validates and executes it, then returns the result to the model. The model never executes anything itself.
That separation is the security boundary: authorisation, input validation and rate limits live in your code, so a manipulated model can only ask, not act.
26. How do you choose a model for an enterprise use case?
Answer: Shortlist on constraints, then decide on evidence. Constraints: data residency and region availability, approved hosting (Amazon Bedrock, models in Microsoft Foundry, Vertex AI), licensing, context length and latency. Evidence: quality on the customer's own evaluation set and cost per successful task.
Smaller models are often enough for routing, classification and extraction; reasoning models suit multi-step planning but cost more and respond more slowly. Design so the model can be swapped behind an interface, because it will be.
27. When would you choose RAG, fine-tuning or a plain API integration?
Answer: RAG when answers depend on changing or permissioned knowledge and need citations. Fine-tuning for consistent format, tone or a narrow task with plenty of examples, not for adding facts that change. A plain integration or SQL query when the answer lives in structured data; not everything needs an LLM.
RAG interview questions for FDEs
RAG is the most common first enterprise use case, so expect depth here. For a larger retrieval question bank, use the RAG interview questions page; this section keeps the FDE angle.
28. Walk me through a production RAG pipeline end to end.
Answer: Ingestion: connect sources, parse, chunk, embed and index with metadata (source, version, access groups). Query time: authenticate the user, optionally rewrite the query, retrieve with hybrid search filtered by permissions, rerank, assemble a bounded context, generate with citations, and abstain when evidence is weak. Around it: evaluation, tracing and refresh jobs.
Sources -> parse -> chunk -> embed -> index(+ACL)
User -> SSO -> query rewrite -> hybrid retrieve
-> ACL filter -> rerank -> LLM + citations
-> answer / abstain -> trace -> eval
29. How do you choose a chunking strategy?
Answer: Follow the document's structure: split on headings, sections and list boundaries rather than fixed character counts, keep chunks small enough to be specific and large enough to stand alone, and carry the section title and document metadata into each chunk.
Parent-child chunking (retrieve a small chunk, send its parent section) helps with policies and manuals. Do not argue chunk sizes in the abstract: compare two strategies on the evaluation set.
30. Why use hybrid search and reranking instead of vector search alone?
Answer: Vector search handles paraphrase but is weak on exact identifiers; keyword search (BM25) catches policy numbers, error codes and product names. Fusing both, then reranking the top candidates with a cross-encoder or reranking model, usually improves precision noticeably.
Real-world example: Consider a GCC IT helpdesk where users paste error codes. Pure vector search returns generically similar articles; keyword matching finds the one article that mentions that exact code.
31. The source documents are scanned PDFs with tables. What do you do?
Answer: Treat parsing as a first-class engineering problem: OCR for scans, layout-aware extraction, tables converted to structured rows or markdown with headers repeated, and page numbers kept for citations. Check extraction quality on samples before indexing anything.
Bad parsing silently caps answer quality, and no model can fix it later.
32. How do you keep the index fresh, including deletions and permission changes?
Answer: Run incremental sync using change feeds or content hashes, delete or tombstone chunks when the source document is removed, re-sync access groups on a schedule or on change events, and version documents so superseded ones stop being retrieved.
Production consideration: Changing the embedding model means re-embedding the whole corpus, because vectors from different models are not comparable. Plan it as a migration with a parallel index and an evaluation comparison.
33. How do you enforce document permissions in RAG?
Answer: Filter at retrieval time using access metadata synced from the source system and the user's identity and groups from the identity provider. Never rely on the prompt to hide content, because anything that reaches the model can leak.
Test with negative cases: a user from group A must get zero chunks from group B's documents, and a revoked user must lose access within the agreed sync window. Question 78 applies this in a full design.
34. When would you use agentic RAG or GraphRAG instead of a single retrieval step?
Answer: Agentic RAG, where the model plans several retrieval steps or chooses sources, helps for multi-part questions across systems. GraphRAG helps when answers depend on relationships between entities, such as which suppliers are linked to which contracts. Both add latency and cost, so use them when evaluation shows single-step retrieval fails.
Agentic AI, tools, MCP and A2A
Agents are where enterprise risk concentrates, because they act on systems of record. Interviewers look for control, not cleverness. The agentic AI interview questions page has more depth on patterns.
35. What makes something an agent rather than a workflow?
Answer: In a workflow, your code decides the sequence of steps; in an agent, the model decides the next step at runtime by choosing tools in a loop until it judges the goal met. Many production "agents" are really workflows with one or two model-driven decisions, and that is usually a good thing.
Explain the trade-off: autonomy buys flexibility at the price of predictability, cost and testability. Bound it with step limits, allowed tool lists per state and explicit stop conditions.
36. What makes a good tool for an agent?
Answer: A narrow, well-named action with a strict input schema, clear descriptions of when to use it, idempotent behaviour where possible, and errors written so the model can recover. Separate read tools from write tools, and make destructive actions require approval.
Prefer propose_status_change plus an approval step over a generic update_ticket that accepts any field. Fewer, sharper tools reduce wrong calls and make policy enforcement simple.
37. What is MCP, and when would you use it instead of calling an API directly?
Answer: MCP (Model Context Protocol) is an open protocol, introduced by Anthropic in late 2024, for connecting AI applications to tools and data. A host application uses MCP clients to talk to MCP servers, which expose tools, resources and prompts over stdio locally or HTTP remotely, with OAuth-based authorisation defined for HTTP transports.
Use MCP when the same capability should be reusable across several AI hosts or agents, or when the customer standardises on it. A direct API call is fine inside a single application with one fixed integration. The MCP interview questions go deeper.
Interview tip: Point out that MCP standardises the connection, not the security. Authorisation, input validation and audit still live in your server.
38. You are building an MCP server that exposes ServiceNow incident lookups. What do you get right?
Answer: Read-only tools first, narrow schemas, the end user's identity propagated to ServiceNow rather than a shared admin account, pagination and field filtering so responses fit the context window, rate limiting, redaction of personal data, and structured logs for every call.
Implementation detail worth knowing in 2026: in version 2 of the official MCP Python SDK the high-level server class is MCPServer (from mcp.server import MCPServer), previously FastMCP; check current SDK documentation before an interview. The MCP server Python tutorial walks through a build.
39. How does A2A differ from MCP?
Answer: MCP connects an agent to tools and data; A2A (Agent2Agent) connects agents to other agents. A2A is an open protocol created by Google and now hosted as a Linux Foundation project. Agents advertise capabilities through Agent Cards and exchange tasks and messages over HTTP using JSON-RPC.
Use A2A when independently built and owned agents, possibly from different vendors, must delegate work to each other. Inside one team's system, plain function calls or a graph framework are simpler. See A2A protocol explained.
40. How do you design agent memory for an enterprise assistant?
Answer: Separate short-term state (the current conversation and task) from long-term memory (preferences or facts kept across sessions), scope long-term memory per user or tenant, store it with provenance and expiry, and let users see and delete it.
Risks to name: personal data retained longer than policy allows, stale facts overriding current records, and memory poisoning where injected text gets saved and reused. Systems of record remain the source of truth; memory is a cache of context, not a database.
41. Design an agent that updates support tickets in ServiceNow or Jira safely.
Answer: First ask whether it needs to be an agent; a fixed workflow may be enough. If it does, give it a small set of narrow tools (read ticket, add comment, propose status change), explicit state, a bounded step count, and human approval before any write that changes priority, assignment or closure. Every action is audited with the user, inputs, tool call and result.
Discuss prompt injection from ticket text, the ticketing API's rate limits, and a kill switch. Graph frameworks such as LangGraph make the approval pause explicit with interrupts and checkpoints; the LangGraph interview questions cover that mechanism.
Enterprise integration and identity
Most FDE time goes into integration and identity, and most go-live delays come from here. These questions separate people who have shipped from people who have demoed.
42. Should an AI agent act as the user, as a service account, or as itself?
Answer: Default to acting on behalf of the user with delegated, scoped tokens, so the agent can never do more than the user could. Use a dedicated agent identity with narrowly scoped permissions for background tasks, and never a shared admin account.
With Microsoft Entra ID that means flows such as on-behalf-of token exchange for downstream APIs; other identity providers have equivalent token-exchange patterns. Log both the human and the agent identity on every action. AI agent identity and access covers the patterns.
43. What practical issues do you expect when integrating with Salesforce, ServiceNow or Jira?
Answer: API rate limits and quotas, pagination, field-level and record-level security, heavily customised fields and workflows per customer, separate sandbox and production instances, API version changes, and webhook or event delivery that retries.
Read the customer's actual instance metadata before writing mappings; two ServiceNow customers can have very different incident forms. Agree a test instance with realistic data early, because many integration bugs only appear with real customisations.
44. How do you wire Entra ID sign-in into an AI app and use it for retrieval permissions?
Answer: Sign users in with OpenID Connect, validate tokens on the API (signature, issuer, audience, expiry), read group or role claims, and map them to the access filters used at retrieval and tool authorisation.
A detail interviewers like: when a user belongs to many groups, the token may carry an overage indicator instead of the full group list, so the API must look groups up through Microsoft Graph. Cache that lookup briefly and handle revocation.
45. Batch sync or event-driven integration: how do you choose?
Answer: Event-driven when freshness matters and the source emits reliable events, for example ticket updates feeding an agent. Batch when the source has no events, volumes are large, or the business accepts a daily refresh. Many designs use events for changes and a periodic batch reconciliation to catch missed ones.
46. The customer's core system is legacy with no REST API. How do you integrate?
Answer: Work down a ladder of options: an existing SOAP or vendor SDK interface, read replicas or views agreed with the DBA, file exports on a schedule, a message queue the system already uses, and UI automation only as a last resort with explicit risk sign-off.
Wrap whatever you use behind a clean internal API so the AI layer never touches the legacy mechanism directly, and the integration can be replaced later without changing agents or prompts.
Cloud (AWS and Azure) for FDE interviews
You are not expected to be a cloud architect, but your design must survive the customer's security review and operations team. Many FDE teams work primarily on AWS, with Azure common in Microsoft-centric enterprises.
47. How would you deploy an AI assistant securely in the customer's AWS or Azure account?
Answer: Containerise the service, deploy through infrastructure as code (Terraform) and a CI/CD pipeline, run it in private subnets, reach the model service over private endpoints where available, use least-privilege roles per component, SSO through the customer's identity provider, encryption in transit and at rest, and separate dev, staging and production.
Add logging with redaction, budgets and alerts, and a documented rollback. Say you would align with the customer's landing zone and tagging standards rather than inventing your own.
48. Where do secrets live, and how do they rotate?
Answer: In a managed secrets store (AWS Secrets Manager, Azure Key Vault or equivalent), never in code, images, environment files in git, or prompts. Workloads get access through their own identity (IAM roles, managed identities), rotation is automated, and the service picks up new credentials without a redeploy.
49. The service works locally but cannot reach the model endpoint inside the customer's VPC.
Answer: Trace the network path hop by hop with evidence, then check identity, instead of changing several things at once.
What I would check:
- DNS: does the endpoint resolve to the private endpoint address from inside the subnet?
- Route tables and any proxy or egress firewall rules
- Security groups and network ACLs on both sides
- The private endpoint or VPC endpoint policy
- IAM or role permissions for the model, and whether the model is enabled in that region
Production consideration: Write the working network path into the runbook; the same issue will reappear in the next environment.
50. On AWS, which services would you reach for to build and run an agent in 2026?
Answer: Amazon Bedrock for model inference, with Knowledge Bases for managed retrieval and Guardrails for content controls; Amazon Bedrock AgentCore for running agents in production, with services such as Runtime, Gateway (which exposes tools to agents, including as MCP tools), Memory, Identity and Observability that can be used independently. Surrounding services are the usual ones: IAM, VPC endpoints, Secrets Manager, CloudWatch, ECS or EKS.
Know the naming change: the original Amazon Bedrock Agents is now Amazon Bedrock Agents Classic, in maintenance mode and closed to new customers from 30 July 2026, and AWS recommends AgentCore for new agent development. More in AWS for AI FDE engineers.
51. What is the equivalent stack on Azure?
Answer: Microsoft Foundry (previously Azure AI Foundry) for models, including Azure OpenAI models, agents and evaluation; Azure AI Search for retrieval; Microsoft Entra ID for identity; Key Vault for secrets; Private Link for network isolation; Azure Monitor and Application Insights for telemetry; AKS or Container Apps for hosting.
For code-first agents, Microsoft Agent Framework is the open-source successor to Semantic Kernel and AutoGen. Service names and features in this space change often, so say "I'd confirm against current Microsoft Learn documentation" when unsure.
52. Containers or serverless for an AI service, and what does the CI/CD pipeline look like?
Answer: Serverless functions suit short, spiky, event-driven steps; containers on ECS, EKS, AKS or Container Apps suit long-running streaming responses, agent loops and services with steady traffic. Many systems use both.
The pipeline should build and scan the image, run unit tests, run the offline evaluation set against the changed prompt or configuration and block on a quality drop, deploy to staging through Terraform, then promote with a canary and an easy rollback.
Reading answers is not the same as defending them under questioning. If you want to practise these rounds against realistic systems, the AI Forward Deployed Engineer course builds a ServiceNow AI agent via MCP, a secure banking AI assistant and an IT-ops multi-agent platform, and includes mock interviews with resume and portfolio review.
Security and governance
Security reviews are a gate every enterprise AI project passes through. Raise these topics unprompted in design rounds. The AI security interview questions page has a deeper bank.
53. What is prompt injection, and how do you defend against it?
Answer: Prompt injection is untrusted text, typed by a user or hidden in a document, email, web page or ticket, that tries to override the system's instructions. Indirect injection through retrieved content is the bigger enterprise risk. Defend in layers, because no prompt fully prevents it.
- Treat retrieved and tool-returned text as data, clearly separated from instructions
- Least-privilege tools, with authorisation enforced in code outside the model
- Approval for high-impact actions and allow-lists for destinations such as email recipients
- Input and output filtering, and red-team test cases in the evaluation set
54. How do you handle personal data and India's DPDP Act in an AI application?
Answer: Minimise what reaches the model, mask or tokenise identifiers where the task allows, keep data in approved regions and services, set retention for prompts, traces and memory, record the purpose of processing, and support deletion requests across indexes and logs.
The Digital Personal Data Protection Act makes consent, purpose limitation and security safeguards concrete obligations; FDEs make sure the architecture can honour them.
55. What layers of guardrails would you put around an LLM application?
Answer: Input checks (injection patterns, off-topic requests, personal data), retrieval controls (permission filters, trusted sources only), generation controls (system instructions, structured outputs, grounding checks), output checks (toxicity, data leakage, policy rules), and action controls (tool policy, approvals, rate limits).
Managed options such as Amazon Bedrock Guardrails or content filters in Microsoft Foundry cover some layers; business rules usually need your own code. Measure false positives too, because over-blocking kills adoption.
56. What does AI governance look like on a real engagement?
Answer: A use-case register with risk ratings, named owners, approved models and data sources, documented evaluations before release, change control for prompts and models, incident handling, and periodic review. FDEs supply the evidence: evaluation reports, data-flow diagrams, audit logs.
Some customers align with ISO/IEC 42001 for an AI management system, and teams serving EU users must consider the EU AI Act.
57. How would you threat-model and red-team an agent before go-live?
Answer: Map assets (data, systems, credentials), entry points (chat, documents, tool outputs, email), and the actions the agent can take. Then attack each path: injection through every input channel, privilege escalation via tools, data exfiltration through outputs or links, denial of wallet through runaway loops.
Evaluation, observability and AgentOps
"How do you know it works?" is the question behind every FDE interview. Have a concrete answer with test sets, metrics, traces and release gates.
58. The assistant is live. How do you know whether it is any good?
Answer: Combine offline and online evaluation. Offline: a test set built from real user questions with expected answers or sources, measuring retrieval relevance, faithfulness and answer correctness on every prompt, model or index change. Online: traces for every request, user feedback, sampled human review, and the business outcome metric.
Tools such as Ragas, LangSmith or Langfuse help, but the curated test set is the real asset. See RAG evaluation metrics for the metrics.
59. What would you put in observability for an AI system?
Answer: Standard logs, metrics and traces (OpenTelemetry), plus AI-specific spans: prompt version, model, retrieved document IDs and scores, token counts, tool calls with inputs and results, latency per step, cost, and guardrail decisions.
Build dashboards for error rate, latency, cost and quality signals, alerts on spikes, and the ability to open the full trace behind any user complaint with sensitive fields redacted.
60. How do you evaluate an agent, not just a single answer?
Answer: Evaluate the outcome and the trajectory: did it complete the task, did it call the right tools with valid arguments in a sensible order, did it respect policy, and how many steps, tokens and seconds did it take.
Use scenario-based test suites with mocked tools and seeded data, including adversarial and edge cases, and track task success, tool-call accuracy, policy violations and cost per task over time.
61. When is LLM-as-a-judge reliable, and what are its pitfalls?
Answer: It is useful for scaling evaluation of qualities like faithfulness or tone, provided you calibrate it against human labels on a sample and use specific rubrics. Pitfalls: position and length bias, preference for its own style, inconsistency across runs, and silent drift when the judge model changes.
Pin the judge model and prompt, report agreement with humans, and keep deterministic checks (schema, citation present, exact-match facts) wherever possible.
62. What is AgentOps, and how do you release changes to prompts, tools and models safely?
Answer: AgentOps is the operational discipline for agents: versioning prompts, tools and model configurations together, gating releases on evaluation, rolling out gradually, monitoring quality and cost in production, and rolling back quickly.
Production troubleshooting scenarios
Scenario rounds hand you a broken system and watch your method. Narrate hypotheses, the evidence you would collect and how you would prevent a repeat.
63. Users report that the RAG chatbot gives wrong answers. How do you debug it?
Answer: Collect concrete failing examples, open their traces, and classify each failure as retrieval (the right content never reached the model), generation (the right content was there but the answer was wrong) or data (the source itself is wrong or outdated). Fix by class, not by tweaking the prompt blindly.
What I would check:
- Is the correct document in the index at all, in its latest version, and parsed correctly?
- Did retrieval return it in the top results? If not, check chunking, hybrid search, query rewriting and metadata filters
- Did a permission filter wrongly exclude it?
- If it was retrieved, was it cut off by context trimming or buried among many chunks?
- Are users asking a question type the system was never designed for?
Production consideration: Add every confirmed failure to the evaluation set before fixing it, so you can prove the fix and catch regressions.
64. Retrieval is correct: the right chunk is in context, but the answer is still wrong. What now?
Answer: This is a generation or context-assembly problem. Check whether conflicting or near-duplicate chunks confuse the model, whether the instructions are ambiguous, whether the answer needs reasoning across several chunks, and whether the model is too small for the task.
What I would check:
- Context order and volume: is the correct chunk drowned among loosely related ones? Rerank and send fewer
- Conflicts: are old and new policy versions both present? Filter by version
- Instructions: do they require answering only from context, citing, and abstaining?
- Format: is the model asked to compute or compare tables? Pre-structure the data or use a tool
- Model: does a stronger model fix it on the eval set, and is the cost justified?
Production consideration: Track faithfulness separately from retrieval metrics, so you can tell which half of the system regressed.
65. An agent created a ServiceNow ticket that the user was not authorised to raise. Walk through the fix.
Answer: Treat it as a security incident first: contain (disable the write tool or the agent), assess impact from the audit log, notify the owners. Then find where the control chain failed, because a model choosing an action must never be enough to execute it.
identity -> authorisation -> tool policy
-> input validation -> risk scoring
-> approval (if needed) -> execution
-> audit logWhat I would check:
- Identity: did the call carry the real user's identity, or a shared service account?
- Authorisation: was the user's permission for this ticket type checked in code, server-side?
- Tool policy: was this tool even allowed for this user, agent and state?
- Validation: were category, assignment group and fields validated against allowed values?
- Risk and approval: should this action type have required human approval?
- Audit: does the log show who, what, why and with which tool arguments, enough to reconstruct the event?
Production consideration: Add the case to the agent's regression suite and to the red-team set, and review every other write tool against the same chain before re-enabling.
66. Cost per request is around ₹8 and the target is ₹0.50 (illustrative). How do you get there?
Answer: Measure where the money goes, then cut the biggest items first. Typical culprits are a large model used for every step, oversized retrieved context, agent loops with many calls, no caching, and retries.
| Illustrative driver | Fix |
|---|---|
| Large model for every step | Route classification and extraction to a small model; keep the large one for hard answers |
| Twenty chunks of context per call | Rerank and send the top few; summarise long histories |
| Agent loop averaging many calls | Cap steps, use a fixed workflow where the path is known |
| Repeated FAQs answered from scratch | Prompt caching for static prefixes; a semantic cache for common questions where safe |
| Retries and failed calls | Fix validation failures upstream; count cost per successful task, not per call |
Report the result as cost per successful task alongside quality on the evaluation set, so nobody hits the cost target by quietly breaking answers. Cloud cost optimisation for AI covers the tooling.
Production consideration: Put per-tenant or per-team budgets and alerts in place; a single runaway loop can wipe out a month of savings.
67. Users say answers have become worse this week. Walk me through it.
Answer: Find what changed, confirm the regression with data, and roll back if needed before investigating at leisure.
What I would check:
- Model or model version, prompt version, retrieval settings or index refresh
- New or changed source documents, or a parsing change
- Re-run the evaluation set on current and previous configurations
- Pull traces for reported bad answers and classify them as retrieval or generation
- Check for a shift in the questions users ask
Production consideration: Add the failing cases to the test set and make the evaluation gate block the type of change that caused it.
68. After launch, p95 latency jumps and some requests fail with throttling errors from the model service.
Answer: You are probably hitting model quota or concurrency limits, made worse by retries and long contexts. Stabilise first, then fix capacity and efficiency.
What I would check:
- Throttling and quota metrics for the model, by region and model ID
- Retry storms: is client retry logic multiplying load? Add jitter and a circuit breaker
- Token volume per request: did context size grow?
- Streaming: are users waiting for full responses unnecessarily?
- Capacity options in the platform: quota increases, provisioned capacity, cross-region inference where permitted by data-residency rules
Production consideration: Load-test with realistic token sizes before go-live, and set a latency budget per step so you know which stage to optimise.
Customer-facing and stakeholder scenarios
In these role-plays the interviewer is the customer or a senior stakeholder. There is rarely one right answer; they are watching what you ask, whether you stay honest and whether you protect the outcome. Answer behavioural questions with a specific situation, what you did, the result and what you would change.
69. The customer's security team is sceptical and wants to block the project.
Answer: Treat them as partners. Ask exactly what worries them: data leaving the tenancy, model providers training on their data, access control, logging, prompt injection. Answer each specifically and offer a written security note with a data-flow diagram, storage locations, retention, identity integration and a threat model.
Accept constraints you cannot argue away, such as a required private deployment, and redesign around them. A security team that co-authored the controls becomes your strongest advocate.
70. Your demo fails in front of the customer's leadership.
Answer: Stay calm, say plainly what happened, and switch to a recorded run or fallback environment if you have one. Do not blame the model or the customer's data in the room. Afterwards, find the root cause, send a short written note with the fix, and add the case to the test set.
Prevention is part of the answer: rehearse on the customer's real data, freeze the configuration before the demo, and keep a recorded backup. The patterns are in why AI demos fail in enterprise production.
71. Mid-project, the customer keeps adding "small" requests.
Answer: Acknowledge the value of each request, then make the trade-off visible: what it costs in time, risk or delay to the agreed milestone. Log changes in writing, propose a later phase for new items, and agree priorities with the person who owns the outcome.
72. The customer insists on an agent that closes tickets with no human review. You think it is a bad idea.
Answer: Say no to the risky version and yes to the goal. Explain the risk in their terms: wrong closures, audit findings, frustrated users. Offer a safer path: draft-and-approve first, measured accuracy over a few weeks, then automation for the lowest-risk categories once evidence supports it.
Put the recommendation and their decision in writing. If they still choose the risky option after hearing the evidence, it is their call, but your note protects both sides.
73. You are behind schedule. Give the steering committee a two-minute update.
Answer: Lead with the status and the decision you need, then the reason, the recovery plan and the risk. Executives want clarity, not a tour of the architecture.
Status: amber. Go-live moves 2 weeks.
Why: HR data access approved late.
Done: RAG answers pass eval on 3 of 4 areas.
Plan: parallel testing, scope cut on area 4.
Decision needed: accept phase-2 for area 4?Interview tip: Bad news delivered early with a plan builds trust. Bad news discovered by the sponsor destroys it.
74. Tell me about something you owned end to end, including unclear requirements.
Answer: Pick a story where you carried the work from an ambiguous start to a working result, including the unglamorous parts: questions you asked, assumptions you wrote down, the small version you shipped for feedback, documentation, handover and monitoring.
75. Walk me through a root cause analysis you wrote.
Answer: Give the timeline, impact, root cause (not just the trigger), contributing factors, the fix, and follow-up actions with owners and dates. Keep it blameless.
If you have never written one, write one for a failure in your own project before the interview; it is one of the most convincing artefacts you can bring.
76. Tell me about a time you were wrong in front of a stakeholder.
Answer: Choose a real mistake, how quickly you acknowledged it, how you fixed it and what you changed so it would not recur.
System design scenarios
Open every design answer with clarifying questions about users, data, systems, volumes, security constraints and success. Then draw, then cover failure modes, evaluation, cost and operations.
77. Design an agent that works across Salesforce, Jira, ServiceNow and AWS for a GCC's customer-escalation process.
Answer: A thin orchestrator (a graph or workflow) calls narrow tools exposed through MCP servers or an internal tool gateway, one per system, each enforcing the end user's permissions. Reads are free; writes go through policy checks and, for high-risk actions, human approval.
User (Teams) -> SSO -> Orchestrator (graph)
-> Tool gateway / MCP servers
-> Salesforce: account, entitlements
-> Jira: linked bugs, release status
-> ServiceNow: incident, change records
-> AWS: CloudWatch metrics (read-only)
-> Policy + approval -> write actions
-> Audit log + traces -> evaluation- Per-system identity: delegated user tokens where supported, scoped agent identities elsewhere
- Consistent correlation ID across all four systems so one escalation can be traced end to end
- Partial failure handling: if Jira is down, return what is known and say what is missing
- Rate limits and quotas per system, with backoff and caching for reference data
- Evaluation scenarios that span systems, with mocked tools and seeded records
Production consideration: Start with read-only summarisation of an escalation across the four systems. It delivers value quickly and earns the trust needed before enabling writes.
78. Design a permission-aware RAG assistant for a bank's internal policies.
Answer: Ingest from SharePoint and Confluence with structure-aware chunking, embed into a vector store such as PostgreSQL with pgvector, and attach access groups, version and effective date to every chunk. At query time, authenticate through Entra ID, retrieve with hybrid search filtered by the user's groups, rerank, generate with citations, and abstain when evidence is weak.
User -> SSO (Entra ID) -> API
-> retrieve(query, user_groups, latest)
-> rerank -> LLM + citations
-> answer / abstain -> trace -> eval
Sync job: docs + ACLs -> index (tombstones)Cover the refresh job so revoked or superseded documents leave the index, personal-data handling in logs, private networking to the model service, an evaluation set reviewed by the compliance team, and negative tests for cross-group leakage.
79. Design an IT-operations multi-agent platform that triages alerts and proposes remediations.
Answer: A triage agent groups related alerts and pulls context (recent deploys, runbooks, similar past incidents); a diagnosis step queries metrics and logs through read-only tools; a remediation step proposes actions from an approved runbook catalogue. Execution of any change goes through change management and human approval.
Discuss why separate agents (clear responsibility, separate permissions) versus one agent with many tools (simpler, fewer hand-offs), and justify the split with permissions rather than fashion. Add loop limits, cost budgets per incident, and evaluation on replayed historical incidents.
80. Design a claims-document processing system for an insurer with strict cost and accuracy targets.
Answer: Use a pipeline, not an agent: classify document type with a small model, extract fields with structured outputs into a validated schema, apply deterministic business rules, and route low-confidence or high-value cases to a human reviewer. Store extractions with source page references for audit.
Batch non-urgent work to reduce cost, measure field-level accuracy against a labelled sample, monitor the human-review rate as a quality and cost signal, and design for model replacement behind an interface.
Advanced scenarios: delivery, escalations and customer pressure
Questions 81 to 120 are the advanced, scenario-based questions that experienced and senior candidates increasingly face. They rarely have one right answer; the interviewer is watching how you break the problem down, where you put controls, and how you would prove the result.
81. A customer wants an enterprise AI agent in production in 30 days. How do you plan it?
Do not promise "an enterprise agent". Promise one measurable workflow, delivered safely, and make the plan visible from day one.
| Days | Focus | Deliverable |
|---|---|---|
| 1 to 5 | Discovery with business owners, security and IT | One-page scope: workflow, users, success metric, data sources, out-of-scope list |
| 6 to 10 | Data, identity and system access; architecture and risks | Architecture, access requests raised, evaluation set of 50 to 100 real cases |
| 11 to 15 | Thin end-to-end prototype on real data, read-only tools | Working demo with baseline evaluation scores |
| 16 to 20 | Integration, permissions, guardrails, human approval for write actions | Tested build in staging, threat model reviewed |
| 21 to 25 | Deployment, monitoring, runbooks, pilot users | Pilot release with dashboards and rollback |
| 26 to 30 | Measure against the baseline, fix the top failures | Results report, demo and next-phase roadmap |
Name the dependencies that can break the timeline (access approvals, security review, data quality) in week one, not week four. See the FDE first 90 days playbook for the longer view.
82. The AI system fails during a live business operation and the customer escalates. What do you do in the first hour?
- Contain first: switch the AI path off or to a safe fallback (human queue, read-only mode, previous version) so the business process can continue.
- Communicate early: acknowledge the incident, state the business impact and the workaround, and give a time for the next update. Do not guess the root cause.
- Preserve evidence: traces, prompts, retrieved context, tool calls, model and prompt versions for the affected requests.
- Diagnose by layer: recent deployments, model or prompt changes, data and index changes, upstream API failures, quota or throttling.
- Close properly: confirm recovery with the customer, then write a blameless root cause analysis with prevention actions (new evaluation cases, alerts, rollout gates).
83. How do you write a P1 incident update for a customer?
Keep it short and factual: what is affected, the business impact, what is being done, the current workaround, and when the next update will come. Example: "Since 10:40 the document assistant has returned errors for about 30% of requests. Claims processing continues through the manual queue. We have rolled back this morning's release and are verifying recovery. Next update at 11:30." Root cause comes later, after investigation, not in the first message.
84. The customer says the agent completed a task, but the transaction never happened. How do you investigate?
Separate what the agent reported from what the system of record shows. Trace the tool call, the HTTP status and response body, the transaction ID, the database commit, any downstream service, the retry history and idempotency records. The usual causes are swallowed errors, a timeout treated as success, or an asynchronous job that later failed. The fix is architectural: the agent may report completion only after verifying the authoritative result, for example by reading the record back.
85. A customer demands a feature you believe is unsafe, such as an agent deleting production records autonomously. What do you say?
Explain the specific risk with an example, then offer a path that meets the business goal: read-only analysis first, a proposed action list, explicit approval, transaction boundaries, soft deletes or backups, and an audit trail. Document the decision and who made it. Never bypass a security control to hit a deadline.
86. A customer architect and your engineering team disagree on the architecture. How do you resolve it?
Move the discussion from opinions to requirements: security, performance targets, cost, operability and the customer's existing standards. Compare the options against those requirements, run a small prototype or benchmark if the disagreement is about performance, and record the decision with an owner. The goal is a defensible decision, not winning the argument.
87. How would you measure the ROI of an enterprise AI solution?
Set the baseline before you build. Example: 5,000 support tickets a month, 20 minutes each by hand, 8 minutes with AI assistance. Time saved = tickets x (old time - new time) = 5,000 x 12 minutes = 1,000 hours a month. Convert that with the customer's real loaded labour cost, then subtract model inference, cloud, human review, development and maintenance costs. Report quality alongside it (error rate, escalations, customer satisfaction), because time saved on wrong answers is not a saving.
88. The customer wants AI in a regulated process, such as loan approval. How do you approach it?
Bring in security, compliance, legal and the business owner before choosing an architecture. Identify the applicable regulations, data residency, retention, audit and human-oversight requirements. Design so the LLM assists (extraction, summarisation, policy lookup) while the regulated decision stays with approved deterministic rules and authorised humans. An LLM should not independently approve a loan.
Advanced practical coding scenarios
89. Call a customer's inventory API that sometimes returns HTTP 500 or takes 20 seconds. Show retries and timeouts.
import logging
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
log = logging.getLogger(__name__)
session = requests.Session()
retry = Retry(
total=3,
backoff_factor=1, # waits grow between attempts
status_forcelist=[429, 500, 502, 503, 504],
allowed_methods=["GET"], # only retry idempotent calls
)
session.mount("https://", HTTPAdapter(max_retries=retry))
def get_inventory(url):
try:
resp = session.get(url, timeout=(3, 10)) # connect, read
resp.raise_for_status()
return resp.json()
except requests.Timeout:
log.warning("inventory API timed out", extra={"url": url})
except requests.HTTPError as err:
log.warning("inventory API error: %s", err)
except requests.RequestException as err:
log.error("inventory API request failed: %s", err)
return None
Follow-up: would you retry a POST? Not blindly. A POST can create a duplicate order or payment. Retry it only with an idempotency key that the server honours, or another deduplication mechanism.
90. Process a 10 GB CSV without loading it into memory.
import pandas as pd
def process_large_csv(path, out_path):
total = 0
for i, chunk in enumerate(pd.read_csv(path, chunksize=100_000)):
valid = chunk.dropna(subset=["account_id", "amount"])
valid.to_csv(out_path, mode="a", header=(i == 0), index=False)
total += len(valid)
return total
In production, also validate the schema, quarantine malformed rows, checkpoint progress so a failure can resume, and avoid double-processing on restart. For much larger or recurring workloads, move to Spark or a managed data service.
91. Implement a rate limiter: 10 requests per minute per user.
import time
class TokenBucket:
def __init__(self, capacity, refill_per_sec):
self.capacity = capacity
self.tokens = capacity
self.refill = refill_per_sec
self.last = time.monotonic()
def allow(self):
now = time.monotonic()
self.tokens = min(self.capacity,
self.tokens + (now - self.last) * self.refill)
self.last = now
if self.tokens >= 1:
self.tokens -= 1
return True
return False
buckets = {}
def allow_request(user_id):
bucket = buckets.setdefault(user_id, TokenBucket(10, 10 / 60))
return bucket.allow()
This is a single-process demonstration. With several API replicas, keep the bucket in a shared store such as Redis with an atomic update (a Lua script or similar), and return HTTP 429 with a Retry-After header when the limit is hit.
92. The same webhook arrives several times. Make processing safe.
Claim each event atomically before doing any work, for example with a unique constraint on the event ID:
from psycopg2.errors import UniqueViolation
def handle_webhook(event, db):
try:
db.execute(
"INSERT INTO processed_events (event_id, status) VALUES (%s, 'processing')",
(event["event_id"],),
)
except UniqueViolation:
return {"status": "already_processed"}
try:
perform_business_operation(event)
db.execute("UPDATE processed_events SET status='done' WHERE event_id=%s",
(event["event_id"],))
except Exception:
db.execute("UPDATE processed_events SET status='failed' WHERE event_id=%s",
(event["event_id"],))
raise
A check-then-insert without a constraint has a race condition under concurrency. The business operation itself should be idempotent or protected by an outbox pattern.
93. How would you structure a FastAPI service for an enterprise AI application?
Separate routes, services (LLM, retrieval), data access, security and schemas, so the AI layer can change without touching the API contract. Include request validation with Pydantic models, authentication and authorisation, centralised exception handling, structured logging with request IDs, health and readiness endpoints, rate limiting, API versioning, secrets from a secret manager, and unit plus integration tests. Use async HTTP clients for model and tool calls, with timeouts and connection-pool limits.
94. When do you use asyncio, threads, processes or a task queue?
- asyncio: many concurrent I/O calls (model APIs, tools, HTTP) inside one service.
- Thread pool: blocking I/O libraries you cannot make async.
- Multiprocessing: CPU-bound work such as parsing or local embedding.
- Task queue (Celery, SQS workers, etc.): long or retryable background jobs such as document ingestion.
Whatever the model, bound concurrency to respect downstream rate limits and add backpressure, so a traffic spike does not turn into a wave of 429s.
Advanced cloud, Kubernetes and Terraform scenarios
95. Your EKS-hosted AI application has high latency. Where do you look?
Work layer by layer and correlate metrics, logs and traces before deciding:
- Kubernetes: pod CPU throttling, memory pressure and OOMKilled events, HPA behaviour, node pressure, pending pods.
- Application: slow database queries, exhausted connection pools, blocking calls on the event loop, sequential calls that could run in parallel.
- AI path: prompt size, retrieval and reranking time, embedding latency, model latency and throttling.
- Network: load balancer, DNS, cross-AZ traffic.
kubectl top pods -A
kubectl top nodes
kubectl describe pod <pod-name>
kubectl get events -A --sort-by=.lastTimestamp
kubectl logs <pod-name> --previous
96. Response time grew from 2 seconds to 15 seconds after a release. How do you find the cause?
Break total latency into stages: authentication + retrieval + reranking + model + network + post-processing. Compare each stage before and after the release using traces; the stage that grew is your lead. Typical fixes: smaller context, fewer retrieved chunks, caching safe repeated results, streaming the response, a faster model for simple steps, fixing a slow query, or parallelising independent calls.
97. A Lambda function processes the same S3 event twice. How do you stop duplicate side effects?
S3 event notifications are delivered at least once, so duplicates are expected. Make processing idempotent: derive a unique key (bucket, object key, version ID or ETag), claim it with a DynamoDB conditional write or a database unique constraint, and skip if it already exists. Add a dead-letter queue and alerting for repeated failures.
98. A pod in EKS cannot reach a private AWS service. How do you troubleshoot?
Check, in order: DNS resolution from the pod, VPC DNS settings, route tables, security groups, network ACLs, the VPC endpoint and its endpoint policy, IAM permissions for the pod's role, Kubernetes network policies and any proxy settings. Separate network failures from authorisation failures: a connection that succeeds and then returns HTTP 403 points to IAM or an endpoint policy, not routing.
99. How do you achieve zero-downtime deployments for an AI service on Kubernetes?
Rolling updates with readiness, liveness and startup probes, at least two replicas, a PodDisruptionBudget, graceful shutdown that drains in-flight requests, backward-compatible API and schema changes, and canary or blue-green releases for risky changes. For prompt or model changes, add an evaluation gate before traffic shifts.
100. Blue-green deployment for an AI application: what is different from a normal app?
Deploy the new version (green) beside the current one (blue), run health checks, integration tests and the AI evaluation suite against green, then shift traffic gradually and keep blue ready for rollback. The AI-specific parts: pin model and prompt versions per environment, make sure index or schema changes stay compatible with blue, and compare quality metrics, not just error rates, during the shift.
101. Your Azure DevOps pipeline builds the image but the AKS deployment fails. How do you narrow it down?
Split the failure into stages: authentication (service connection, cluster credentials, Azure RBAC), image (ACR login, tag, pull permission), Kubernetes (manifest, namespace, RBAC, quotas, scheduling) and application (environment variables, secrets, probes, container logs). The pipeline log usually shows which stage failed; kubectl describe pod and kubectl get events show image-pull and scheduling errors.
102. A pod is in CrashLoopBackOff. What is your process?
kubectl describe pod <pod-name>
kubectl logs <pod-name> --previous
kubectl get events --sort-by=.lastTimestamp
Common causes: a startup exception, a missing secret or environment variable, a database it cannot reach, a wrong command, failing probes (often a liveness probe that is too aggressive for a slow-loading model) or out-of-memory kills. Find the actual error before raising limits or restarting.
103. The Terraform state file is corrupted. What do you do?
Stop all Terraform runs and preserve the current state and backups. Check the backend's version history (for example S3 versioning), confirm the lock status, and compare state with the real infrastructure using terraform state list, terraform state pull and terraform plan. Restore a known-good version if appropriate, reconcile differences carefully (import rather than recreate), and document what happened. Never run terraform apply blindly against a state you do not trust.
104. What is Terraform drift, and how do you handle it?
Drift is when real infrastructure no longer matches the configuration, for example someone edited a security group in the console. Detect it with a scheduled terraform plan, decide whether the change was intentional, then either codify it or revert it through the normal reviewed workflow. Treat unexplained drift on security resources as a security event.
105. Design a CI/CD pipeline for an agentic AI application.
- Lint, unit tests, static security analysis and dependency scanning.
- Build the container image with an SBOM, then scan it.
- Push to the registry; run
terraform planand get approval for infrastructure changes. - Deploy to staging.
- AI evaluation gate: retrieval quality, answer faithfulness, agent task success, tool-permission tests and prompt-injection tests against a fixed evaluation set.
- Approved, gradual production release with monitoring and automatic rollback.
The difference from an ordinary pipeline is step 5: prompts, retrieval, tool permissions and agent behaviour are tested, not only code.
106. How do you design disaster recovery for an enterprise AI platform?
Start from the RTO and RPO the business needs. Then cover infrastructure as code, database backups with tested restores, object storage replication, a plan to rebuild or restore the vector index, model endpoint failover, secrets and configuration recovery, image replication and multi-AZ deployment, plus cross-region recovery if required. Run restore drills: an untested backup is not a recovery plan.
Senior AI architecture scenarios
107. A customer wants an LLM inside its own data centre, with no data leaving. How do you approach it?
Assess GPU capacity against the model size, expected concurrency and latency targets, model licensing, data sensitivity and the customer's operations skills. Shortlist approved open-weight models, serve them with a production inference server in containers on the customer's Kubernetes or VMs, and add authentication, network isolation, GPU monitoring, model version management, evaluation and audit logging. Be honest about the quality and cost trade-off against hosted models.
108. A customer has 100 million records and wants AI search. Do you put everything in a vector database?
No. Classify the data and the questions first: relational stores for transactional lookups, a warehouse for analytics, a search engine for keyword and hybrid retrieval, a vector index for semantic similarity on unstructured text, object storage for documents. Benchmark indexing cost, query latency, filtering and update frequency before deciding.
109. The model provider has an outage. How should your application behave?
Health checks and circuit breakers, bounded retries with jitter, queueing for work that can wait, an approved fallback model that meets the same security and data-residency rules, graceful degradation (for example search results without a generated answer), a clear user message and an alert. Test the fallback path regularly so it works when needed.
110. An agent must run SQL against a production database. How do you secure it?
Put a query execution service between the agent and the database: a read-only role by default, an allowlist of schemas and tables, SQL parsing and validation, statement timeouts, row and result limits, tenant filters applied server-side, masking of sensitive columns, query audit logs, and human approval for any write. Never trust generated SQL because the model says it is safe.
111. How do you stop an agent taking unauthorised actions even if its prompt is manipulated?
Enforce authorisation outside the model. The model can propose an action; a trusted execution layer checks the user's permissions, the tool's scope and policy rules, validates the arguments, requires approval for sensitive actions and records an immutable audit entry. Least-privilege identities mean a manipulated prompt can only reach what the user could reach anyway.
112. How do you design an application that supports several LLM providers?
Add an LLM gateway between the application and the providers. It handles authentication, model routing, rate limits, retries, token and cost tracking, fallback and evaluation hooks. Do not pretend every provider is identical: expose provider-specific features where they matter and run the evaluation set per model before switching traffic.
113. How do you migrate a RAG application from one vector database to another?
Export documents and metadata, recreate the index schema, reuse embeddings only if the model and dimensions match (otherwise regenerate), import, then validate metadata and access filters. Run both indexes in parallel, compare retrieval quality and latency on the evaluation set, shift a small share of traffic, and keep the old index for rollback until the new one is proven.
114. How do you handle context-window overflow?
Retrieve fewer but better chunks (reranking), cut unnecessary prompt text, summarise older conversation turns, keep structured state outside the prompt, and use hierarchical retrieval for long documents. Choose a larger-context model only if needed. Never truncate arbitrarily: dropping the wrong paragraph silently changes the answer.
115. The multi-agent system is too expensive and hard to maintain. What do you recommend?
Ask what each agent adds that a deterministic function cannot. Remove or merge agents without measurable value, replace predictable steps with code, use smaller models for classification and routing, cache safe repeated calls and track cost per completed task. Multiple agents should be justified by results, not by architecture diagrams.
116. Design a production document-ingestion pipeline.
Upload -> Object storage -> Event -> Queue -> Workers
-> OCR / parsing -> Validation -> Chunking + metadata
-> Embeddings -> Vector index -> Evaluation + monitoring
Build in retries, a dead-letter queue, document versioning, duplicate detection, checkpoints, deletion handling and access-control metadata on every chunk, so permissions survive into retrieval.
117. The application works in development but fails in production. What do you compare?
Environment variables, secrets, IAM permissions, network paths and DNS, model deployment names and versions, dependency versions, resource limits, concurrency, data volume and quality, and real traffic patterns. Reproduce with a production-like configuration and synthetic or masked data, never with raw customer data on a laptop.
118. How would you design a multi-region, highly available AI application?
Start from RTO and RPO. Choose active-active or active-passive based on cost and data consistency, then plan regional deployments, traffic routing, database and object storage replication, vector index rebuild or replication, model endpoint availability in each region (including quota), regional secrets, monitoring and automated failover, and test failover regularly.
119. Migrate a monolith to an AI-enabled microservices architecture.
Do not rewrite everything. Map business boundaries and database dependencies, put an API boundary in front of the monolith, extract one low-risk capability first (often a read-only AI feature such as search or summarisation), add observability and CI/CD, and continue incrementally using the strangler pattern. Avoid splitting into more services than the team can operate.
120. You join a customer project on day one. How do you show value within 30 days?
Use the plan in question 81, but the interviewer is listening for priorities: meet the stakeholders, agree one measurable outcome, get access and data early, ship a thin working slice by the middle of the month, put it in front of real users with monitoring, and finish with a results report and a roadmap. Delivering one outcome beats starting ten features. See AI POC to production for the stage gates.
Extended questions 121–150: Python, RAG depth, LangGraph, cloud platforms and projects
These 30 questions extend the guide with topics interviewers increasingly ask in 2026 FDE loops: production Python patterns, embeddings and evaluation, LangGraph internals, Bedrock and Microsoft Foundry, Kubernetes autoscaling for LLM services, data design and presenting your portfolio.
121. How do you implement structured logging in a production Python service?
Log JSON lines rather than free text so the log platform can filter and aggregate. Every line carries a timestamp, level, service name, environment and a request or trace ID that follows the request through the AI layer, tool calls and downstream APIs.
import json, logging, sys, uuid, contextvars
request_id = contextvars.ContextVar("request_id", default="-")
class JsonFormatter(logging.Formatter):
def format(self, record):
return json.dumps({
"ts": self.formatTime(record),
"level": record.levelname,
"service": "kb-assistant",
"request_id": request_id.get(),
"msg": record.getMessage(),
})
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(JsonFormatter())
log = logging.getLogger("app")
log.addHandler(handler)
log.setLevel(logging.INFO)
request_id.set(str(uuid.uuid4()))
log.info("retrieval finished")Never log passwords, API keys, access tokens or full prompts containing personal data; log document IDs and token counts instead. Interviewers like hearing that you redact at the logger, not at each call site.
122. How do you validate request payloads and configuration in a Python AI service?
Use Pydantic models at every boundary: incoming API requests, LLM structured outputs and application settings. Reject bad input early with a clear 4xx error instead of letting it reach the model.
from pydantic import BaseModel, Field
from pydantic_settings import BaseSettings
class AskRequest(BaseModel):
question: str = Field(min_length=3, max_length=2000)
top_k: int = Field(default=5, ge=1, le=20)
class Settings(BaseSettings):
kb_id: str
aws_region: str = "ap-south-1"
request_timeout_s: float = 20.0
settings = Settings() # reads KB_ID, AWS_REGION ... from the environmentSettings come from environment variables or a secret store, never from code. Bounding fields such as top_k also caps cost and latency.
123. Write a function that splits text into overlapping chunks.
A favourite live-coding task because it tests off-by-one thinking. A simple word-based version:
def chunk_text(text: str, size: int = 200, overlap: int = 40) -> list[str]:
if overlap >= size:
raise ValueError("overlap must be smaller than size")
words = text.split()
chunks, step = [], size - overlap
for start in range(0, len(words), step):
piece = words[start:start + size]
if piece:
chunks.append(" ".join(piece))
if start + size >= len(words):
break
return chunksThen discuss what production adds: counting tokens with the embedding model's tokenizer instead of words, respecting headings and sentence boundaries, keeping tables intact, and storing metadata (document ID, section, page) with every chunk. See RAG chunking strategies.
124. Implement top-k similarity search in Python without a vector database.
This checks that you understand what a vector database actually does.
import numpy as np
def top_k(query_vec, doc_vecs, k=5):
q = query_vec / np.linalg.norm(query_vec)
d = doc_vecs / np.linalg.norm(doc_vecs, axis=1, keepdims=True)
scores = d @ q # cosine similarity
idx = np.argpartition(-scores, k)[:k]
return idx[np.argsort(-scores[idx])], scoresThis is exact (brute-force) search: fine for thousands of vectors, too slow for millions. Vector databases use approximate nearest-neighbour indexes such as HNSW to trade a little recall for much lower latency, and add metadata filtering, persistence and updates. See vector databases explained.
125. How do you test code that calls an LLM?
Split the tests into layers:
- Unit tests with the model client mocked, covering prompt assembly, parsing, validation, retries and error paths. These are fast and deterministic.
- Contract tests checking that structured outputs match the schema the downstream system expects.
- Evaluation runs against a versioned golden dataset of real questions with expected sources and answer criteria, scored on every prompt, model or retrieval change.
- Adversarial cases: prompt injection, out-of-scope questions and requests for other users' data.
Never assert on exact LLM wording; assert on properties such as the cited document, required fields and refusal behaviour.
126. How do you stream LLM responses to the browser from FastAPI?
Return a StreamingResponse that yields tokens as server-sent events, so users see the answer start in about a second instead of waiting for the full completion.
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
app = FastAPI()
@app.get("/ask/stream")
async def ask_stream(q: str):
async def events():
async for token in llm_stream(q): # your model client
yield f"data: {token}\n\n"
yield "data: [DONE]\n\n"
return StreamingResponse(events(), media_type="text/event-stream")Production points: disable proxy buffering for this route, send heartbeats on long pauses, stop generation when the client disconnects (to save tokens), and run output checks on the streamed text or on sentence boundaries if the use case is sensitive.
127. What are embeddings, and how do you choose an embedding model?
Embeddings are fixed-length numeric vectors that place text (or images) so that similar meanings sit close together, which makes semantic search possible. Choose a model by testing it, not by leaderboard alone:
- Recall on your own question-to-document pairs.
- Language coverage: multilingual if users mix English, Hindi or Telugu.
- Dimensions, cost and latency: more dimensions mean more storage per vector.
- Where it can run: data residency and the customer's approved cloud.
Changing the embedding model means re-embedding the whole corpus, so version the index. More in embeddings explained.
128. When does keyword search beat vector search?
Keyword (BM25) search wins on exact tokens: error codes, product SKUs, ticket numbers, policy IDs, people's names and rare acronyms that embeddings blur. Vector search wins on paraphrases ("close my account" vs "terminate a deposit account"). Enterprise assistants usually need both, so use hybrid search and merge the result lists (for example with reciprocal rank fusion), then rerank. A practical tip: route queries that look like identifiers straight to keyword search.
129. What is reranking, and why is a cross-encoder more accurate than the first retrieval step?
First-stage retrieval (bi-encoder vectors or BM25) is fast because the query and documents are encoded separately. A cross-encoder reranker reads the query and each candidate passage together, so it can judge relevance far more precisely, but it is too slow to run over the whole corpus. The usual pattern is to retrieve 30–50 candidates cheaply, rerank them, and pass only the best 3–8 to the LLM. That improves answer quality and cuts prompt tokens.
130. How do you build a RAG evaluation set, and which metrics do you track?
Collect 100–300 real questions from users or subject-matter experts, each with the documents that should answer it and the facts a correct answer must contain. Include questions with no answer in the corpus, and permission-sensitive ones.
| Stage | Metric |
|---|---|
| Retrieval | Recall@k, MRR (mean reciprocal rank), context precision |
| Generation | Faithfulness to retrieved context, answer correctness, citation accuracy |
| Behaviour | Correct refusals, permission leaks (must be zero) |
| Operations | p95 latency, cost per query |
Run the set in CI on every prompt, chunking, model or index change. See RAG interview questions for more on evaluation.
131. What should a production prompt template contain, and how do you manage prompt changes?
A production prompt states the role and task, the business context, how to treat retrieved content (as data, never as instructions), the output schema, refusal and escalation rules, and the citation format, with one or two examples where they help.
Treat prompts like code: store them in version control with an ID, review changes, run the evaluation set before release, roll out gradually, and log the prompt version with every request so a quality drop can be traced to a change.
132. What are prompt caching and semantic caching, and when are they safe?
Prompt caching (offered by several model providers) reuses the processed form of a long, unchanging prompt prefix, such as system instructions or a large shared document, which cuts cost and latency on repeat calls. It does not change answers.
Semantic caching returns a stored answer when a new question is similar enough to an old one. It is safe only for answers that are identical for everyone: never for user-specific or permission-filtered results, and the cache must be cleared when the source documents change.
133. Apart from temperature, which generation settings matter?
- Max output tokens: caps cost and latency, but too low a limit truncates JSON.
- Top-p (nucleus sampling): samples only from the most likely tokens whose probabilities add up to p. Tune temperature or top-p, not both at once.
- Stop sequences: end generation at a delimiter.
- Structured output or JSON mode: where the provider supports it, enforces a schema.
For extraction and tool calls, start with low randomness and validate the output; for drafting text, allow more variety.
134. How do you route requests between small and large models?
Send easy, high-volume tasks (classification, extraction, short FAQ answers) to a small, cheap model and escalate to a larger model only when needed: when the query is complex, the small model's output fails validation, or retrieval confidence is low. Measure quality per route on the evaluation set; routing that saves money but quietly drops accuracy is a regression. Log which model served each request for cost reporting.
135. What is LangGraph, and what are its core concepts?
LangGraph is a framework for building stateful agent and workflow applications as graphs. Its core concepts:
- State: a typed object shared and updated across steps.
- Nodes: functions that read the state and return updates (call a model, run a tool, validate).
- Edges and conditional edges: fixed or decision-based routing between nodes, including loops.
- Checkpointers: save the state after each step, per conversation thread, so runs can resume.
- Interrupts: pause the graph for human input or approval.
Compared with a free-running agent loop, the graph makes the control flow explicit and testable. See LangGraph interview questions and LangGraph for enterprise AI.
136. How do checkpointing and human-in-the-loop approval work in LangGraph?
Compile the graph with a checkpointer (an in-memory one for tests, a database-backed one such as Postgres in production) and pass a thread_id with each run. Before a risky node, for example "update the ticket", the graph calls an interrupt; the state is saved and the run stops. The application shows the proposed action to an authorised person, then resumes the same thread with their decision. Because the state is persisted, the approval can come minutes or days later, and the run survives restarts. Log who approved what, against the thread ID, for the audit trail.
137. How do you stop an agent from looping forever or running up a huge bill?
- A hard cap on steps or recursion depth per run.
- Token and cost budgets per request and per user, with a clean failure message when exceeded.
- Timeouts on every tool and model call.
- Detecting repetition: the same tool called with the same arguments several times means stop and escalate.
- Prefer a fixed workflow for steps that do not need model decisions.
- Alerts on unusual cost per conversation.
Interviewers want to hear that limits are enforced by the code around the agent, not requested in the prompt.
138. Single agent or multi-agent: how do you decide?
Start with one agent (or a workflow) and a small set of well-designed tools. Split into several agents only when there is a clear reason: different permissions or data boundaries, very different tool sets that confuse one agent, separate teams owning parts, or steps that can run in parallel. Every extra agent adds latency, cost, more failure points and harder debugging. A supervisor-plus-specialists design is common for IT operations, but each specialist still needs narrow tools and shared trace IDs.
139. What does Amazon Bedrock provide, and what would you use for RAG on it?
Amazon Bedrock is AWS's managed service for using foundation models from several providers through one API, with capabilities around them: Knowledge Bases for managed RAG, Guardrails for content and sensitive-data filters, Agents and the newer AgentCore services for building and running agents, plus model evaluation. For a document assistant you would typically use a Knowledge Base with the Retrieve or RetrieveAndGenerate API, Guardrails, IAM roles and CloudWatch. For depth, see Amazon Bedrock Knowledge Bases interview questions, AWS Bedrock interview questions and Bedrock AgentCore interview questions.
140. Amazon Bedrock or Microsoft Foundry: how do you choose for a customer?
Mostly by the customer's existing cloud, not by features. Check:
- Which cloud already holds their data, identity and landing zone (AWS IAM and Organizations, or Microsoft Entra ID and Azure subscriptions).
- Whether the models they need are available in their approved region.
- Networking and private connectivity requirements.
- Existing enterprise agreements and committed spend.
- Native connectors to their systems (SharePoint and Microsoft 365 often point to Azure).
Microsoft Foundry (formerly Azure AI Foundry) with Azure AI Search plays the role on Azure that Bedrock with Knowledge Bases plays on AWS. See Azure AI interview questions.
141. AWS Lambda or ECS/Fargate for an AI API?
Lambda suits spiky, event-driven work that finishes well within its 15-minute limit: document ingestion triggers, webhooks, light API calls. ECS on Fargate (or EKS) suits long-running services, streaming responses, large dependencies, steady traffic and connection pooling. Many FDE builds use both: Lambda for S3-triggered ingestion, containers for the chat API. Decide by execution time, cold-start tolerance, package size, concurrency and cost at the expected volume.
142. Why containerise GenAI applications, and how do you keep the images small and safe?
Containers give the same runtime in development, testing and the customer's production, which matters when you deploy into environments you do not control. Good practice:
- Multi-stage builds and slim base images; install only runtime dependencies.
- Do not bake model weights or documents into the image unless required; mount or download them.
- Run as a non-root user and scan images in CI.
- No secrets in the image; inject them at runtime from a secret store.
- Pin dependency versions for reproducible builds.
See Docker interview questions.
143. How would you autoscale an LLM-backed service on Kubernetes?
The Horizontal Pod Autoscaler (HPA) adds or removes pods based on metrics. CPU is often a poor signal for an app that mostly waits on a remote model API, so scale on requests in flight, queue depth or latency using custom metrics or KEDA. Set resource requests and limits, readiness probes and a PodDisruptionBudget. Also remember the external limit: scaling pods does not raise the model provider's rate limits, so add client-side rate limiting and backoff. See Kubernetes interview questions and Kubernetes for AI applications.
144. A customer wants to self-host an open model on Kubernetes with GPUs. What do you plan for?
- GPU node pools with the NVIDIA device plugin, and taints so only inference pods land there.
- A serving engine built for LLMs (for example vLLM or similar) for batching and throughput.
- Model weights in object storage, cached on nodes to avoid slow cold starts.
- Autoscaling on queue depth, with a minimum warm replica.
- Capacity and cost: GPUs are expensive when idle, so compare against a managed API at the expected volume.
- The same security, evaluation and monitoring as any other model endpoint.
145. How do you structure Terraform for an AI platform across dev, staging and production?
Use reusable modules (network, identity, storage, vector store, compute, monitoring) and thin environment configurations that call them with different variables. Keep state remote with locking and separate state per environment, run plan in CI with a human review before apply to production, and tag resources for cost tracking. Keep fast-changing items such as prompts and document ingestion out of Terraform; they belong in the application's release process. See Terraform interview questions.
146. Design database tables for conversations, feedback and traces in an AI assistant.
A minimal relational design:
conversations(id, user_id, tenant_id, created_at)
messages(id, conversation_id, role, content, model, prompt_version,
input_tokens, output_tokens, latency_ms, created_at)
retrievals(id, message_id, document_id, chunk_id, score, rank)
tool_calls(id, message_id, tool_name, arguments_json, status,
approved_by, created_at)
feedback(id, message_id, user_id, rating, comment, created_at)This lets you answer "which documents produced this bad answer?" and "which prompt version caused the drop?". Add retention rules and masking for personal data, and index by tenant and time.
147. Postgres with pgvector or a dedicated vector database?
pgvector is a strong default when the corpus is up to a few million chunks, the team already runs Postgres, and you want vectors, metadata filters, permissions and transactions in one place; it supports HNSW and IVFFlat indexes. A dedicated or managed vector store (OpenSearch, a managed vector service or a managed RAG service) makes sense for very large corpora, heavy query volume, built-in hybrid search, or when the customer prefers a managed service. Benchmark recall and latency with filters on real data before deciding. See vector database interview questions.
148. Which three portfolio projects best prepare you for FDE interviews, and how do you present them?
- Permission-aware RAG assistant (for example banking or HR policies): Python, FastAPI, a managed knowledge base or vector store, citations, refusals and an evaluation report.
- Agentic IT-operations assistant: LangGraph, MCP or tool calling against a ticketing and monitoring sandbox, human approval for write actions, and an audit trail.
- Cloud-deployed enterprise app: containers, Terraform, CI/CD with an evaluation gate, single sign-on, and dashboards.
Present each in a few minutes: problem, architecture diagram, your decisions and trade-offs, one failure you debugged, the metrics, and what you would do next. Our guide on how to explain your project in an interview covers the structure, and 10 AI engineer projects every FDE should build has more ideas.
149. A business leader asks, "Can AI replace our support team?" How do you answer?
Reframe it around outcomes and evidence: "Let's find out which ticket types AI can resolve safely and measure it." Propose a pilot that drafts answers for agents or automates a narrow, low-risk category, with a human checking the output, then report deflection rate, handling time, accuracy and customer satisfaction against a baseline. Be clear about limits: sensitive, ambiguous and high-impact cases stay with people, and someone must own quality. Honest expectations build more trust than a bold promise.
150. How do you hand over a solution to the customer's team at the end of an engagement?
- Runbooks for deployment, rollback, key rotation, re-indexing and common incidents.
- Architecture and decision records explaining why things are built as they are.
- Dashboards and alerts that the customer's on-call team already receives.
- The evaluation set and how to run it before every change.
- Paired working sessions where their engineers make a real change while you watch.
- A named owner for each component and an agreed support window.
A good handover means the system keeps improving after you leave, which is how FDEs earn the next engagement.
12-week FDE interview preparation roadmap
| Week | Focus | Practical outcome |
|---|---|---|
| 1 | FDE fundamentals and Python | A Python REST API with tests |
| 2 | SQL, APIs and integrations | A service that syncs data between two systems safely |
| 3 | LLMs, tokens and tool calling | An LLM feature with structured JSON output |
| 4 | RAG and vector search | Document search with citations and an evaluation set |
| 5 | LangGraph and agents | A stateful agent with human approval for write actions |
| 6 | MCP and enterprise tools | An MCP server exposing two read-only tools |
| 7 | AWS | The RAG app deployed on AWS with IAM and secrets |
| 8 | Azure | The same app on Azure with Entra ID sign-in |
| 9 | Docker, Kubernetes and Terraform | Containerised, provisioned with Terraform, running on Kubernetes |
| 10 | CI/CD, observability and SRE | A pipeline with an AI evaluation gate, dashboards and alerts |
| 11 | Security and system design | A threat model and one full enterprise design written up |
| 12 | Mock interviews | Coding, system design and customer role-play rounds, recorded and reviewed |
Build the five portfolio projects in 10 AI engineer projects every FDE should build as you go, each with a README, an architecture diagram, infrastructure code, tests, an evaluation report and a short demo video. For cloud depth, see AWS for AI engineers, Azure for AI engineers and Kubernetes for AI applications. For behavioural rounds, practise with behavioural interview questions for AI engineers. If you want guided, hands-on preparation with mock interviews, see the FDE PRO training program.
Key takeaways
- FDE is a high-impact engineering role: you own a production system and its business outcome inside one customer's environment.
- Start every design and role-play with discovery: users, data, systems, permissions and the success metric.
- Enforce security in code, not prompts: identity, authorisation, tool policy, validation, approval and audit.
- Debug RAG by classifying failures as retrieval, generation or data before changing anything.
- Measure everything that matters: quality on a test set, cost per successful task, latency per step.
- Use the simplest architecture that works; saying "this does not need an agent" is a strong signal.
- Communicate early and in writing, especially bad news, scope changes and risky customer requests.
Interview preparation checklist
- Two or three deployed projects you can defend: a permission-aware RAG assistant, an agent with approvals touching a ticketing system, and an integration service. See projects every AI FDE should build.
- For each project: a design note, an evaluation report with a test set, a security note and a cost estimate.
- Timed coding drills: messy file parsing, paginated and rate-limited APIs, schema mapping, idempotent webhooks, validated LLM JSON.
- Out-loud system design practice on questions 77 to 80, always ending with evaluation and operations.
- One clear explanation each of MCP, A2A, tool calling and agent memory, with when not to use them.
- Current cloud naming for your target stack: Amazon Bedrock and AgentCore, or Microsoft Foundry and Agent Framework.
- Customer role-plays with a friend playing a vague, sceptical or demanding stakeholder; record and review for over-promising.
- Six behavioural stories (ownership, ambiguity, failure, conflict, an RCA, saying no) and a two-minute executive update.
- A resume and portfolio that show outcomes, not tool lists.
- Read the job description as a test plan: prepare one example for each responsibility it lists.
FAQ
What skills are required to clear an FDE interview?
Practical Python and API coding, RAG and agent design, enterprise integration and identity, cloud deployment basics, security and evaluation, plus customer communication. Interviewers look for all of them working together rather than deep expertise in one.
How should I prepare for FDE interview questions in 2026?
Build two or three deployed projects you can defend, drill practical coding, practise system design out loud, rehearse customer role-plays with another person, and learn the current names of the cloud AI services your target employers use.
Is there DSA in FDE interviews?
Some loops include a data structures and algorithms screen, so keep basic fluency with arrays, hash maps, sorting and complexity. Most of the weight usually goes to practical coding, AI system design and customer rounds.
Which programming language should I use in FDE coding rounds?
Use the language you write most fluently unless the posting names one. Python is the most common choice for AI FDE roles because of its libraries for APIs, data handling and LLM frameworks.
Can freshers get Forward Deployed Engineer roles?
Freshers are more often hired into adjacent roles first, but a fresher with deployed projects, clear documentation and good communication can be competitive. Expect deep questions on every decision in your projects.
Is FDE a good career for DevOps and cloud engineers?
It can be a strong fit, because deployment, identity, networking and observability are a large part of FDE work. The usual gaps to close are RAG, agents, evaluation and customer-facing communication.
Is a Forward Deployed Engineer a support role?
No. An FDE designs, builds and deploys production systems inside a customer's environment and is accountable for the business outcome. Customer exposure adds to the engineering responsibility rather than replacing it.
Do I need cloud certifications for FDE interviews?
Certifications can help with resume screening, but interviews test whether you can deploy and secure a real system. A project running in a cloud account with infrastructure as code, private networking and scoped permissions is stronger evidence.
How do I prepare for the customer role-play round?
Practise with someone playing a vague or hostile customer. Ask discovery questions, summarise what you heard, be honest about limits, propose a small first step, and review a recording for over-promising.
If you want structured preparation rather than practising alone, Cloudsoft's FDE PRO program runs 12 weeks of live sessions, four engineering sessions and one Customer Engagement Lab each week, with 60+ labs, five enterprise projects and the simulated "GlobalBank" customer engagement capstone, plus mock interviews and placement support until you're placed. Join in the classroom beside Ameerpet Metro or live online, and call +91 96660 19191 for a free demo. If you want a broader foundation across AI, ML, cloud and cyber security first, look at the APEX AI, ML, Cloud and Cyber Security program.




