Start here. The platform you will be asked about was called Azure AI Studio in 2023, Azure AI Foundry in 2024–25, and is now branded Microsoft Foundry. Documentation for the older portal sits under "Foundry (classic)." Candidates who only know the old names sound two years behind, and almost every interview list online still uses them.
The five things being asked about right now: the Foundry rebrand, Foundry IQ (the knowledge plane that replaces hand-built RAG plumbing), Toolboxes and MCP endpoints, hosted agents in Foundry Agent Service, and agent evaluation and observability. All covered below.
Built from placement feedback across Cloud Soft Solutions' APEX batches — questions candidates actually faced at product companies, GCCs, consultancies and startups in Hyderabad, Bengaluru and Pune, plus what our trainers ask when screening. Answers are written the way you should say them: direct first, detail second.
Part 1 — What changed in 2026 (read this first)
1. Azure AI Foundry is now Microsoft Foundry
The naming lineage is Azure AI Studio → Azure AI Foundry → Microsoft Foundry. The Foundry portal reached general availability, and documentation for the previous portal now lives under Foundry (classic). Know all three names and which era each belongs to — interviewers use them interchangeably and are watching whether you follow.
2. Foundry IQ — the knowledge plane
Announced at Build 2026, Foundry IQ turns retrieval into a platform service: serverless retrieval, knowledge bases, multiple knowledge sources, Web IQ and agentic retrieval. The point of it is to remove hand-built RAG plumbing. If you are asked "how would you build RAG on Azure today," the modern answer starts with Foundry IQ, not with writing your own chunker.
3. Toolboxes, Tool Search and versioned MCP endpoints
A Toolbox groups tools behind a single versioned MCP endpoint. Tool Search matters once the tool list grows, because the model no longer needs every tool schema on every request — it retrieves the relevant ones. This directly addresses the context-bloat problem that breaks large agent deployments.
4. Hosted agents in Foundry Agent Service
Foundry Agent Service reached GA at Build 2025. Hosted agents — a managed runtime with sandboxed sessions, state, filesystem access and multi-framework support — were announced at Build 2026 with GA expected around early July 2026. Also new: Routines, Memory, Skills and Voice Live.
5. The model catalog is now multi-vendor and large
Past 1,900 models at Build 2025 and considerably larger since. Notably: Anthropic's Claude models reached GA on Foundry in June 2026, Grok 4.3 from xAI is in the catalog, plus Microsoft's own MAI models and Fireworks AI. Saying "Azure OpenAI" when you mean the model catalog is a dated reflex.
6. New agent tools: Computer Use, Browser Automation, Deep Research
Computer Use (preview) interacts with applications through their UI. Browser Automation (preview) runs real browser tasks in an isolated session via Microsoft Playwright Workspaces. Deep Research runs a multi-step research process on the o3-deep-research model grounded with Bing Search.
7. Evaluation moved into the agent lifecycle
You can now evaluate agents directly and convert agent traces into evaluation datasets (preview). Being able to describe how you measure an agent — not just build one — is the clearest senior/junior dividing line in 2026 interviews.
Part 2 — Fundamentals (0–2 years)
1. What is the difference between AI, machine learning, deep learning and generative AI?
AI is the broad field of machines performing tasks that require intelligence. ML is the subset that learns patterns from data rather than following coded rules. Deep learning is the subset of ML using multi-layer neural networks. Generative AI is the class of models that produce new content — text, image, audio, code — rather than only classifying or predicting.
2. What are Azure AI Services?
Pre-built APIs for common AI tasks, formerly branded Cognitive Services: Vision, Speech, Language, Document Intelligence (formerly Form Recognizer), Translator, Content Safety, and Content Understanding for multimodal extraction. You consume them via endpoint and key or managed identity — no training required.
3. Azure AI Services vs Azure Machine Learning vs Microsoft Foundry — when do you use each?
| Service | Use when |
|---|---|
| Azure AI Services | The task is standard — OCR, transcription, sentiment, translation. No training. |
| Azure Machine Learning | You are training custom models on your own data with your own pipelines. |
| Microsoft Foundry | You are building generative AI apps and agents — model selection, grounding, evaluation, deployment, governance. |
4. What is a token?
The unit a model reads and writes — roughly four characters or three-quarters of a word in English. Pricing, context limits and latency are all measured in tokens. Non-English text and code tokenise less efficiently, which is why an Indian-language application can cost noticeably more per request than its English equivalent.
5. What is a context window?
The maximum tokens a model can consider in one request — system prompt, conversation history, retrieved documents, tool schemas and the response, all together. Exceeding it truncates or errors. Managing it is most of what production engineering on LLM apps actually is.
6. What is an embedding?
A vector of floating-point numbers representing the semantic meaning of text, so that similar meanings sit close together in vector space. Embeddings power semantic search, clustering, classification and the retrieval half of RAG.
7. What is temperature, and what is top_p?
Temperature scales the randomness of token selection — 0 for deterministic extraction and classification, 0.7–1.0 for creative generation. top_p (nucleus sampling) limits selection to the smallest set of tokens whose cumulative probability exceeds p. Tune one, not both — adjusting both together makes behaviour hard to reason about.
8. What is a hallucination and why does it happen?
Confident output that is not grounded in fact or in the supplied source. It happens because the model predicts plausible next tokens rather than retrieving verified facts. Mitigation is grounding (RAG), instructing the model to say it does not know, groundedness evaluation, and citation-required output formats.
9. What is prompt engineering?
Structuring input to reliably get the output you need — clear role and task, explicit constraints, examples, output schema, and separation of instruction from data. In production it is a versioned artefact with tests, not a string someone tweaks in a notebook.
10. What is RAG?
Retrieval-Augmented Generation: retrieve relevant content from your own data at query time and pass it to the model as context, so answers are grounded in your sources rather than in training data. It solves freshness, private data and citability in one pattern.
11. RAG vs fine-tuning — how do you choose?
RAG for knowledge that changes, is private, or must be cited. Fine-tuning for behaviour — tone, format, a domain-specific style, or reducing prompt length. They are not alternatives: the common production answer is RAG for facts and a light fine-tune for format, and most problems need neither because a better prompt fixes them.
The trap: candidates propose fine-tuning to "teach the model our documents." Fine-tuning teaches style, not recall. Say so.12. What is grounding?
Constraining a model's response to supplied, verifiable source material. Grounding is the mechanism; groundedness is the measurable property — the degree to which the output is supported by the source, which Foundry can score for you.
13. What is an AI agent, as distinct from a chatbot?
An agent is given a goal and can decide which tools to call, in what order, over multiple turns, using memory and state, until the goal is met. A chatbot responds. The differences that matter operationally are tool access, autonomy and state — and all three are what make agents harder to secure and evaluate.
14. What is a vector database, and does Azure need one?
A store optimised for similarity search over embeddings. On Azure you would typically use Azure AI Search with vector fields, or Cosmos DB for NoSQL / PostgreSQL with pgvector where the data already lives there. With Foundry IQ, much of this becomes a managed knowledge base rather than infrastructure you assemble.
15. What is Content Safety?
An Azure AI service that classifies text and images across hate, sexual, violence and self-harm categories at four severity levels, and adds Prompt Shields for jailbreak and indirect prompt injection detection, protected material detection, and groundedness detection.
16. What is multimodal, and which Azure models are?
Accepting or producing more than one modality — text plus image, audio or video. GPT-4o and its successors handle text, image and audio; Content Understanding does multimodal extraction across documents, images, audio and video.
17. What is Document Intelligence?
Extracts structure from documents — text, tables, key-value pairs, selection marks — using prebuilt models (invoice, receipt, ID, health insurance card, W-2), a general layout model, and custom models you train on 5+ samples. For most document workloads it beats sending a scan to an LLM: cheaper, deterministic and it returns coordinates.
18. What is Azure AI Speech used for?
Speech-to-text, text-to-speech with neural voices, speech translation, speaker recognition, and custom neural voice under a gated approval process. In Foundry, Voice Live covers real-time conversational voice for agents.
19. What is the Responses API?
The stateful successor to Chat Completions for building conversational and agentic applications — it manages conversation state server-side and integrates tool calling more cleanly. It now also supports WebSocket mode, and the Realtime API has moved from preview to GA.
20. What is MCP?
Model Context Protocol — an open standard for exposing tools and data sources to models in a uniform way, so a tool built once works across frameworks and hosts. In Foundry, Toolboxes are published as versioned MCP endpoints. Knowing MCP is close to mandatory for agent roles in 2026.
Part 3 — Microsoft Foundry platform
21. What is Microsoft Foundry?
Microsoft's unified platform for building, grounding, evaluating, deploying and governing AI apps and agents. It brings the model catalog, agent runtime, retrieval, evaluation, observability and policy under one control plane, with native integration into Azure services and Microsoft 365 data, and support for open protocols like MCP.
22. What is a Foundry project, and why does the structure matter?
A project is the workspace containing your deployments, connections, data, evaluations and agents. It provides the RBAC and networking boundary. Structure matters because production and development should not share a project — you want separate quota, separate connections and separate access.
23. What is a connection in Foundry?
A managed reference to an external resource — AI Search, Storage, a data source, another model endpoint — with credentials held centrally. Using connections with managed identity rather than keys in code is the answer interviewers want.
24. What is the Foundry model catalog?
A single catalog spanning Azure OpenAI models, Microsoft's MAI models, Meta Llama, Mistral, Cohere, Phi, DeepSeek, xAI Grok, Anthropic Claude and partner-hosted options like Fireworks AI — past 1,900 models and growing. Deployment options differ by model: serverless API, managed compute, or Azure OpenAI-style deployment.
25. Serverless API vs managed compute — which do you pick?
Serverless is pay-per-token, no infrastructure, fastest to start, best for variable load. Managed compute gives you dedicated instances with predictable latency, network isolation and the ability to run models not offered serverless — better for steady high volume and strict data-residency requirements.
26. What are Toolboxes and Tool Search?
A Toolbox groups related tools behind one versioned MCP endpoint, so agents consume a stable contract rather than a growing list of individual tool definitions. Tool Search retrieves only the relevant tool schemas per request instead of sending all of them — which keeps context usage flat as your tool inventory grows.
27. What are Routines, Skills and Memory in Foundry?
Routines are repeatable multi-step agent procedures. Skills are packaged capabilities an agent can be granted. Memory gives agents persistent state across sessions. Together they are Foundry's answer to the fact that agents built as one long prompt do not survive production.
28. What is Foundry Local?
Running Foundry models on local hardware for development, offline scenarios and data that cannot leave the device. Useful for prototyping and for edge deployments; not a substitute for cloud capacity.
29. How do you secure a Foundry deployment at the network level?
Private endpoints and Private Link for the Foundry resource and every connected service, VNet integration for managed compute, disabling public network access, managed identity instead of keys, Key Vault for any remaining secrets, and customer-managed keys where policy requires. Then RBAC scoped per project.
30. How do you promote a Foundry solution from dev to production?
Separate projects per environment, infrastructure as code (Bicep or Terraform) for resources, prompts and agent definitions versioned in Git, an evaluation suite as a pipeline gate, and deployment through Azure DevOps or GitHub Actions. Say explicitly that an evaluation run is the gate — that is what separates an AI engineer from someone who ships prompts by hand.
Part 4 — Models & Azure OpenAI
31. What is Azure OpenAI Service, and how does it differ from OpenAI's own API?
The same models delivered through Azure with enterprise controls — regional deployment and data residency, private networking, managed identity and RBAC, Azure Monitor integration, compliance certifications, and the commitment that your prompts and completions are not used to train models. The trade-off is that new models often appear on OpenAI's API first.
32. How do you choose between a reasoning model and a general model?
Reasoning models (the o-series and successors) spend inference-time compute on multi-step problems — maths, complex code, planning, analysis. They cost more and are slower. General models are right for summarisation, extraction, chat and classification. The mature answer is that most production traffic should be on the cheapest model that passes your evaluation set, with reasoning reserved for the requests that need it.
33. What is a model router?
A deployment that automatically routes each request to an appropriate model based on complexity, so simple requests use a cheap model and hard ones escalate. It is the pragmatic way to get reasoning-model quality without reasoning-model cost across the whole workload.
34. What are small language models and when would you use one?
Models like the Phi family — small enough to run on modest hardware or at the edge, with strong performance on focused tasks. Use them for classification, routing, extraction, on-device scenarios and cost-sensitive high-volume paths.
35. What is a deployment, and what deployment types exist?
A named endpoint bound to a model and a capacity allocation. Types include Standard (pay-as-you-go, regional), Global Standard (routed worldwide, higher throughput, lower cost, less data-residency control), Data Zone (routed within a geography — the compromise), Provisioned (PTU) and Batch.
36. What are PTUs and when are they worth it?
Provisioned Throughput Units reserve dedicated capacity with predictable latency and no shared-tenant throttling. They are worth it when volume is high and steady, or when latency variance is unacceptable. Below the crossover volume, pay-as-you-go is cheaper — and being able to say "I'd model the crossover before committing" is the answer.
37. What are TPM and RPM?
Tokens per minute and requests per minute — the quota limits on a deployment. Exceeding them returns HTTP 429. TPM is allocated per deployment from a regional subscription pool, so multiple deployments compete for the same underlying quota.
38. How do you handle a 429?
Exponential backoff with jitter, honouring the Retry-After header. Then structurally: spread load across regions, request quota increases, use Global Standard for higher ceilings, add PTU for the guaranteed floor, and cache repeated prompts. Retry alone is not an architecture — say what you would change.
39. What is the Batch API?
Asynchronous processing at roughly half the cost with a 24-hour completion target, using separate quota from real-time deployments. Right for bulk classification, offline enrichment, embedding generation and evaluation runs.
40. What is prompt caching?
Reuse of the model's processing of a repeated prompt prefix, reducing cost and latency on the cached portion. It rewards putting stable content — system prompt, few-shot examples, schema — at the start of the prompt and variable content at the end. Getting the ordering right is free money.
41. What is structured output / JSON mode?
Constraining the model to emit valid JSON conforming to a supplied schema, so downstream parsing does not fail. Prefer schema-enforced structured output over asking politely for JSON in the prompt and then writing regex to repair it.
42. What is function calling / tool calling?
Supplying the model with function definitions so it can return a structured request to invoke one, which your code executes and returns. The model decides whether and which; your application always controls execution. That boundary is a security question as much as a design one.
Part 5 — Prompting & context
43. What goes in a well-structured system prompt?
Role, task, explicit constraints and refusals, output format, tone, and how to behave when information is missing. Keep instructions separate from user data with clear delimiters — merging them is how prompt injection gets in.
44. Zero-shot, few-shot and chain-of-thought — when does each apply?
Zero-shot for well-understood tasks. Few-shot when output format or edge-case handling needs demonstrating — three to five diverse examples usually beats twenty similar ones. Chain-of-thought for multi-step reasoning, though with reasoning models you generally should not prompt for it explicitly; the model already does it internally.
45. What is prompt injection, and how is indirect injection different?
Direct injection is a user telling the model to ignore its instructions. Indirect injection hides instructions in content the model retrieves — a document, a web page, an email — so the attack arrives through your RAG pipeline rather than the chat box. Indirect is the harder problem and the one agents make worse, because agents act on what they read.
Strong answer: "Prompt Shields for detection, treat all retrieved content as untrusted data, never grant a tool more privilege than the least-trusted content in context, and require human approval for irreversible actions."46. How do you manage a long conversation that exceeds the context window?
Summarise older turns into a running summary, keep the most recent N turns verbatim, store full history externally and retrieve only relevant parts, and keep tool schemas out of context via Tool Search. Truncating from the front blindly loses the system prompt — a common bug.
47. What is context rot, and why does it matter for agents?
Degrading answer quality as context fills with accumulated history, retrieved chunks and tool schemas — the model attends less reliably to any one part. It is why Tool Search, memory summarisation and aggressive retrieval filtering exist. Long context is a budget, not a free resource.
48. How do you version and test prompts in production?
Prompts in source control, a golden dataset of inputs with expected characteristics, automated evaluation on every change, and a canary rollout comparing metrics before full release. Treat a prompt change like a code change, because it has the same blast radius.
Part 6 — RAG, Azure AI Search & Foundry IQ
49. Walk me through a RAG pipeline end to end.
Ingest → chunk → embed → index → (query) embed the query → retrieve top-k → rerank → assemble prompt with citations → generate → evaluate. Each stage has failure modes, and in interviews the follow-up is always which stage do you tune first. The answer is retrieval — a generation problem is usually a retrieval problem wearing a disguise.
50. How do you choose a chunking strategy?
Fixed-size with overlap is the baseline (roughly 300–800 tokens, 10–15% overlap). Better: chunk on document structure — headings, sections, table boundaries — so a chunk is a coherent unit. Preserve metadata (title, section, page) for filtering and citation. Chunk size is the single highest-leverage RAG parameter and should be tuned against an evaluation set, not guessed.
51. What is hybrid search and why is it better than pure vector search?
Combining keyword (BM25) and vector retrieval, fused with Reciprocal Rank Fusion. Vector search handles paraphrase and semantics; keyword search handles exact identifiers, product codes, names and acronyms that embeddings blur. Hybrid plus semantic reranking is the default recommendation in Azure AI Search for good reason.
52. What is the semantic ranker?
A second-stage reranking model that rescores the top results from the initial retrieval using deeper language understanding, and can return captions and answers. It typically produces the largest single quality jump in an Azure RAG pipeline for the least engineering effort.
53. What is integrated vectorisation?
Azure AI Search handling chunking and embedding inside the indexer pipeline, rather than you running a separate ingestion job. It covers both initial indexing and query-time embedding, which removes a common source of drift where index and query embeddings use different models.
54. What is Foundry IQ and how does it change RAG?
Foundry IQ is a knowledge plane providing serverless retrieval, knowledge bases, multiple knowledge sources, Web IQ and agentic retrieval as managed capabilities. Instead of assembling chunker, embedder, index, reranker and query orchestration yourself, you register knowledge sources and query them. It reduces custom RAG plumbing substantially — which is exactly how you should phrase it.
55. What is agentic retrieval?
Letting the model decompose a complex question into sub-queries, retrieve for each, and synthesise — rather than a single embedding lookup on the raw question. It markedly improves multi-hop and comparative questions, at the cost of more calls and latency.
56. How do you enforce security trimming in RAG?
Store the permitted principals or security groups as filterable metadata on each document, and apply a filter derived from the caller's identity at query time. Never rely on the prompt to withhold content the retriever returned — if it reached context, treat it as disclosed.
This one separates candidates. Many will answer "instruct the model not to reveal it." That is not access control.57. Your RAG system returns irrelevant chunks. Diagnose it.
Test retrieval in isolation from generation first. Then check: chunk size too large (topic dilution) or too small (lost context); embedding model mismatch between index and query; missing hybrid search for identifier-style queries; no reranking; missing metadata filters; and query formulation — user questions often need rewriting before embedding.
58. How do you keep an index fresh?
Indexers on a schedule with change detection for incremental updates, deletion tracking so removed source documents leave the index, and a full rebuild path for schema changes. State your RPO — "documents are searchable within 15 minutes" is an answer; "we re-index nightly" invites the question of what happens at 9am.
59. When is RAG the wrong tool?
When the answer requires aggregation across the whole corpus ("how many contracts expire this quarter") — that is a database query. When the task is behavioural rather than factual — that is fine-tuning or prompting. And when the corpus is small enough to fit in context, where retrieval only adds a failure mode.
Part 7 — Agents, tools & MCP
60. What is Foundry Agent Service?
The managed service for building and running agents — model, instructions, tools, memory and state in one hosted runtime, with threads, tool orchestration and observability handled for you. GA since Build 2025, with hosted agents adding managed sandboxed sessions, filesystem access and multi-framework support.
61. What tools can a Foundry agent use?
Code Interpreter, File Search, Function calling, OpenAPI-described APIs, Azure Functions, Grounding with Bing Search, Azure AI Search, Fabric data agents, Deep Research (o3-deep-research + Bing grounding), Browser Automation via Playwright Workspaces, Computer Use, and any MCP server.
62. What is the Microsoft Agent Framework?
The consolidation of Semantic Kernel and AutoGen into a single framework — Semantic Kernel's enterprise orchestration and plugin model, AutoGen's multi-agent conversation patterns. If asked which to learn, the answer is the Agent Framework; the older two are its lineage.
63. Single agent vs multi-agent — how do you decide?
Start single. Split only when you have genuinely distinct responsibilities, separate tool permissions, or different models suited to different sub-tasks. Multi-agent multiplies latency, cost and failure modes, and most systems described as multi-agent would work better as one agent with well-designed tools.
What impresses: arguing against multi-agent. Interviewers hear enthusiasm for it constantly; judgement is rarer.64. What are the common multi-agent orchestration patterns?
Sequential (pipeline), concurrent (fan out, aggregate), handoff (agent transfers control based on intent), group chat (agents deliberate under a manager), and magentic (a planner dynamically assembles the approach). Choose by the shape of the work, not by novelty.
65. How do you keep an agent from taking a destructive action?
Least-privilege tool design — read and write as separate tools, scoped credentials per tool. Human-in-the-loop approval for irreversible operations. Idempotency and dry-run modes. Rate and spend limits. Full tracing of every tool call. And critically: an agent's effective privilege should never exceed the trust level of the least-trusted content in its context.
66. What is agent memory and what types are there?
Short-term (thread state within a session) and long-term (persisted across sessions — user preferences, prior outcomes, learned facts). Foundry's Memory feature covers persistence. The design question is retention and deletion policy, since agent memory is personal data.
67. Why does Tool Search matter at scale?
Every tool definition sent on every request consumes context and dilutes attention. With fifty tools, schemas can dominate the prompt. Tool Search retrieves only the relevant schemas per request, keeping context flat and tool selection accurate as the inventory grows.
68. How do you deploy an agent to Microsoft 365 Copilot or Teams?
Foundry provides a governed publishing path from the agent definition into Microsoft 365 Copilot and Teams, so the agent inherits organisational policy and identity rather than being rebuilt per surface. This is a strong point to raise for enterprise roles.
69. What does a production agent's failure handling look like?
Tool-level timeouts and retries with backoff; a maximum step or iteration budget to stop loops; fallback to a simpler model or a canned response; structured error returns to the model so it can recover rather than hallucinate; a circuit breaker on repeated failure; and an alert on step-budget exhaustion, which is usually the first signal something has regressed.
Part 8 — Fine-tuning & optimisation
70. What fine-tuning methods does Azure support?
Supervised fine-tuning (SFT) on prompt–completion pairs, Direct Preference Optimisation (DPO) on preferred versus rejected pairs, distillation from a larger model's outputs into a smaller one, and reinforcement fine-tuning with a grader for objectively verifiable tasks. Build 2026 added Frontier Tuning for large-model customisation.
71. How much data do you need to fine-tune?
Meaningful improvement typically starts around 50–100 high-quality examples and improves to a few thousand; beyond that returns flatten. Quality and consistency beat volume — a hundred examples that agree with each other outperform a thousand that contradict.
72. What is distillation and when is it the right move?
Using a large model's outputs to train a smaller one for a narrow task. Right when you have a working expensive solution, stable requirements and high volume — you keep most of the quality at a fraction of cost and latency. Wrong when the task is still changing.
73. What is catastrophic forgetting?
A fine-tuned model losing general capability it previously had, because training over-specialised it. Mitigate with lower learning rates, fewer epochs, mixing general examples into the training set, and always evaluating on a general benchmark alongside your task-specific one.
74. Give me the optimisation ladder in cost order.
Prompt improvement → few-shot examples → structured output → prompt caching and prompt ordering → RAG → model downgrade with evaluation → routing → batch for offline work → distillation → fine-tuning → PTU. Work down the list, not up. Most teams jump to fine-tuning at step one and spend ten times what they needed to.
Part 9 — Evaluation & observability
75. What built-in evaluators does Foundry provide?
Quality: groundedness, relevance, coherence, fluency, similarity, retrieval and F1. Safety: violence, sexual, self-harm, hate and unfairness, protected material, indirect attack. Agent-specific: intent resolution, task adherence, tool call accuracy. Plus custom evaluators you define.
76. What is LLM-as-judge, and what are its limits?
Using a model to score outputs against a rubric — scalable and correlates reasonably with human judgement. Limits: position bias, verbosity bias, self-preference for its own outputs, and instability across runs. Mitigate with a strong rubric, a different judge model than the one under test, and periodic human calibration on a sample.
77. How do you evaluate an agent, as opposed to a single response?
Evaluate the trajectory, not only the final answer: did it resolve the user's intent, adhere to the task, select the right tools, pass correct parameters, and recover from failures? Foundry supports evaluating agents directly and converting agent traces into evaluation datasets, which is the practical way to build a regression suite from real traffic.
This is the senior signal in 2026. Anyone can demo an agent. Describing how you measure one — and how you turn production traces into a test set — is what gets you the offer.78. What does observability mean for an AI application?
Distributed tracing across the request — model calls, retrievals, tool invocations, token counts, latency per span — exported through OpenTelemetry into Application Insights, plus continuous online evaluation sampling production traffic. Without traces you cannot answer "why did it say that," which is the only question that ever gets asked.
79. How do you build a golden dataset?
Sample real user queries across the distribution, including edge cases and known failures. Have domain experts write or approve expected outputs. Keep it small enough to run on every commit (100–300 items) with a larger nightly set. Version it, and add every production failure to it — that is how the suite stays honest.
80. What is the AI Red Teaming Agent?
An automated adversarial testing capability that probes your application with known attack techniques — jailbreaks, injections, harmful content elicitation — and reports which succeeded. It is scale for the routine attacks; it does not replace human red teaming for novel ones.
Part 10 — Responsible AI & security
81. What are Microsoft's Responsible AI principles?
Fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability. Worth knowing verbatim — it is a common opener and an easy mark to lose.
82. How do content filters work in Azure OpenAI?
Both prompts and completions are classified across hate, sexual, violence and self-harm at four severity levels, with configurable thresholds per category and direction. Optional filters cover jailbreak detection, protected material for text and code, and groundedness. Turning filters off requires an approved application.
83. What are Prompt Shields?
Content Safety detection for jailbreak attempts in user input and for indirect prompt injection in retrieved documents. The second is the one that matters for RAG and agents, where the attack arrives inside content the system fetched rather than from the user.
84. How do you handle PII in an AI application?
Detect and redact before the model sees it — Azure AI Language PII detection, or Presidio. Decide whether the use case needs the real value at all. Do not log raw prompts containing PII; log hashes or redacted forms. Apply retention limits, and remember that agent memory is personal data subject to deletion requests.
85. Is my data used to train Microsoft's models?
No. Prompts and completions in Azure OpenAI are not used to train, retrain or improve Microsoft or OpenAI models, and are not shared with other customers. Data may be stored briefly for abuse monitoring, which can be disabled through an approved application for eligible customers.
86. How do you authenticate to Azure AI services without keys?
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
client = AIProjectClient(
endpoint="https://<project>.services.ai.azure.com/api/projects/<name>",
credential=DefaultAzureCredential(),
)Managed identity in Azure, developer credentials locally, no secrets in code or config. Assign the minimum RBAC role — reader-equivalent for inference, not Contributor.
87. What does the EU AI Act mean for a system you build?
Risk-tiered obligations: unacceptable-risk uses prohibited; high-risk uses (employment, credit, education, biometrics) require risk management, data governance, logging, human oversight and conformity assessment; limited-risk requires disclosure that the user is interacting with AI. Even for Indian delivery teams this matters, because clients in scope push the obligations down the supply chain.
88. What is a transparency note?
Microsoft's published documentation of a service's intended uses, limitations, fairness considerations and evaluation approach. Interviewers ask because reading them is a signal you think about deployment context rather than only capability.
Part 11 — Azure Machine Learning
89. What is an Azure ML workspace?
The top-level resource holding experiments, models, datastores, compute, endpoints and the registry, backed by Storage, Key Vault, Container Registry and Application Insights. It is the RBAC and lineage boundary for ML work.
90. Compute instance vs compute cluster vs inference cluster?
A compute instance is a single-user development VM. A compute cluster autoscales for training and batch jobs, scaling to zero when idle. An inference cluster (AKS) serves real-time predictions at scale. Cost questions usually hinge on whether your clusters actually scale to zero.
91. Managed online endpoint vs batch endpoint?
Online for low-latency real-time scoring with autoscaling and blue-green deployment via traffic splitting. Batch for large asynchronous scoring jobs writing to storage. Choose by latency requirement and payload size.
92. What is MLflow's role in Azure ML?
The native tracking, model packaging and registry interface — experiments, parameters, metrics, artefacts, and the model format used for deployment. Using MLflow makes your work portable rather than Azure-locked, which is worth saying.
93. What is data drift and how do you monitor it?
Divergence between production input distribution and training distribution, degrading accuracy silently. Monitor with statistical distance measures per feature against a baseline, alert on threshold breach, and pair with prediction drift and — where labels arrive late — delayed accuracy measurement.
94. What is responsible AI dashboard in Azure ML?
Tooling for error analysis, model interpretability (SHAP-based), fairness assessment across cohorts, counterfactuals and causal analysis — assembled into one view for model review and sign-off.
Part 12 — Cost, quota & deployment
95. Your GenAI application costs are three times budget. What do you do?
Instrument first — cost per request by feature, model and user, or you are guessing. Then: check whether a cheaper model passes evaluation; add prompt caching and reorder prompts so stable content is at the front; trim retrieved context and system prompt; cap output tokens; move offline work to Batch; add a model router; cache repeated queries; and only then consider PTU or distillation. Most overspend is retrieved context nobody measured.
96. How do you decide between region-specific and Global Standard deployment?
Global Standard gives higher throughput and lower price by routing worldwide, but weakens data-residency guarantees. Data Zone routes within a geography as a middle ground. If the client has residency obligations — common for Indian BFSI and healthcare — regional or Data Zone is the answer regardless of price.
97. How do you plan capacity for a new AI application?
Estimate requests per second at peak, average input and output tokens per request, then derive TPM. Compare against regional quota. Load-test before launch. Plan a fallback region. And budget for the fact that real prompts are always longer than the ones in your prototype.
98. What does an AI landing zone include?
Hub-and-spoke networking with private endpoints, Foundry and Azure OpenAI resources per environment, API Management in front of model endpoints for throttling, routing and per-team token accounting, Key Vault, Log Analytics and Application Insights, policy for allowed regions and models, and cost allocation by tag. APIM in front is the pattern most enterprises converge on.
Part 13 — Scenario questions (3–8 years)
99. Build a support assistant over 40,000 internal documents with strict access control.
Ingest to Storage with document-level permission metadata. Index in Azure AI Search with integrated vectorisation, hybrid search, semantic reranking and filterable security fields — or register as a Foundry IQ knowledge source. At query time derive a filter from the caller's Entra identity so retrieval is trimmed before generation. Ground responses with mandatory citations. Content Safety with Prompt Shields on retrieved content. Evaluate for groundedness and retrieval quality against a golden set. Trace everything to App Insights. Deploy behind APIM with per-team quota.
100. A customer says the assistant "makes things up." How do you investigate?
Get specific failing examples first — "hallucinating" often turns out to be retrieval missing the document. Replay each through the trace: did retrieval return the right chunk? If no, it is a retrieval problem — chunking, hybrid search, reranking. If yes and the answer still diverged, it is a generation problem — tighten the prompt to require citation and to refuse when sources are insufficient, lower temperature, and add groundedness evaluation as a gate. Then add every case to the golden set.
101. Design an agent that can raise purchase orders in SAP.
Split tools by privilege: read-only lookup tools the agent uses freely, and a single write tool that creates a draft PO. Human approval required before submission — an agent should not be the last signature on a financial commitment. Scoped service credentials per tool, not a shared admin account. Idempotency keys to prevent duplicate creation on retry. Step budget and spend cap. Full trace of every call, retained for audit. And explicit handling of indirect injection, since supplier emails and documents entering context are untrusted.
What they want: that you designed for the failure case first. Candidates who describe the happy path and stop are marked as inexperienced.102. Latency is 8 seconds; the business wants under 2. What do you change?
Measure the span breakdown first — retrieval, model, tools. Then: stream the response so time-to-first-token is what the user feels; use a smaller or routed model; cut retrieved context; parallelise independent retrievals and tool calls; enable prompt caching; use PTU if queueing is the cause; move non-essential work post-response. Also question the requirement — for a research task, users tolerate latency if progress is visible.
103. The business wants to fine-tune GPT on 200 PDFs so it "knows our products."
Push back, with reasoning. Fine-tuning teaches format and behaviour, not factual recall — and the products will change next quarter, requiring retraining. RAG gives current answers with citations and updates by re-indexing. Propose RAG first, measure, and consider a small fine-tune later only for tone or output format. This question is testing whether you will tell a stakeholder they are wrong.
104. How would you migrate a working prototype into production?
Separate Foundry projects per environment. Infrastructure as code. Prompts and agent definitions in Git with review. A golden evaluation set as a CI gate. Private networking and managed identity replacing keys. APIM in front for throttling and quota. Tracing to App Insights with online evaluation sampling live traffic. Canary rollout with metric comparison. Documented rollback. And a named owner for the evaluation set, because unowned test suites decay.
105. Regulated client, data cannot leave India. What are the constraints?
Deploy in Central India or South India and check model availability there — the newest models often arrive in those regions later, so model choice may be constrained. Use regional Standard rather than Global Standard. Request abuse-monitoring opt-out if the data cannot be stored at all. Private endpoints, no public access, customer-managed keys. And set expectations early: the client may have to accept a slightly older model, and that trade-off is better surfaced in week one than month six.
Part 14 — SDK & code round
106. Call a Foundry model in Python.
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
project = AIProjectClient(
endpoint="https://<project>.services.ai.azure.com/api/projects/<name>",
credential=DefaultAzureCredential(),
)
client = project.get_openai_client(api_version="2025-01-01-preview")
resp = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "Answer only from the supplied context."},
{"role": "user", "content": "Summarise the Q3 revenue drivers."},
],
temperature=0,
max_tokens=500,
)
print(resp.choices[0].message.content)Note: azure-ai-projects reached 2.2.0 across Python, JS/TS and .NET, adding external agent definitions, skills, toolboxes, a model weight registry, routines and optimisation jobs. Pin your version — this SDK moves fast.
107. Write a retry wrapper that handles 429 correctly.
import time, random
from openai import RateLimitError
def call_with_retry(fn, max_attempts=5):
for attempt in range(max_attempts):
try:
return fn()
except RateLimitError as e:
if attempt == max_attempts - 1:
raise
retry_after = getattr(e, "retry_after", None)
delay = retry_after if retry_after else (2 ** attempt)
time.sleep(delay + random.uniform(0, 1)) # jitterThe details interviewers look for: honouring Retry-After, exponential growth, jitter, and a bounded attempt count.
108. Create an agent with a tool.
agent = project.agents.create_agent(
model="gpt-4o",
name="orders-assistant",
instructions="You help staff check order status. Use get_order_status for any order question.",
tools=[{
"type": "function",
"function": {
"name": "get_order_status",
"description": "Return the current status of an order by ID.",
"parameters": {
"type": "object",
"properties": {"order_id": {"type": "string"}},
"required": ["order_id"],
},
},
}],
)109. Run an evaluation.
from azure.ai.evaluation import evaluate, GroundednessEvaluator, RelevanceEvaluator
model_config = {"azure_endpoint": ENDPOINT, "azure_deployment": "gpt-4o"}
result = evaluate(
data="golden_set.jsonl",
evaluators={
"groundedness": GroundednessEvaluator(model_config),
"relevance": RelevanceEvaluator(model_config),
},
output_path="./eval_results.json",
)
print(result["metrics"])Wire the metric thresholds into your pipeline so a regression fails the build.
110. Enable tracing.
from azure.monitor.opentelemetry import configure_azure_monitor
from azure.ai.projects.telemetry import AIAgentsInstrumentor
configure_azure_monitor(connection_string=APP_INSIGHTS_CONN_STR)
AIAgentsInstrumentor().instrument()Every model call, retrieval and tool invocation then appears as a span in Application Insights with token counts and latency.
Part 15 — Salary bands and the HR round
Indicative 2026 ranges for Azure AI / GenAI engineering roles in India. Product companies, GCCs and US-shift roles pay above these; service-company bands sit at the lower end.
| Experience | Typical title | Range (₹ LPA) |
|---|---|---|
| 0–2 years | AI/ML Engineer (Associate) | 5 – 10 |
| 2–4 years | Azure AI Engineer | 10 – 19 |
| 4–7 years | Senior AI Engineer / GenAI Engineer | 19 – 34 |
| 7–10 years | AI Architect / Lead | 34 – 58 |
| 10+ years | Principal AI Architect / Practice Head | 58 – 95+ |
Questions you should ask them
- Is anything in production with real users, or is this still pilots?
- Do you have an evaluation suite, and does it gate deployment?
- Are you on Foundry, or calling Azure OpenAI endpoints directly?
- Who owns cost, and is spend per feature measured?
- Are you building agents with tool access to production systems, and what is the approval model?
Certifications that carry weight in 2026
- AI-102: Azure AI Engineer Associate — the core certification for these roles
- AI-900: AI Fundamentals — useful for career changers
- DP-100: Data Scientist Associate — where the role includes classical ML
- AZ-204 — because you will be building and deploying applications, not only calling models
- DP-600 / DP-700 — where the role touches Microsoft Fabric
Prepare for these interviews with Cloud Soft Solutions
Our APEX program covers AI, GenAI, Cloud and Cyber Security — including a Microsoft Foundry track spanning RAG, agents, MCP, evaluation and Responsible AI, with live lab environments, capstone projects, mock interviews and resume preparation. 5,500+ alumni placed.
📍 513, 5th Floor, Aditya Enclave, Nilagiri Block, Beside Ameerpet Metro Station, Ameerpet, Hyderabad – 500016
📞 +91 96660 19191 · +91 99496 16388 · ✉️ info@cloudsoftsol.com
How to prepare (do not memorise this list)
- Build one RAG application end to end on your own documents, and measure it.
- Build one agent with at least two tools and deliberately break it — bad tool responses, injection in a retrieved document, a loop. Fix each.
- Create a golden dataset of 50 items and run Foundry evaluators against it.
- Do a cost teardown. Know your cost per request and where it goes.
- Read "What's new in Microsoft Foundry" monthly.
- Prepare three stories: something you shipped, something that failed in production and what you changed, and a time you talked a stakeholder out of the wrong approach.
Note on accuracy: the Azure AI platform ships continuously and features move between preview and GA quickly — naming in particular has changed twice in two years. Details here reflect the platform as of August 2026. Verify against Microsoft Learn and the Microsoft Foundry blog before any interview.