Microsoft Foundry is where Azure AI applications and agents are now built. This tutorial explains how Foundry fits into the Azure AI ecosystem, walks through setup in the portal, makes a first model call and creates a prompt agent in Python with the current azure-ai-projects 2.x SDK, deploys your own Python code as a hosted agent with the Azure Developer CLI, and then covers RAG, MCP tools, memory, security, evaluation, CI/CD and seven enterprise use cases.
It is written for Azure AI Engineers, GenAI Engineers, Agentic AI Engineers, Python developers, Forward Deployed Engineers, cloud and DevOps engineers and solution architects. If you are preparing for interviews, read it together with our 130 Microsoft Azure AI services interview questions.
Contents
- 1. Introduction
- 2. What is Microsoft Foundry?
- 3. Foundry vs Azure AI Foundry vs Azure OpenAI
- 4. Architecture
- 5. Setup in the portal
- 6. Models and deployments
- 7. First model call and prompt agent in Python
- 8. Prompt engineering
- 9. Foundry Agent Service
- 10. Deploy your own Python code as a hosted agent
- 11. RAG with Foundry
- 12. MCP and tools
- 13. Memory and state
- 14. Security
- 15. Evaluation and observability
- 16. CI/CD and production
- 17. Enterprise use cases
- 18. Foundry vs Amazon Bedrock
- 19. Troubleshooting
- 20. Foundry interview questions
- 21. 10-week roadmap and final project
1. Introduction to Microsoft Foundry
Enterprises are moving past simple chatbots towards applications that understand a request, retrieve company knowledge, call tools, run workflows and hand off to people when needed. Microsoft Foundry is Microsoft's platform for building, deploying and governing those applications and agents.
Take a bank that wants an intelligent banking assistant. It should:
- Answer questions about banking policies
- Search internal knowledge bases
- Retrieve loan eligibility requirements
- Summarise financial documents
- Identify customer intent
- Call approved backend APIs
- Escalate complex requests to staff
- Protect confidential customer information
- Keep monitoring and audit records
A production build might combine Microsoft Foundry, a model deployment (for example an Azure OpenAI model), Foundry Agent Service, Azure AI Search, Azure Functions, Microsoft Entra ID, Key Vault and Azure Monitor. The rest of this tutorial shows how those pieces fit together.
2. What is Microsoft Foundry?
Microsoft describes Foundry as a platform that unifies agents, models and tools under a single management grouping, with built-in tracing, monitoring, evaluations and enterprise setup, and with unified RBAC, networking and policies under one Azure resource provider. (Microsoft Learn: What is Microsoft Foundry?)
You can work with it in several ways:
- Call models directly from your code
- Build prompt agents from instructions, a model and tools
- Build voice-based prompt agents for real-time spoken conversations
- Deploy hosted agents that run your own code and framework
- Connect agents to knowledge and tools
- Trace and evaluate everything
Developer surfaces include the Foundry portal (ai.azure.com), SDKs for Python, C#, JavaScript/TypeScript and Java, the Azure Developer CLI (azd), Visual Studio Code with the Foundry Toolkit, and coding agents through the Foundry Skill and MCP server. Some platform capabilities are still in preview, so check the general availability status before relying on a feature in production.
3. Microsoft Foundry vs Azure AI Foundry vs Azure OpenAI
Microsoft's naming has changed several times, and interviewers notice when candidates use old terms for new things.
| Name | What it means today |
|---|---|
| Microsoft Foundry | The current unified platform for models, agents and tools |
| Azure AI Foundry / Azure AI Studio | Previous names for the platform |
| Foundry (classic) | The older portal, where hub-based projects still live |
| Azure OpenAI | OpenAI models sold by Microsoft on Azure; an Azure OpenAI resource can be upgraded to a Foundry resource |
| Foundry Models | The model catalog (Microsoft, OpenAI, Anthropic, Meta, open models and more) and deployments |
| Foundry Agent Service | Managed runtime for prompt, voice-based prompt and hosted agents |
| Foundry Tools | Prebuilt AI services: Speech, Language, Translator, Vision, Document Intelligence, Content Understanding, Content Safety and others (previously "Azure AI services") |
| Foundry IQ | Knowledge layer for grounding agents in enterprise data, built on Azure AI Search agentic retrieval |
Microsoft's own mapping of old to new is useful to memorise:
| Area | Previous | Current |
|---|---|---|
| Agent API | Assistants API (threads, messages, runs) | Responses API (conversations, items, responses, agent versions) |
| API versioning | Monthly api-version parameters | v1 stable routes (/openai/v1/) |
| Resource model | Hub + Azure OpenAI + Azure AI services | One Foundry resource with projects |
| SDK | Several packages and endpoints | azure-ai-projects 2.x plus the OpenAI client against one project endpoint |
So a tutorial that uses hub-based projects, AzureOpenAI() with monthly API versions or the Assistants API is describing the classic experience. For new work, follow the current Foundry documentation.
4. Microsoft Foundry architecture
Enterprise users
-> Web / mobile app
-> API / backend (FastAPI, Functions, Container Apps)
-> Microsoft Foundry project endpoint
|-- Models (inference through the Responses API)
|-- Agents (prompt, voice-based prompt, hosted)
|-- Tools (toolbox: file search, web search, code interpreter, MCP, OpenAPI, functions)
|-- Knowledge (Azure AI Search / Foundry IQ)
-> Enterprise data (Blob, SharePoint, SQL, business APIs)
-> Grounded response with sources
Cross-cutting:
Identity Microsoft Entra ID, managed identities, agent identities
Security RBAC, private networking, Key Vault, guardrails
Observability Tracing, evaluations, Application Insights, Azure Monitor
Delivery GitHub Actions / Azure DevOps / azd, Terraform or Bicep4.1 Foundry resource
The Azure resource that holds the platform: model deployments, networking, identity and policy settings. It is governed like any Azure resource with RBAC, Azure Policy and network rules.
4.2 Foundry project
A workspace inside the resource where a team builds: agents, conversations, files, evaluations, connections and traces. Each project has its own endpoint, in the form https://<resource-name>.services.ai.azure.com/api/projects/<project-name>.
4.3 Foundry Models
A catalog of more than 10,000 models from Microsoft, OpenAI, Anthropic, Meta and others. What you can deploy depends on region, quota, deployment type and licensing.
4.4 Foundry Agent Service
The managed runtime for agents — conversations, tool calls, versioning, publishing, identity and scaling. Covered in section 9.
4.5 Foundry Tools
Prebuilt services you can call from apps and agents: Speech, Language, Translator, Vision, Document Intelligence, Content Understanding, Content Safety and Azure AI Search.
4.6 Foundry IQ
Knowledge for agents: knowledge bases built on Azure AI Search agentic retrieval that can combine several knowledge sources and plan multi-step queries. Compare it with a plain Azure AI Search index plus your own retrieval code before choosing.
4.7 Observability and evaluation
Traces of every model call and tool call, built-in evaluators, dashboards and Application Insights integration. Without them you can't prove an agent is good enough to ship.
5. Microsoft Foundry setup: step by step
Prerequisites
- An Azure subscription
- Permission to create resources (Owner at resource-group scope to create a new project, or Foundry Project Manager on an existing project for hosted-agent work)
- A region where your chosen model is available
- Python 3.13+ if you will deploy hosted agents (Python 3.10+ works for the SDK samples)
- Azure CLI, plus Azure Developer CLI 1.36.0 or later for hosted agents
Step 1: Open the portal
Go to ai.azure.com and sign in with your work or Microsoft account.
Step 2: Create a Foundry resource and project
- Choose the subscription.
- Choose or create a resource group.
- Name the resource.
- Choose a region.
- Set networking and security options.
- Create the resource and project.
Copy the project endpoint from the project overview — your code uses it.
Step 3: Set permissions
Use Entra ID groups and Azure RBAC with least privilege. Don't give every developer subscription Owner. Separate platform administrators, AI developers, project managers, application identities, and monitoring and security teams.
Step 4: Deploy a model
Open the model catalog and choose by task, quality, context window, latency, modalities, cost, region, quota and deployment type. Deploy it and note the deployment name — that is the model value in your code.
Step 5: Test in the playground
You are an enterprise banking assistant.
Explain the difference between a savings account and a current account in simple language.
Do not invent bank-specific policies.Check the answer for correctness, relevance and safety.
Step 6: Connect monitoring
Connect Application Insights to the project so traces and agent metrics are captured from day one.
Official guide: Set up Microsoft Foundry resources.
6. Foundry Models: model catalog and deployment
| Model category | Typical use |
|---|---|
| Language models | Chat, summarisation, generation |
| Reasoning models | Multi-step reasoning, planning, coding |
| Embedding models | Semantic search and RAG |
| Vision-language models | Image and text understanding |
| Speech / realtime models | Voice applications |
| Open-weight models | Customisation, cost control, specific licensing needs |
For the banking assistant you might need a chat model for questions, an embedding model for retrieval, a vision-capable model for uploaded documents and a realtime model for voice. Benchmark candidates on your own test set rather than choosing the most popular model.
Deployment types control where data is processed and how capacity is billed — for example global, data-zone and regional standard (pay per token), provisioned throughput (reserved capacity) and batch. Not every model supports every type. Check capacity, quota, throughput, latency, region, billing and data-residency requirements before you commit. Our Azure for AI engineers guide explains quotas and 429 handling.
7. Python quickstart: first model call and a prompt agent
These samples follow Microsoft's current SDK quickstart and use azure-ai-projects 2.x, which is not compatible with 1.x code you may find in older blogs.
pip install "azure-ai-projects>=2.3.0" azure-identity
az loginCall a model with the Responses API
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
PROJECT_ENDPOINT = "https://<resource-name>.services.ai.azure.com/api/projects/<project-name>"
project = AIProjectClient(endpoint=PROJECT_ENDPOINT, credential=DefaultAzureCredential())
openai = project.get_openai_client()
response = openai.responses.create(
model="gpt-5-mini", # your deployment name
input="Explain a home loan EMI in two sentences.",
)
print(response.output_text)DefaultAzureCredential uses your az login session locally and a managed identity in Azure, so there are no keys in the code.
Create a prompt agent
from azure.ai.projects.models import PromptAgentDefinition
agent = project.agents.create_version(
agent_name="banking-assistant",
definition=PromptAgentDefinition(
model="gpt-5-mini",
instructions=(
"You are a banking information assistant. Answer only from approved information. "
"Never invent rates or eligibility rules. If you don't know, say so and offer a human agent."
),
),
)
print(agent.name, agent.version)Each create_version call creates a new, versioned definition, so you can compare and roll back.
Chat with the agent across turns
openai = project.get_openai_client(agent_name="banking-assistant")
conversation = openai.conversations.create()
r1 = openai.responses.create(conversation=conversation.id, input="What documents do I need for a home loan?")
print(r1.output_text)
r2 = openai.responses.create(conversation=conversation.id, input="And for a self-employed applicant?")
print(r2.output_text)The conversation keeps the history, so the follow-up understands "and for a self-employed applicant". Source: Microsoft Foundry SDK quickstart.
8. Prompt engineering in Microsoft Foundry
Weak prompt:
Tell me about home loans.No scope, no format and no boundaries.
Strong enterprise prompt:
ROLE: You are an approved banking information assistant.
OBJECTIVE: Answer using only the retrieved banking information.
RULES:
1. Be accurate and concise.
2. Never invent interest rates or eligibility rules.
3. Never reveal customer-specific information.
4. If the retrieved information is insufficient, say so.
5. Cite sources.
6. Never execute transactions.
7. Escalate sensitive or complex requests.
OUTPUT: direct answer, conditions, sources, escalation recommendation if needed.Techniques to know:
- Role prompting — define role and scope.
- Context injection — supply approved retrieved content.
- Few-shot examples — show the expected input and output.
- Structured output — request JSON against a schema.
- Grounding instructions — answer only from evidence; admit gaps.
- Prompt versioning — agent versions in Foundry, or prompts in Git.
- Prompt evaluation — test every change against a fixed dataset.
A strong prompt never replaces identity, authorisation, validation and secure design.
9. Microsoft Foundry Agent Service
An agent is a model plus instructions plus tools that can reason over a request and act. Example: "Can you check the status of my loan application?"
- The agent understands the request.
- It decides the loan-status tool is needed.
- The application has already authenticated the user.
- The tool checks the user owns that application.
- The tool calls the authorised loan-status API.
- The agent interprets the result.
- It replies.
The model never gets direct access to customer records — the application and backend enforce access.
Agent types
| Prompt agent | Voice-based prompt agent | Hosted agent | |
|---|---|---|---|
| How you build | Instructions, model, tools (portal, SDK, REST or azd) | Prompt agent with audio settings, via Voice Live | Your own code: Microsoft Agent Framework, LangGraph, OpenAI Agents SDK, plain Python |
| Runtime code you maintain | None | None for voice orchestration | Your agent logic |
| Compute | Fully managed | Fully managed | Container compute managed by Foundry |
| Identity | Entra identity | Entra identity | Dedicated Entra identity per agent |
| Best for | Fast start and production agents without custom orchestration | Call centres and spoken experiences | Agents that call your own code, custom orchestration, multi-agent systems |
You can also skip the agent resource entirely and call the Responses API from an agent you run elsewhere — Microsoft calls this an ephemeral agent. Source: What is Foundry Agent Service?
10. Deploy your own Python code as a hosted agent
This follows Microsoft's deploy your own code quickstart. The hosting library handles the HTTP server, health checks and protocol; your code does the thinking. The hosting packages are pre-release at the time of writing, so pin and test versions.
Step 1: Prerequisites
azd version # 1.36.0 or later
azd ext install microsoft.foundry
azd auth login
python --version # 3.13 or laterYou need the Foundry Project Manager role on an existing project, or Owner on the resource group to create one.
Step 2: Project layout
banking-agent/
├── main.py
├── requirements.txt
├── app/
│ ├── tools.py
│ └── security.py
└── tests/
└── test_agent.pyStep 3: requirements.txt
azure-ai-agentserver-responses>=1.0.0b7
azure-ai-projects>=2.3.0
azure-identityStep 4: main.py — a hosted agent that calls a model
import asyncio, os
from azure.ai.agentserver.responses import (
CreateResponse, ResponseContext, ResponsesAgentServerHost, TextResponse,
)
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential
project = AIProjectClient(
endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], # set by azd for local runs; configure it for the hosted agent
credential=DefaultAzureCredential(),
)
openai = project.get_openai_client()
SYSTEM = "You are a banking information assistant. Never invent rates. Say when you don't know."
app = ResponsesAgentServerHost()
@app.response_handler
async def handler(request: CreateResponse, context: ResponseContext, _cancel: asyncio.Event):
user_input = await context.get_input_text() or ""
result = openai.responses.create(
model=os.environ.get("MODEL_DEPLOYMENT", "gpt-5-mini"),
instructions=SYSTEM,
input=user_input,
)
return TextResponse(context, request, text=result.output_text)
app.run()Microsoft's quickstart starts with an echo handler; this version replaces the "your agent logic here" block with a real model call. In production, add retrieval, tools, input validation and error handling, and use the async client so one slow call doesn't block others.
Step 5: Initialise, provision, test and deploy
azd ai agent init --protocol responses --deploy-mode code
# then add PYTHONNOUSERSITE: "1" to the agent service's env map in azure.yaml
azd provision # creates resources such as Application Insights
azd ai agent run # runs locally and opens the agent inspector
azd ai agent invoke --local "Hello" # test from a second terminal
azd deploy # deploys to Foundry Agent Service
azd ai agent invoke "Hello" # calls the deployed agentazd deploy prints the agent playground link and the agent endpoint. If the current azd environment created the project, azd down deletes the whole resource group, so be careful in shared subscriptions.
LangGraph or Microsoft Agent Framework? Use their built-in hosting integrations instead of the generic library: langchain-azure-ai[hosting] for LangGraph, LangChain and Deep Agents, and the Agent Framework hosting package for Agent Framework. See our LangGraph interview questions and LangGraph for enterprise AI.
11. Microsoft Foundry and RAG architecture
A bank has 20,000 policy documents. A customer asks: "What are the eligibility requirements for a home loan?" A general model does not know the bank's current policy, so you retrieve the approved documents first. Background: what is RAG.
Document sources -> Blob Storage -> Extraction (Document Intelligence Layout)
-> Chunking -> Embeddings -> Azure AI Search index (text + vectors + metadata + access groups)
User question -> Entra sign-in -> Hybrid query with security filter -> Top chunks
-> Foundry model or agent -> Grounded answer with sourcesWays to connect it in Foundry:
- Your own retrieval code in the backend or hosted agent (full control; see the Python retrieval example in our Azure AI services guide).
- Built-in tools on a prompt agent, such as file search or an Azure AI Search connection.
- Foundry IQ / agentic retrieval for complex multi-part questions across several sources.
Implementation steps:
- Upload documents to approved storage.
- Extract text.
- Clean and normalise.
- Chunk by headings — see RAG chunking strategies.
- Generate embeddings.
- Index vectors and metadata in Azure AI Search.
- Authenticate the user.
- Retrieve only documents the user may see.
- Build a grounded prompt.
- Generate the answer.
- Check citations and quality.
- Log telemetry without sensitive content.
| RAG | Fine-tuning |
|---|---|
| Retrieves external information | Adapts behaviour through training |
| For changing enterprise knowledge | For specialised behaviour or format |
| Gives sources | No citations |
| Update by re-indexing | Update by retraining |
More in RAG vs fine-tuning and hybrid search and reranking.
12. MCP, tools and toolboxes
The Model Context Protocol (MCP) is a standard way to connect AI applications to tools and context. Foundry agents can use built-in tools (web search, file search, code interpreter, memory), custom functions, OpenAPI specs and remote MCP servers — for example the Azure DevOps MCP server, or your own MCP server on Azure Functions.
A toolbox curates a set of tools once and exposes them behind a single managed MCP-compatible endpoint, with central authentication, governance and versioning, so many agents can share the same tools. Tool connections support key-based auth, Microsoft Entra (agent or project managed identity) and OAuth on-behalf-of passthrough.
User request -> Foundry agent -> tool selection -> toolbox / MCP endpoint
-> Search | Tickets | Reports -> tool result (untrusted) -> agent -> answerSecurity rules for MCP:
- Authenticate every connection; prefer managed identity or OBO over shared keys
- Expose only the tools the agent needs
- Validate tool arguments
- Don't expose internal admin APIs
- Log sensitive operations
- Require approval for privileged actions
- Treat tool responses as untrusted input (they can carry prompt injections)
Learn more: what is MCP, build an MCP server in Python and MCP interview questions.
13. Memory and state management
Agents need different kinds of state:
- Conversation history — Foundry conversations hold the turns of a chat.
- Session state — hosted agents get session-level state persistence.
- Task state and checkpoints — for long workflows (LangGraph checkpointers, Agent Framework workflows).
- Long-term memory — user preferences and facts across sessions (Foundry memory is in preview; check availability).
Design memory deliberately. A banking assistant must not store account details in an unrestricted long-term memory. Apply identity-scoped access, data minimisation, retention limits and encryption, and let users see and delete what is remembered.
14. Security in Microsoft Foundry
- Microsoft Entra ID for users and applications; disable key-based auth where possible.
- Managed identities so apps reach Foundry, Azure AI Search and Key Vault without stored credentials. Hosted agents get a dedicated Entra identity automatically.
- Azure RBAC with least privilege — separate resource administration, project management, application execution, data access and monitoring.
- Key Vault for any secrets that remain. Never commit keys or connection strings to Git.
- Network security — private endpoints, disabled public access, VNet integration (prompt agents support private networking; hosted agents support bring-your-own VNet), controlled outbound traffic and private DNS.
- Prompt injection protection — separate instructions from data, treat retrieved content as untrusted, give tools minimal permissions, validate inputs and outputs, red-team, and require approval for sensitive actions. Foundry guardrails include protection against cross-prompt injection attacks (XPIA).
- Content safety — content filters and guardrails on models and agents. Never rely on instructions alone.
15. Evaluation and observability
A chatbot that answers one demo question is not production-ready. Measure it. See also RAG evaluation metrics.
| Metric | Question it answers |
|---|---|
| Answer relevance | Does it address the question? |
| Groundedness | Is it supported by retrieved evidence? |
| Retrieval relevance | Were the right documents retrieved? |
| Tool call accuracy | Did the agent pick the right tool with the right arguments? |
| Task completion | Did it achieve the goal? |
| Safety | Did it stay within safety rules? |
| Latency | How long did it take? |
| Cost | What did each interaction cost? |
Example evaluation set:
| Input | Expected behaviour |
|---|---|
| Explain the home loan policy | Retrieves and summarises the approved policy |
| What is the travel reimbursement limit? | Retrieves the authorised finance policy |
| Show another customer's balance | Refuses — unauthorised |
| What is the office Wi-Fi password? | Follows the secret-disclosure policy |
| Explain a policy that doesn't exist | Says the information is unavailable |
Trace the full path: request → API → identity check → agent → model call → tool calls → retrieval → response. Monitor model latency, tool time, tokens, failures, retrieval quality, safety events and cost trends — with redaction and retention rules so traces don't leak personal data.
16. Production deployment: CI/CD for Foundry agents
Tools: GitHub Actions or Azure DevOps, the Azure Developer CLI, Terraform or Bicep, Azure Container Registry where needed, Azure Monitor and Application Insights.
Commit -> Lint and unit tests -> Security scan -> Build agent package or image
-> Infrastructure validation (terraform plan) -> Deploy to test project (azd deploy)
-> Evaluation gate -> Approval -> Production -> Monitoring and rollbackTerraform can provision resource groups, storage, Key Vault, networking, Azure AI Search, monitoring and supported AI resources (provider versions matter). Use azd or the Foundry SDK for agent definitions and versions, and run both from the same pipeline. See our Terraform for AI infrastructure guide and Azure DevOps interview questions.
Release checklist: infrastructure validated · model available with quota · identity permissions verified · network paths tested · prompts and tools reviewed · evaluation set passed · security scenarios tested · monitoring live · rollback ready · release approved.
17. Real-time enterprise use cases
Use case 1: Banking AI agent
Stack: Foundry, Agent Service, a chat model deployment, Azure AI Search, Azure Functions, Entra ID, Key Vault, Azure Monitor.
"Explain the home loan eligibility requirements." → authenticate → identify intent → retrieve authorised home-loan documents → grounded answer → sources → telemetry. Account-specific questions additionally require verified customer identity and backend authorisation.
Use case 2: Healthcare staff assistant
Approved clinical and admin documents -> processing -> knowledge index
-> Foundry agent -> authorised retrieval -> grounded answer -> healthcare staffControls: data minimisation, access permissions, audit trail, clinical review and applicable privacy rules.
Use case 3: IT support agent
Searches troubleshooting docs, checks service status, creates tickets, suggests fixes and escalates. Privileged actions (password resets, access changes) need approval workflows.
Use case 4: Retail assistant
Combines product knowledge retrieval, inventory and order-status APIs, return-policy documents and support workflows.
Use case 5: HR knowledge assistant
Retrieves only what the employee may see, and directs them to approved HR workflows instead of making changes itself.
Use case 6: Legal contract assistant
Extracts, indexes and retrieves contract sections and summarises clauses with references. Lawyers review important interpretations.
Use case 7: Finance report assistant
Answers questions from approved reports or authorised analytics APIs, checks permissions and never presents generated estimates as audited figures.
18. Microsoft Foundry vs Amazon Bedrock
| Capability | Microsoft | AWS |
|---|---|---|
| AI application platform | Microsoft Foundry | Amazon Bedrock |
| Model access | Foundry Models (incl. Azure OpenAI) | Bedrock foundation models |
| Agents | Foundry Agent Service | Amazon Bedrock AgentCore, Bedrock Agents |
| RAG | Azure AI Search / Foundry IQ | Bedrock Knowledge Bases |
| Identity | Microsoft Entra ID | AWS IAM |
| Secrets | Azure Key Vault | AWS Secrets Manager |
| Monitoring | Azure Monitor / Application Insights | Amazon CloudWatch |
| Infrastructure as code | Terraform / Bicep | Terraform / CloudFormation |
They are not one-to-one equivalents. Compare requirements, model availability, integrations, security and cost. For the AWS side see Amazon Bedrock Knowledge Bases interview questions and AWS for AI engineers.
19. Troubleshooting Microsoft Foundry
| Problem | Likely cause | What to check |
|---|---|---|
| Model deployment fails | Region, quota or deployment type not available | Regional availability and quota |
| Access denied / AuthorizationFailed | Missing RBAC role | Identity and role scope (Foundry Project Manager for hosted agents) |
azd ai agent init fails | Old azd or extension | azd version ≥ 1.36.0; update the Foundry extension |
FOUNDRY_PROJECT_ENDPOINT not set | Running the app directly | Use azd ai agent run or set it in .env |
| Agent stuck in provisioning | First deploy installing dependencies | azd ai agent show; wait 2–3 minutes |
| Agent can't call a tool | Auth or network | Tool identity, permissions, private DNS |
| RAG returns irrelevant results | Chunking or retrieval | Inspect chunks and raw search results |
| High latency | Slow model, retrieval or tool | Trace each stage |
| Wrong answers | Poor grounding or prompt | Inspect context; run evaluations |
| No traces | Telemetry not connected | Application Insights connection and access |
20. Microsoft Foundry interview questions
1. What is Microsoft Foundry and how is it different from Azure OpenAI? Foundry is the platform for models, agents, tools, evaluation and governance; Azure OpenAI is one model family available inside it.
2. What is the difference between a Foundry resource and a project? The resource is the Azure boundary for deployments, networking and policy; a project is a team workspace with its own endpoint for agents, files, evaluations and traces.
3. Prompt agent or hosted agent — how do you choose? Start with a prompt agent; move to a hosted agent when you need custom code, custom orchestration, multi-agent logic or a framework like LangGraph.
4. What replaced the Assistants API? The Responses API, with conversations, items, responses and agent versions instead of threads, messages and runs.
5. What is a toolbox? A curated, versioned set of tools exposed through one managed MCP-compatible endpoint, shared across agents with central auth and governance.
6. How does a hosted agent authenticate to other Azure services? With its dedicated Entra agent identity — grant it RBAC roles on Azure AI Search, Storage or Key Vault, with no keys in code.
7. How do you ground a Foundry agent in company documents? Azure AI Search (own retrieval code or a search tool), file search, or Foundry IQ knowledge bases — always with document-level authorisation.
8. How do you stop an agent from taking an unsafe action? Least-privilege tools, argument validation, authorisation inside the tool, human approval for sensitive actions, guardrails and red-team tests.
9. How do you evaluate an agent before release? A versioned dataset of normal, edge and adversarial cases; built-in evaluators for groundedness, relevance, tool-call accuracy and task completion; an evaluation gate in CI/CD.
**10. A team's old code uses AzureOpenAI() with a monthly api-version and a hub project. What do you tell them?** It's the classic pattern: plan a move to a Foundry project, azd/azure-ai-projects 2.x and the v1 OpenAI() client, migrating and testing agent by agent.
For 130 more, see Microsoft Azure AI services interview questions, the advanced Azure AI interview questions and the FDE interview guide.
21. 10-week learning roadmap and final project
| Week | Topic | Hands-on |
|---|---|---|
| 1 | Azure fundamentals and Foundry | Create a resource and project |
| 2 | Models and inference | Deploy a model; first Responses API call |
| 3 | Prompt engineering | Structured-output assistant |
| 4 | Agent Service | Prompt agent with a multi-turn conversation |
| 5 | Hosted agents | Deploy your own Python agent with azd |
| 6 | RAG and Azure AI Search | Enterprise knowledge assistant |
| 7 | MCP and tools | Connect an MCP server through a toolbox |
| 8 | Identity and security | Managed identity, RBAC, private networking |
| 9 | Evaluation and monitoring | Evaluation dataset and tracing |
| 10 | CI/CD and production | End-to-end pipeline with an evaluation gate |
Final project: enterprise banking AI assistant
Functional requirements: answer policy questions · search policy documents · country-specific knowledge collections · RAG retrieval with sources · intent detection · approved backend tools · user permissions · confidential data protection · escalation of sensitive operations · tracing · quality evaluation.
| Component | Technology |
|---|---|
| AI platform | Microsoft Foundry |
| Agent | Foundry Agent Service (prompt agent, then hosted agent) |
| LLM | Foundry model deployment |
| RAG | Azure AI Search |
| Backend | Python and FastAPI |
| Storage | Azure Blob Storage |
| Extraction | Azure Document Intelligence |
| Identity | Microsoft Entra ID |
| Secrets | Azure Key Vault |
| Monitoring | Azure Monitor and Application Insights |
| Infrastructure | Terraform |
| CI/CD | GitHub Actions or Azure DevOps |
Deliverables: architecture diagram, Terraform, Python app, ingestion pipeline, agent and tools, prompt templates, evaluation dataset, security configuration, Docker or azd deployment, CI/CD pipeline, README and troubleshooting guide. This shows the full lifecycle of an enterprise AI application, not just a chatbot — exactly what interviewers for Azure AI and FDE roles want to hear about. Our AI FDE engineer projects list more ideas.
Want to build it with a mentor? Our Generative and Agentic AI course in Hyderabad and the FDE PRO programme cover Foundry, Azure AI Search, agents, MCP and production deployment with real projects. When you're ready, browse verified AI engineer jobs.
Official Microsoft documentation
- What is Microsoft Foundry?
- Set up Microsoft Foundry resources
- Get started with the Microsoft Foundry SDK
- What is Foundry Agent Service?
- Deploy your own code as a hosted agent
- Foundry product and capability map
📞 Call +91 96660 19191 · WhatsApp · Flat No. 513, 5th Floor, Nilagiri Block, Aditya Enclave, Ameerpet, Hyderabad
Frequently Asked Questions
What is Microsoft Foundry used for?
Building, deploying and governing AI applications and agents with models, tools, knowledge, evaluation, tracing and enterprise identity and networking.
Is Microsoft Foundry the same as Azure OpenAI?
No. Foundry is the broader platform; Azure OpenAI models are one family available in its model catalog.
What is the difference between prompt agents and hosted agents?
Prompt agents are defined by instructions, a model and tools and run fully managed. Hosted agents run your own code and framework in a Foundry-managed container.
Can I use Python with Microsoft Foundry?
Yes. Use azure-ai-projects 2.x with DefaultAzureCredential to call models and create agents, and the Azure Developer CLI to deploy hosted agents.
Can I deploy LangGraph agents to Foundry?
Yes, as hosted agents using the langchain-azure-ai hosting integration.
Does Microsoft Foundry support MCP?
Yes. Agents can use remote MCP servers, and toolboxes expose curated tools through a managed MCP-compatible endpoint.
What replaced the Assistants API in Foundry?
The Responses API, with conversations, items, responses and agent versions instead of threads, messages and runs.
What should I learn before Microsoft Foundry?
Azure fundamentals, Python, prompting, embeddings and RAG, Azure AI Search, Entra ID and RBAC, APIs and basic container deployment.
