Microsoft Foundry (formerly Azure AI Foundry)AuthorCloudBrum editorial teamLast reviewed30 Oct 2026Reading time~12 minLevelBeginner to advanced

• A catalogue with two kinds of modelModels sold by Azure (the OpenAI family among them) come with Microsoft’s own support terms. Partner and community models, such as Llama or Mistral, and Claude from Anthropic, are hosted by Microsoft but priced and licensed by their providers.
• A resource, then projectsOne Foundry resource holds the model deployments, the keys or identities and the network settings. Projects sit inside it to group agents, files and evaluations, and each has its own endpoint for the SDK.
• Choose how a deployment is billedGlobal Standard, Data Zone Standard and regional Standard are pay per token. Provisioned deployments reserve capacity in PTUs and bill by the hour. Batch is cheaper and slower, and partner models can be called through a serverless API.
• Familiar APIsThe Responses API and chat completions work with the OpenAI client libraries. The model name in a call is your deployment name, and the Foundry SDK hands you an authenticated client for a project.
• Foundry Agent ServiceBuild agents from a prompt, a workflow or your own container. Creating an agent costs nothing extra; you pay for the model tokens, the tools it uses and any hosted-agent compute.
• Foundry IQ for your knowledgeConnect an agent to documents, SharePoint, Fabric or Azure AI Search so that it answers from your content. Billing follows the services behind it.
• Content filters on by defaultEvery deployment runs behind content filtering. Prompt Shields look for jailbreak and injection attempts in the user prompt by default, and the check for attacks hidden in documents is a setting you turn on.
• Keyless access and private networksMicrosoft Entra ID roles replace API keys, and private endpoints take the traffic off the public internet. Microsoft recommends keyless access for production.

Foundry resource

The Azure resource (kind AIServices) that holds deployments, identity, networking and the project list.

Project

A child of the resource that groups agents, files and evaluations and has its own endpoint. Older hub-based projects sit on an Azure Machine Learning hub instead.

Model deployment

A named, running copy of a model with a deployment type and a capacity. Calls use its name in the model field.

Global Standard

Pay-per-token deployment that may process a prompt in any Azure Region. It is the usual starting point, gets new models first and has the lowest price.

Data Zone Standard

Pay-per-token deployment that keeps processing inside a zone that Microsoft defines, such as the US, the EU or Asia Pacific.

Provisioned throughput (PTU)

Reserved capacity in provisioned throughput units, billed by the hour whatever you send. It gives steadier delay for steady traffic.

Batch deployment

Asynchronous processing with a 24-hour target at a lower price, for jobs that nobody waits on.

Tokens per minute (TPM)

The quota unit for a deployment. Requests per minute follow from it, at about six per thousand TPM.

Responses API

The newer call that takes input and instructions and returns output items. Calling it from your code makes an agent whose definition lives in your application.

Content filter

A set of classifiers for harmful content in prompts and answers, with severity thresholds you can set per category.

Prompt Shields

A check for jailbreak attempts in user prompts and for instructions hidden in documents.

Model router

A deployment that picks a model for each prompt, trading cost against quality.

Foundry IQ

The knowledge layer for agents, with connections to Azure AI Search, SharePoint, Fabric and other sources.

Cognitive Services User

The role that lets an identity call models and other AI services with a Microsoft Entra ID token.

Create a Foundry resource, deploy a model and call it without keys, with the Azure CLI

1. Pick a Region and check what it offersCentral India (Pune) and South India (Chennai) are the Foundry Regions in India. Not every model or deployment type is offered in every Region, so look at the Region’s model list and your quota before you promise anything.
2. Create the Foundry resourceMake an AIServices resource with a custom domain, a managed identity and project management switched on. The custom domain is required for Entra ID sign-in, and its name becomes part of the endpoint.
3. Add a projectCreate a project inside the resource. It gives you the project endpoint that the SDK uses.
4. Deploy a modelChoose the model, a version and a deployment type. Start with Global Standard unless a rule about data location says otherwise, and start the capacity low.
5. Turn off keys and grant a roleDisable local authentication, then give people and apps the Cognitive Services User role at the resource scope. A new role can take a few minutes to apply.
6. Call the model with a tokenSend a request with an Entra ID token from the CLI or the SDK, using the deployment name as the model. Read the status, the usage numbers and the content filter result.
7. Check quota and set alertsLook at tokens-per-minute use for the Region, and alert on 429 responses and token use. Ask for more quota well before launch.
8. Send logs somewhere you controlTurn on diagnostic settings to a Log Analytics workspace, and decide who may read them and for how long they are kept.
9. Add agents and knowledge lastBring in the Agent Service, tools and Foundry IQ once a plain call works. Every one of them carries its own cost, its own permissions and its own ways of failing.

• Choose the smallest model that passes your tests, and use the model router where several sizes would do.
• Stream the answer, so people see text while the rest is generated.
• Keep prompts short and put the stable part first, so prompt caching can reuse it where the model supports it.
• Set max_output_tokens to the length you need. It also keeps the quota estimate honest.
• Use provisioned throughput when you need steady delay for steady traffic, and Global Standard for the rest.
• Send offline work to Batch.
• Add a second deployment in another Region and fail over, so one busy backend doesn’t stall the app.
• Retrieve fewer, better passages for Foundry IQ and test the result on real questions.

FinOps is the habit of managing cloud spend together with engineering. These are the FinOps checks that matter most for Microsoft Foundry.

• Start on Global Standard and move to Data Zone or Provisioned only for a rule or a measured need.
• Pick a smaller model first and judge it by cost per correct answer on your own questions.
• Use Batch for work that can wait, at a lower price.
• Cut history and retrieved text where you can. Each turn sends it again and bills it again.
• Cap output length with max_output_tokens.
• Size PTUs from measured use, consider a reservation for steady load, and delete idle deployments.
• Remember the tools and services an agent uses (search, code execution, vector store, hosted-agent compute), which have their own meters.
• Watch partner model prices, which their providers set and which can differ from Azure OpenAI rates.
• Tag the resource, group Cost Management by meter, and set a budget alert.
• Switch off or remove test deployments the same day.

• Use Microsoft Entra ID and the Cognitive Services User role, and disable local authentication.
• Run apps on managed identities, and keep any remaining secrets in Key Vault.
• Turn off public network access and add private endpoints with the three private DNS zones.
• Keep content filters and Prompt Shields on, and turn on the document attack check for agents that read files.
• Give every agent and tool a narrow identity of its own, and check what a tool receives before acting on it.
• Choose Data Zone or regional deployments when location rules apply.
• Send audit and request logs to a protected workspace with a retention period.
• Use Azure Policy to limit Regions, deployment types and public access.
• Read the license and data terms of partner and community models before you approve one.
• Alert on new deployments, role assignments and filter changes in the activity log.

Azure Activity Log shows who changed a resource, Azure Monitor holds metrics and logs, Microsoft Defender for Cloud scores configuration, and the Service Trust Portal has compliance reports. Microsoft runs the hosting of its models. Which models you allow, what goes into prompts, who holds which identity, how filters are set and how answers are tested stay with you.

• A smaller model needs less compute for every request, so right-sizing is the first and easiest saving.
• Reuse the stable start of a prompt through caching where the model supports it.
• Group non-urgent requests into Batch jobs, not single calls.
• Delete idle PTU deployments and test resources.
• If rules on data location and response time permit, a Region on a cleaner grid is the greener pick.
• Microsoft’s sustainability tools report at subscription level. We found no verifiable per-request figure for hosted models, so none is given.

Purchase options

What you actually pay for

• Input and output tokens for every call, on each deployment
• Provisioned throughput units for every hour they exist
• Agent tools, search and any container compute for hosted agents
• Azure AI Search, storage, Log Analytics and private endpoint charges around the model
• Partner model fees set by the provider

A first Foundry setup: a resource with a custom domain, a project, a model deployment, keys switched off, a role for the caller, a token-based call, a quota check and logs. The Azure CLI tab builds it step by step, and Terraform describes the resource, the deployment, the role and the diagnostic setting. The Python tab calls the deployment through the project client, waits out 429 responses with the delay the service sends, streams an answer and tells a cut-off answer from a finished one. The C# tab audits every Foundry and Azure OpenAI account for keys, public access and deployments. Swap in your own names, model and Region before you run anything.

# Placeholders: choose your own names. The resource name becomes part of the endpoint, so it must be unique in Azure.
# Pick a model and version the Region offers (step 3 lists them) and a deployment type your quota allows.
RG=rg-foundry-demo
LOCATION=centralindia
NAME=foundry-demo-$RANDOM
PROJECT=chat-app
MODEL_NAME=gpt-4o-mini
MODEL_VERSION=2024-07-18
DEPLOYMENT=chat

# 1. A resource group, then the Foundry resource (kind AIServices) with a custom domain, a managed identity and project management switched on.
az group create --name $RG --location $LOCATION
az cognitiveservices account create --name $NAME --resource-group $RG --location $LOCATION --kind AIServices --sku S0 \
  --custom-domain $NAME --allow-project-management --assign-identity --yes

# 2. A project inside it. Projects group agents, files and evaluations, and give you the project endpoint for the SDK.
az cognitiveservices account project create --name $NAME --resource-group $RG --project-name $PROJECT --location $LOCATION

# 3. Which models can this resource deploy here?
az cognitiveservices account list-models --name $NAME --resource-group $RG \
  --query "[].{model:name,version:version,format:format}" --output table

# 4. Deploy one. GlobalStandard is the usual starting point: pay per token, no reserved capacity. The capacity number is thousands of tokens per minute.
az cognitiveservices account deployment create --name $NAME --resource-group $RG --deployment-name $DEPLOYMENT \
  --model-format OpenAI --model-name $MODEL_NAME --model-version $MODEL_VERSION --sku-name GlobalStandard --sku-capacity 50

# 5. Keyless only: turn off API keys, and give your user (or app identity) the role that lets it call models with Microsoft Entra ID.
ACCOUNT_ID=$(az cognitiveservices account show --name $NAME --resource-group $RG --query id --output tsv)
az resource update --ids $ACCOUNT_ID --set properties.disableLocalAuth=true
az role assignment create --assignee "$(az ad signed-in-user show --query id --output tsv)" --role "Cognitive Services User" --scope $ACCOUNT_ID

# 6. Call the deployment with a token instead of a key. "model" is the deployment name. Role changes can take a few minutes to apply.
TOKEN=$(az account get-access-token --resource https://cognitiveservices.azure.com --query accessToken --output tsv)
curl -s "https://$NAME.openai.azure.com/openai/v1/responses" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d "{\"model\":\"$DEPLOYMENT\",\"input\":\"What does a content filter do in Microsoft Foundry?\",\"max_output_tokens\":200}"

# 7. How much of the quota is in use in this Region?
az cognitiveservices usage list --location $LOCATION --query "[?contains(name.value, 'GlobalStandard')].{quota:name.value,used:currentValue,limit:limit}" --output table

# 8. Send the audit and request logs to a Log Analytics workspace, so you can see who called what and which calls were throttled.
WORKSPACE_ID=$(az monitor log-analytics workspace create --resource-group $RG --workspace-name law-foundry-demo --query id --output tsv)
az monitor diagnostic-settings create --name to-logs --resource $ACCOUNT_ID --workspace $WORKSPACE_ID \
  --logs '[{"category":"Audit","enabled":true},{"category":"RequestResponse","enabled":true}]' \
  --metrics '[{"category":"AllMetrics","enabled":true}]' --output none

# Placeholders: choose your own names and a model that your Region and quota allow. This builds the Foundry resource,
# one model deployment, keyless access for an app identity, and logs. Projects are not shown; add them with the CLI step
# or with the project resource of the azurerm version you use.
variable "location" {
  type    = string
  default = "centralindia"
}

variable "name" {
  type        = string
  description = "Resource name; it also becomes the endpoint name, so it must be unique in Azure."
}

variable "app_principal_id" {
  type        = string
  description = "Object ID of the user, group or managed identity that calls the models."
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "foundry" {
  name     = "rg-foundry-demo"
  location = var.location
}

resource "azurerm_cognitive_account" "foundry" {
  name                  = var.name
  location              = azurerm_resource_group.foundry.location
  resource_group_name   = azurerm_resource_group.foundry.name
  kind                  = "AIServices"
  sku_name              = "S0"
  custom_subdomain_name = var.name

  # Keyless only: callers use Microsoft Entra ID.
  local_auth_enabled = false

  identity {
    type = "SystemAssigned"
  }
}

resource "azurerm_cognitive_deployment" "chat" {
  name                 = "chat"
  cognitive_account_id = azurerm_cognitive_account.foundry.id

  model {
    format  = "OpenAI"
    name    = "gpt-4o-mini"
    version = "2024-07-18"
  }

  sku {
    name     = "GlobalStandard"
    capacity = 50
  }
}

# The role that lets a caller use the models with an Entra ID token.
resource "azurerm_role_assignment" "caller" {
  scope                = azurerm_cognitive_account.foundry.id
  role_definition_name = "Cognitive Services User"
  principal_id         = var.app_principal_id
}

resource "azurerm_log_analytics_workspace" "logs" {
  name                = "law-foundry-demo"
  location            = azurerm_resource_group.foundry.location
  resource_group_name = azurerm_resource_group.foundry.name
  sku                 = "PerGB2018"
  retention_in_days   = 30
}

resource "azurerm_monitor_diagnostic_setting" "foundry" {
  name                       = "to-logs"
  target_resource_id         = azurerm_cognitive_account.foundry.id
  log_analytics_workspace_id = azurerm_log_analytics_workspace.logs.id

  enabled_log {
    category = "Audit"
  }

  enabled_log {
    category = "RequestResponse"
  }

  metric {
    category = "AllMetrics"
  }
}

output "endpoint" {
  value = azurerm_cognitive_account.foundry.endpoint
}

# pip install azure-ai-projects azure-identity
# Calls a Foundry model deployment with Microsoft Entra ID (no API key), waits out 429 responses the way the service asks,
# streams an answer, and tells a cut-off or filtered answer from a finished one.
import os
import time

import openai
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

ENDPOINT = os.environ["FOUNDRY_PROJECT_ENDPOINT"]       # https://<resource>.services.ai.azure.com/api/projects/<project>
DEPLOYMENT = os.environ.get("FOUNDRY_DEPLOYMENT", "chat")   # the deployment name you chose, not the model name

project = AIProjectClient(endpoint=ENDPOINT, credential=DefaultAzureCredential())
client = project.get_openai_client(max_retries=0)       # retries are handled below so that retry-after-ms is respected


def ask(question, instructions="Answer in at most three sentences.", max_output_tokens=300, attempts=6):
    """Returns (text, status, reason, usage). Quota is estimated from max_output_tokens, so keep it close to the answer size you need."""
    delay = 1.0
    for attempt in range(attempts):
        try:
            reply = client.responses.create(model=DEPLOYMENT, instructions=instructions, input=question, max_output_tokens=max_output_tokens)
            reason = reply.incomplete_details.reason if reply.incomplete_details else None
            return reply.output_text, reply.status, reason, reply.usage
        except (openai.RateLimitError, openai.InternalServerError, openai.APIConnectionError) as error:
            if attempt == attempts - 1:
                raise
            headers = getattr(getattr(error, "response", None), "headers", {}) or {}
            wait = float(headers["retry-after-ms"]) / 1000 if "retry-after-ms" in headers else delay
            time.sleep(wait)
            delay = min(delay * 2, 30)


def ask_streaming(question, max_output_tokens=300):
    """Prints the answer as it arrives and returns the full text."""
    pieces = []
    stream = client.responses.create(model=DEPLOYMENT, input=question, max_output_tokens=max_output_tokens, stream=True)
    for event in stream:
        if event.type == "response.output_text.delta":
            print(event.delta, end="", flush=True)
            pieces.append(event.delta)
    print()
    return "".join(pieces)


if __name__ == "__main__":
    text, status, reason, usage = ask("What does a content filter do in Microsoft Foundry?")
    print(text)
    print("status:", status, "| incomplete because:", reason, "| tokens in/out:", usage.input_tokens, usage.output_tokens)
    if reason == "max_output_tokens":
        print("The answer was cut off. Raise max_output_tokens only if you need the rest.")
    elif reason == "content_filter":
        print("A content filter stopped this answer.")

// dotnet add package Azure.Identity
// dotnet add package Azure.ResourceManager.CognitiveServices
// Audits every Foundry (AIServices) and Azure OpenAI account in the subscription: are API keys still allowed, is the public
// endpoint open, and which model deployments exist, with their type and capacity.
using Azure.Identity;
using Azure.ResourceManager;
using Azure.ResourceManager.CognitiveServices;
using Azure.ResourceManager.CognitiveServices.Models;

var arm = new ArmClient(new DefaultAzureCredential());
var subscription = await arm.GetDefaultSubscriptionAsync();
int problems = 0;

await foreach (CognitiveServicesAccountResource account in subscription.GetCognitiveServicesAccountsAsync())
{
    string kind = account.Data.Kind ?? "";
    if (kind != "AIServices" && kind != "OpenAI") continue;

    Console.WriteLine($"{account.Data.Name} ({kind}, {account.Data.Location})");

    if (account.Data.Properties.DisableLocalAuth != true)
    {
        Console.WriteLine("  API keys are still allowed. Use Microsoft Entra ID and turn local authentication off.");
        problems++;
    }
    if (account.Data.Properties.PublicNetworkAccess != ServiceAccountPublicNetworkAccess.Disabled)
    {
        Console.WriteLine("  The public endpoint is open. Use a private endpoint where the data needs it.");
        problems++;
    }

    int deployments = 0;
    await foreach (CognitiveServicesAccountDeploymentResource deployment in account.GetCognitiveServicesAccountDeployments().GetAllAsync())
    {
        deployments++;
        var model = deployment.Data.Properties.Model;
        string sku = $"{deployment.Data.Sku?.Name} x {deployment.Data.Sku?.Capacity}";
        Console.WriteLine($"  deployment {deployment.Data.Name}: {model?.Name} {model?.Version} | {sku}");
        if (deployment.Data.Sku?.Name?.Contains("Provisioned") == true)
            Console.WriteLine("    Provisioned capacity bills every hour, used or not. Check that it is busy enough.");
    }
    if (deployments == 0)
        Console.WriteLine("  No deployments. An empty account costs nothing, but check whether it is still needed.");
}

Console.WriteLine(problems == 0 ? "No problems found." : $"{problems} problem(s) found.");
return problems == 0 ? 0 : 1;

Videos picked for this service. Each one is labelled with its source, opens on YouTube if you prefer, and loads only when you press play.

Typical companions are Azure Blob Storage for documents, Azure Functions for glue code, Azure Monitor for logs and alerts and Azure Key Vault for secrets. To train or host a model of your own, look at Azure Machine Learning.