Azure AI Vision (now Azure Vision in Foundry Tools)AuthorCloudBrum editorial teamLast reviewed31 Oct 2026Reading time~12 minLevelBeginner to advanced

• CaptionsOne sentence that describes the picture, with a confidence score. A gender-neutral setting avoids guessing someone’s gender. Captions are English only at the time of writing, and captions and dense captions are offered only in some Regions, so check yours first.
• Dense captionsA description for up to ten regions of the same picture, each with its bounding box, on top of the one for the whole image. Good for long alt text and for pictures with many things in them.
• Tags and objectsTags list what is in the image with a confidence. Object detection adds a bounding box for each thing it can place, and people detection does the same for people only.
• Smart cropsGive one or more aspect ratios, such as 1:1 for a thumbnail, and get the rectangle that keeps the most interesting part of the picture.
• Read (OCR)Reads printed and handwritten text in photos and scans, in many languages, and returns blocks, lines and words with polygons. Dense documents are the job of Document Intelligence.
• Multimodal embeddingsTurns an image or a sentence into a vector in the same space, so a text query such as "red bicycle by a wall" can find matching pictures in a vector index.
• Vision Studio and the Foundry portalTry each feature on your own pictures in a browser, with sample code, before you write any. Handy for judging whether the accuracy is good enough.
• Neighbouring servicesFaces are handled by the separate Azure Face service, unsafe-content classification by Azure AI Content Safety, and your own categories by Custom Vision or a model in Foundry.

Resource

The Azure object you create to call the API. Kind ComputerVision is for Vision alone; the multi-service kind covers several AI services under one key.

Endpoint and custom subdomain

The address you call, such as https://name.cognitiveservices.azure.com. A custom subdomain is required for Microsoft Entra ID sign-in and for network rules.

Image Analysis 4.0

The current API, one call to imageanalysis:analyze with the features you list. The older 3.2 API has separate operations.

Visual feature

One thing you ask for, such as caption, tags, read, objects, people or smart crops. Each feature in a request counts as a transaction.

Caption and dense caption

A caption describes the whole picture. Dense captions describe regions of it, each with a bounding box.

Tag

A word for something in the image, with a confidence from 0 to 1.

Read result

Text found in the image, organised in blocks, then lines, then words, each with a bounding polygon and a confidence.

Bounding polygon

The corners of the area where something was found, in pixels.

Transaction

The billing unit. One call asking for caption and tags is two transactions.

SKU (F0 and S1)

The pricing tier. F0 is free, with 5,000 transactions a month and 20 a minute. S1 is paid, bills per 1,000 transactions and allows up to about 20 a second.

Multimodal embedding

A vector that represents an image or a text in the same space, so they can be compared.

Azure Content Understanding

The Foundry service Microsoft names as the replacement for general image analysis. It also handles documents, audio and video.

Document Intelligence

The service for forms, invoices, tables and dense text. Its Read model is the route Microsoft points OCR users to.

Azure Face

A separate service for detecting and analysing faces. Identification and verification are Limited Access features that need an approved application.

Cognitive Services User

The built-in Azure role that lets an identity call the API’s data plane and nothing else.

Create a locked-down Azure AI Vision resource and call it with the Azure CLI

1. Read the retirement notice firstImage Analysis (3.2 and 4.0) is due to retire on 25 September 2028, and the older Computer Vision API versions 1.0 to 3.1 stopped working on 13 September 2026. Decide how long your project needs to live, and keep this API behind one function.
2. Pick a Region that offers your featuresCaptions and dense captions are limited to some Regions. If you need them, create the resource there and check the answer on a sample before you commit.
3. Create the resource with a custom subdomainUse kind ComputerVision and give it a custom domain name. Start on F0 while you experiment. Your very first Vision resource in a subscription may need to be created in the portal so that the responsible-AI terms can be accepted.
4. Switch keys off and close the public endpointSet disableLocalAuth and turn public network access off. After that, only callers with an Entra ID token on your network can reach the resource.
5. Add a private endpoint and its DNS zoneA private endpoint of group account plus the privatelink.cognitiveservices.azure.com zone, linked to the virtual network the app runs in.
6. Give the app an identity and a roleUse a managed identity with the Cognitive Services User role on this one resource.
7. Call it with the features you needAsk for caption, tags or read, not all of them. Check the confidence before you show a caption to anyone.
8. Handle throttlingA 429 answer carries a Retry-After header. Wait that long, add jitter, and cap parallel calls.
9. Test the way outRun twenty of your own pictures through Azure Content Understanding, Document Intelligence Read or the Face API, whichever matches your use, and write down the differences while you still have time.

• Ask for only the features you need, since each one adds work and cost.
• Resize large photos before sending. Most features work fine at about 1,500 pixels on the long side.
• Use the synchronous Analyze call for single images. It returns the answer in the same request.
• Run calls from the same Region as the resource, and use the private endpoint from inside the network.
• Send bytes for pictures the service can’t reach by URL, and URLs for public pictures to save bandwidth.
• Look an image hash up in your own store before calling the API, and save the answer after.
• Cap parallel calls and honour Retry-After, so throttling does not turn into a retry storm.
• Create the embedding for each image once, store it, and search the stored vectors.

FinOps is the habit of managing cloud spend together with engineering. These are the FinOps checks that matter most for Azure AI Vision.

• Use F0 for learning and tests, and expect it to throttle quickly.
• Request one or two features per call instead of everything.
• Cache results and skip duplicates.
• Batch offline work during quiet hours and keep the call rate steady to avoid throttling retries.
• Check whether Read is enough for simple text before moving to a document service that bills per page.
• Watch the transactions metric in Azure Monitor and set a budget alert.
• Delete unused resources. A forgotten S1 resource with no traffic costs nothing per call but can still collect charges through attached features.
• Tag every resource with an owner and an end date, which helps when the retirement review comes.

• Turn local authentication off and call with Microsoft Entra ID tokens.
• Use a managed identity with the Cognitive Services User role on the one resource.
• Disable public network access and use a private endpoint with the private DNS zone.
• Send diagnostic logs to a Log Analytics workspace and alert on spikes in calls or errors.
• Use customer-managed keys where the service and your rules call for them.
• Avoid sending personal images unless you have a purpose, and delete source images on a schedule.
• Put Azure Policy in place to deny Vision resources with public access or local auth enabled.
• Restrict who can list keys and who can change network settings.
• Review the responsible AI notes for any feature that analyses people.
• Keep an inventory of resources with API versions in use, because the retirement affects every one of them.

Azure Activity Log records who created or changed a resource, Azure Policy enforces the settings above, and the Microsoft Service Trust Portal has the compliance reports. Microsoft runs the models. You decide what is sent, how long it is kept and what users are told.

• Skip repeat analysis with a cache.
• Send resized pictures, which means less data to move and process.
• Request only the features you show.
• Delete unused resources and old test images.
• Pick the Azure Region with the cleaner grid where your data rules allow it.
• Microsoft’s Emissions Impact Dashboard reports per subscription and service family, so quote no Vision figure of your own.

Purchase options

What you actually pay for

• Transactions: one per requested feature per call
• Higher rates for some features than others
• Data transfer and private endpoint charges
• Log Analytics ingestion for diagnostics
• Separate meters if you also use Face, Content Safety, Document Intelligence or Content Understanding

A Vision resource that only Microsoft Entra ID callers can reach, over a private endpoint. The CLI tab creates the resource with a custom domain, turns keys and the public endpoint off, adds the private endpoint and DNS zone, assigns the Cognitive Services User role and makes a REST call with a token. Terraform describes the same resources. The Python tab asks for caption, tags and text in one call, with throttling handled by the SDK, and keeps a caption only when its confidence is high. The C# tab makes the same call and audits Vision-type resources in a subscription for keys, public access and missing owner tags. Replace the names, Region and object ID with your own.

# Placeholders: replace the names, and the object ID of the identity your application runs as.
# Image Analysis 4.0 is retiring (see the retirement note on this page), so treat this setup as a bridge and keep the calls behind one function.
RG=vision-demo
NAME=catalog-vision
LOCATION=centralindia
VNET=app-vnet
SUBNET=pe-subnet
APP_OBJECT_ID=00000000-0000-0000-0000-000000000000

az group create --name $RG --location $LOCATION

# 1. The resource. A custom domain name is required for Microsoft Entra sign-in. Use --sku F0 for the free tier while you experiment.
az cognitiveservices account create --resource-group $RG --name $NAME --location $LOCATION \
  --kind ComputerVision --sku S1 --custom-domain $NAME --assign-identity --tags owner=platform-team env=dev --yes

# 2. Switch keys off (callers must use Entra ID) and close the public endpoint. Not every Region offers every feature, so check yours before this step.
ACCOUNT_ID=$(az cognitiveservices account show --resource-group $RG --name $NAME --query id --output tsv)
az resource update --ids $ACCOUNT_ID --api-version 2024-10-01 \
  --set properties.disableLocalAuth=true properties.publicNetworkAccess=Disabled

# 3. A private endpoint with its private DNS zone, so the app reaches the resource on a private address.
az network private-endpoint create --resource-group $RG --name $NAME-pe --vnet-name $VNET --subnet $SUBNET \
  --private-connection-resource-id $ACCOUNT_ID --group-id account --connection-name $NAME-link
az network private-dns zone create --resource-group $RG --name privatelink.cognitiveservices.azure.com
az network private-dns link vnet create --resource-group $RG --zone-name privatelink.cognitiveservices.azure.com \
  --name vision-dns-link --virtual-network $VNET --registration-enabled false
az network private-endpoint dns-zone-group create --resource-group $RG --endpoint-name $NAME-pe --name default \
  --private-dns-zone privatelink.cognitiveservices.azure.com --zone-name cognitiveservices

# 4. Let the application's identity call the API, and nothing more.
az role assignment create --assignee-object-id $APP_OBJECT_ID --assignee-principal-type ServicePrincipal \
  --role "Cognitive Services User" --scope $ACCOUNT_ID

# 5. Call Image Analysis 4.0 with a Microsoft Entra token (from a machine that can reach the private endpoint).
TOKEN=$(az account get-access-token --resource https://cognitiveservices.azure.com --query accessToken --output tsv)
curl -s -X POST "https://$NAME.cognitiveservices.azure.com/computervision/imageanalysis:analyze?api-version=2024-02-01&features=caption,tags,read&gender-neutral-caption=true" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/images/shopfront.jpg"}'

# 6. Send logs to a workspace, and look at the quota and usage of the resource.
az monitor diagnostic-settings create --name vision-logs --resource $ACCOUNT_ID --workspace $LOG_WORKSPACE_ID \
  --logs '[{"categoryGroup":"allLogs","enabled":true}]' --metrics '[{"category":"AllMetrics","enabled":true}]'
az cognitiveservices account list-usage --resource-group $RG --name $NAME --output table

# 7. Clean up. A deleted resource stays recoverable for a while; purge it to free the custom domain name.
az cognitiveservices account delete --resource-group $RG --name $NAME
az cognitiveservices account purge --location $LOCATION --resource-group $RG --name $NAME

# An Azure AI Vision (ComputerVision) resource that only Microsoft Entra ID callers can reach, over a private endpoint.
# Written for azurerm 4.x. Version 5 changed some arguments (for example the DNS zone link takes private_dns_zone_id), so read the upgrade guide before moving.
terraform {
  required_providers {
    azurerm = { source = "hashicorp/azurerm", version = "~> 4.0" }
  }
}

provider "azurerm" {
  features {}
}

variable "app_object_id" {
  description = "Object ID of the managed identity or service principal the application runs as"
  type        = string
}

variable "subnet_id" {
  description = "Subnet that holds the private endpoint"
  type        = string
}

variable "vnet_id" {
  description = "Virtual network that should resolve the private DNS name"
  type        = string
}

resource "azurerm_resource_group" "vision" {
  name     = "vision-demo"
  location = "centralindia"
}

resource "azurerm_cognitive_account" "vision" {
  name                  = "catalog-vision"
  resource_group_name   = azurerm_resource_group.vision.name
  location              = azurerm_resource_group.vision.location
  kind                  = "ComputerVision"
  sku_name              = "S1"            # F0 is the free tier: a monthly transaction allowance and a low call rate
  custom_subdomain_name = "catalog-vision" # needed for Entra ID sign-in and for network rules

  local_auth_enabled            = false   # no keys: callers must present an Entra ID token
  public_network_access_enabled = false

  identity {
    type = "SystemAssigned"
  }

  tags = { owner = "platform-team", env = "dev" }
}

resource "azurerm_private_dns_zone" "vision" {
  name                = "privatelink.cognitiveservices.azure.com"
  resource_group_name = azurerm_resource_group.vision.name
}

resource "azurerm_private_dns_zone_virtual_network_link" "vision" {
  name                  = "vision-dns-link"
  resource_group_name   = azurerm_resource_group.vision.name
  private_dns_zone_name = azurerm_private_dns_zone.vision.name
  virtual_network_id    = var.vnet_id
}

resource "azurerm_private_endpoint" "vision" {
  name                = "catalog-vision-pe"
  resource_group_name = azurerm_resource_group.vision.name
  location            = azurerm_resource_group.vision.location
  subnet_id           = var.subnet_id

  private_service_connection {
    name                           = "catalog-vision-link"
    private_connection_resource_id = azurerm_cognitive_account.vision.id
    subresource_names              = ["account"]
    is_manual_connection           = false
  }

  private_dns_zone_group {
    name                 = "default"
    private_dns_zone_ids = [azurerm_private_dns_zone.vision.id]
  }
}

# The application may call the API (data plane) and nothing else.
resource "azurerm_role_assignment" "app_calls_vision" {
  scope                = azurerm_cognitive_account.vision.id
  role_definition_name = "Cognitive Services User"
  principal_id         = var.app_object_id
}

output "endpoint" {
  value = azurerm_cognitive_account.vision.endpoint
}

"""Alt text and text extraction with Azure AI Vision Image Analysis 4.0 (azure-ai-vision-imageanalysis), signed in with Microsoft Entra ID."""
from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures
from azure.identity import DefaultAzureCredential

ENDPOINT = "https://catalog-vision.cognitiveservices.azure.com/"
MIN_CAPTION_CONFIDENCE = 0.6     # below this the caption is not published as alt text; a person writes it instead


def make_client(endpoint=ENDPOINT, credential=None, **options):
    """DefaultAzureCredential uses the managed identity in Azure and your developer sign-in on a laptop. No key is stored anywhere.
    The SDK retries a throttled (429) call and honours the Retry-After header; retry_total and retry_backoff_factor set how long it keeps trying."""
    options.setdefault("retry_total", 6)
    options.setdefault("retry_backoff_factor", 1.0)
    return ImageAnalysisClient(endpoint=endpoint, credential=credential or DefaultAzureCredential(), **options)


def analyze(client, image_bytes):
    """One call for caption, tags and text. Each visual feature you ask for is a separate billable transaction, so ask only for what you use."""
    return client.analyze(image_data=image_bytes,
                          visual_features=[VisualFeatures.CAPTION, VisualFeatures.TAGS, VisualFeatures.READ],
                          gender_neutral_caption=True, language="en")


def summarise(result):
    """Plain dictionary for the rest of the application: alt text only when the model is sure, tags above 0.8, text line by line."""
    caption = result.caption
    alt = caption.text if caption is not None and caption.confidence >= MIN_CAPTION_CONFIDENCE else None
    tags = [t.name for t in (result.tags.list if result.tags else []) if t.confidence >= 0.8]
    lines = [line.text for block in (result.read.blocks if result.read else []) for line in block.lines]
    return {"alt_text": alt, "tags": tags, "text_lines": lines}


def describe(client, image_bytes):
    return summarise(analyze(client, image_bytes))

// Part 1 reads alt text from a picture with a Microsoft Entra token. Part 2 audits every Vision-type resource in the subscription for keys and public access.
// Needs the Cognitive Services User role on the resource (part 1) and Reader on the subscription (part 2).
using Azure;
using Azure.AI.Vision.ImageAnalysis;
using Azure.Identity;
using Azure.ResourceManager;
using Azure.ResourceManager.CognitiveServices;
using Azure.ResourceManager.CognitiveServices.Models;

var credential = new DefaultAzureCredential();

// ---- part 1: caption, tags and text for one picture, from a URL
var endpoint = new Uri("https://catalog-vision.cognitiveservices.azure.com/");
var vision = new ImageAnalysisClient(endpoint, credential);
ImageAnalysisResult result = await vision.AnalyzeAsync(
    new Uri("https://example.com/images/shopfront.jpg"),
    VisualFeatures.Caption | VisualFeatures.Tags | VisualFeatures.Read,
    new ImageAnalysisOptions { GenderNeutralCaption = true, Language = "en" });

Console.WriteLine($"Alt text: {result.Caption.Text} ({result.Caption.Confidence:P0})");
Console.WriteLine("Tags: " + string.Join(", ", result.Tags.Values.Where(t => t.Confidence >= 0.8).Select(t => t.Name)));
foreach (var block in result.Read.Blocks)
    foreach (var line in block.Lines)
        Console.WriteLine("Text: " + line.Text);

// ---- part 2: audit
var arm = new ArmClient(credential);
var subscription = await arm.GetDefaultSubscriptionAsync();
var kinds = new HashSet<string>(StringComparer.OrdinalIgnoreCase) { "ComputerVision", "CognitiveServices", "AIServices", "Face", "CustomVision.Prediction" };
var findings = 0;

await foreach (CognitiveServicesAccountResource account in subscription.GetCognitiveServicesAccountsAsync())
{
    var d = account.Data;
    if (d.Kind is null || !kinds.Contains(d.Kind)) continue;

    var props = d.Properties;
    var notes = new List<string>();
    if (props.DisableLocalAuth != true) notes.Add("keys still work");
    if (props.PublicNetworkAccess != ServiceAccountPublicNetworkAccess.Disabled) notes.Add("public endpoint open");
    if (d.Tags is null || !d.Tags.ContainsKey("owner")) notes.Add("no owner tag");
    if (d.Sku?.Name == "F0") notes.Add("free tier");

    findings += notes.Count;
    Console.WriteLine($"{d.Name,-28} {d.Kind,-18} {d.Sku?.Name,-4} {d.Location,-14} {(notes.Count == 0 ? "ok" : string.Join("; ", notes))}");
}
Environment.ExitCode = findings == 0 ? 0 : 1;

Videos picked for this service. Each one is labelled with its source, opens on YouTube if you prefer, and loads only when you press play.

Typical callers are Azure Functions triggered by new blobs in Azure Blob Storage, with results saved to Azure Cosmos DB and routed on with Azure Event Grid. Metrics and alerts live in Azure Monitor, and secrets and keys, if you still use any, in Azure Key Vault.