Azure Machine LearningAuthorCloudBrum editorial teamLast reviewed28 Oct 2026Reading time~12 minLevelBeginner to advanced

• A workspace for everythingOne resource holds your experiments, jobs, models, endpoints and settings, next to a storage account, a key vault, a container registry and Application Insights. Teams usually get one per project or environment.
• Three kinds of computeA compute instance is your personal dev machine. A compute cluster is a pool that grows with the queue and shrinks to nothing. Serverless compute lets a job ask for machines without you creating a cluster first.
• Jobs you can repeatA command job runs a script in an environment on some compute. Sweep jobs tune hyperparameters, pipeline jobs chain steps, and every run is tracked with its code, inputs, metrics and outputs.
• Automated MLGive it a table, a target column and a time limit, and it tries algorithms and settings, then ranks the models. Handy for a first baseline.
• Managed endpointsA managed online endpoint hosts a model on VMs that Azure patches and scales. Several deployments can sit behind one endpoint with a traffic split, which is how blue-green rollouts work. Batch endpoints score files in bulk.
• MLflow built inThe workspace speaks MLflow for tracking and the model registry, so code written for open source MLflow works with little change, and MLflow models deploy without a scoring script.
• Registries for sharingA registry lets several workspaces share models, environments and components, so a model trained in dev can be promoted to prod without copying files by hand.
• Responsible AI toolsDashboards for model explanations, error analysis, fairness and counterfactuals help you look at a model before people rely on it.

Workspace

The top-level resource for a project. It stores run history and assets, and it points at its storage account, key vault, container registry and Application Insights.

Compute instance

A single cloud workstation for one person, with Jupyter, VS Code and a terminal. It bills while it’s running.

Compute cluster

A set of identical VMs that scales between a minimum and a maximum node count. With a minimum of zero it costs nothing when idle.

Serverless compute

On-demand compute chosen per job. You state the VM size and count in the job, and Azure ML creates and removes the machines.

Job

One tracked run of code: a command, a sweep, a pipeline or an AutoML task. Its inputs, outputs, logs and metrics are saved.

Environment

The software a job or deployment runs in: a base image plus a conda or pip file. Curated environments come ready-made.

Datastore and data asset

A datastore is a saved connection to storage such as Blob or Data Lake. A data asset is a named, versioned pointer to files or a table.

Model

A registered, versioned artifact. MLflow-format models carry their own signature and dependencies.

Managed online endpoint

A stable HTTPS URL for real-time scoring. Behind it sit one or more deployments, each with its own VM size, count and model.

Deployment

One version of a model running on a set of VMs inside an endpoint, commonly named blue and green.

Batch endpoint

An endpoint that starts a job to score a folder or table asynchronously, on a cluster, and writes the results to storage.

Component

A reusable pipeline step with declared inputs, outputs and an environment, which can be shared through a registry.

Registry

A shared catalogue, separate from any workspace, that holds models, components and environments for many workspaces to use.

MLflow

An open source tracking and model format. Azure ML workspaces work as MLflow tracking servers.

Create a workspace, a cluster and your first endpoint in the Azure portal

1. Check the Region and quotaPick a Region close to your data (Central India for many Indian teams). In Quotas, look at the VM family you want. GPU families often need a request, and low-priority quota is separate from dedicated quota.
2. Create the workspaceSearch for Azure Machine Learning, choose Create, then a resource group, a name and a Region. Let it create the storage account, key vault, Application Insights and container registry, or point it at existing ones. You can’t swap them later.
3. Open the studioLaunch the studio from the workspace page. It’s a separate web app where you’ll see jobs, models, endpoints and compute.
4. Create a compute clusterUnder Compute, make a cluster with a small CPU size, minimum nodes 0, maximum 2 and a short idle time before scale down (the default is 120 seconds). The minimum of zero is what keeps the bill at nothing between runs.
5. Add your dataUpload a CSV to the workspace’s default blob store, then register it as a data asset so jobs refer to it by name and version.
6. Run a jobCreate a command job: your script, an environment, the data asset as an input, the cluster as compute. Watch the logs and metrics as it runs. Have the script log the model with MLflow.
7. Register the modelFrom the finished job, choose Register model and keep the MLflow format. The model gets a name and a version number.
8. Deploy to an online endpointCreate a managed online endpoint, add a deployment with a VM size and instance count, and send all traffic to it. Because the model is MLflow, no scoring script or environment is needed.
9. Test, watch and set limitsUse the Test tab with a sample row, open the endpoint’s metrics, set up autoscale, and add a budget alert. Delete the endpoint when you’re done experimenting.

• Match the VM size to the job. Try a CPU size for tree models, and use GPU sizes only when the framework really uses them.
• Keep data in the same Region as compute, and use a mounted or direct-read input mode for large files rather than copying them into every job.
• Set the cluster’s minimum nodes above zero only when start-up delay hurts, for example during working hours, because zero is the cheap setting.
• Use distributed training (PyTorch distributed, MPI) with multi-node clusters for large models, with a fast interconnect on the VM sizes that offer one.
• Use sweep jobs with early termination so poor hyperparameter trials stop early.
• Scale endpoints with autoscale rules on CPU, memory or request count, and test the rules under load before launch.
• Pick batch endpoints for large scoring jobs: they split work across cluster nodes with settings for mini-batch size and parallelism.
• Reduce model size with quantisation or ONNX export to cut latency and VM size.

FinOps is the habit of managing cloud spend together with engineering. These are the FinOps checks that matter most for Azure Machine Learning.

• Set cluster minimum nodes to 0, and keep idle time before scale down short (the default is 120 seconds) for experimental work.
• Use low-priority or spot VMs for training that can restart, and write checkpoints so interruptions are cheap.
• Turn on idle shutdown and schedules for compute instances, and stop them at the end of the day.
• Prefer serverless compute for occasional jobs, so no cluster has to exist.
• Delete test online endpoints the same day. Endpoint VMs bill all the time, and an endpoint needs at least one running.
• Use batch endpoints for scheduled scoring rather than an always-on endpoint.
• Look at Azure Reservations or savings plans for steady baseline VM use once the pattern is clear.
• Right-size by watching CPU, GPU and memory metrics, and move to a smaller VM when utilisation is low.
• Use tags on workspaces, compute and endpoints (managed online endpoints need tags to show up properly in cost views), and set budgets.
• Remember the neighbours: storage, container registry, Key Vault, Application Insights and network egress all bill separately.

• Use managed identities for the workspace, compute and endpoints, and avoid account keys and connection strings in code.
• Give people the least role they need: the built-in AzureML Data Scientist and Compute Operator roles cover most work without full contributor rights.
• Put the workspace behind private endpoints or the managed virtual network, and turn public network access off where you can.
• Use customer-managed keys where policy requires them, and turn on soft delete and purge protection for the key vault.
• Authenticate endpoint callers with Microsoft Entra tokens where possible, and keep endpoint keys in Key Vault.
• Restrict outbound traffic from training and deployment with the managed network’s outbound rules.
• Build custom environments in a pipeline from frozen package versions, and scan what lands in the container registry.
• Use Azure Policy to enforce private networking, approved regions, tags and customer-managed keys.
• Send diagnostic logs and the activity log to Log Analytics and review who created compute and endpoints.
• Run the Responsible AI dashboard before release and keep its output with the registered model.

Use the Activity log and Azure Monitor to see who created or changed compute and endpoints, Azure Policy for the rules above, Microsoft Defender for Cloud for posture checks, and Microsoft Purview for data catalogue and lineage. The Service Trust Portal has compliance reports. Responsibility is split: Microsoft runs the platform, and you are accountable for the data, what the model does, who has access and how changes are approved.

• Train less often. Retrain when the data has moved, not on a calendar habit.
• Begin with a small model or AutoML baseline. If a simple model is good enough, a large one only adds energy use.
• Let clusters scale to zero and stop compute instances, so no VM sits idle.
• Use spot or low-priority nodes, which use capacity that would otherwise sit unused.
• Choose a Region with a cleaner grid for training that no one is waiting on, within your data rules.
• Shrink a model (quantise it or distil it) before you deploy, so each prediction runs on a smaller VM.
• The Microsoft Emissions Impact Dashboard gives account-level estimates. See what it covers for Azure ML before you quote a number from it.

Purchase options

What you actually pay for

• Virtual machine time for compute instances, clusters, serverless jobs and endpoint deployments, by size and hour
• Storage for data, run outputs, models and logs
• Container Registry for the images your environments and deployments build
• Application Insights, Key Vault and network traffic (egress, private endpoints)
• Optional extras: reservations, GPU sizes, managed network egress, and hosted model usage billed separately

This example trains a churn model from a CSV, registers it and publishes it on a managed online endpoint. The Azure CLI and Python tabs do each step (the model is logged with MLflow, so the deployment needs no scoring script). The Terraform tab builds the workspace and a cluster that scales to zero, and the C# tab calls the finished endpoint. Names, the subscription and the data path are examples. Remember to delete the endpoint afterwards, since billing continues while it exists.

# Needs the ml extension:  az extension add -n ml
# Names are examples; the resource group and workspace must exist (see the Terraform tab)
az configure --defaults group=rg-ml workspace=mlw-churn

# A cluster that costs nothing while idle
az ml compute create --name cpu-cluster --type amlcompute --size Standard_DS3_v2 \
  --min-instances 0 --max-instances 2 --idle-time-before-scale-down 120

# conda.yml lists python, pip, mlflow, scikit-learn and pandas; job.yml runs src/train.py on the cluster,
# and the script logs the model with MLflow
cat > job.yml <<'EOF'
$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json
display_name: churn-train
experiment_name: churn
code: ./src
command: python train.py --data ${{inputs.churn}}
inputs:
  churn:
    type: uri_file
    path: azureml://datastores/workspaceblobstore/paths/churn/train.csv
environment:
  image: mcr.microsoft.com/azureml/openmpi4.1.2-ubuntu20.04:latest
  conda_file: ./conda.yml
compute: azureml:cpu-cluster
EOF

JOB=$(az ml job create -f job.yml --query name -o tsv)
az ml job stream --name $JOB

# Register the MLflow model the job produced
az ml model create --name churn-model --type mlflow_model \
  --path azureml://jobs/$JOB/outputs/artifacts/paths/model/

# An endpoint with one deployment ("blue"); an MLflow model needs no scoring script
cat > endpoint.yml <<'EOF'
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
name: churn-ep
auth_mode: key
EOF

cat > blue.yml <<'EOF'
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: blue
endpoint_name: churn-ep
model: azureml:churn-model@latest
instance_type: Standard_DS3_v2
instance_count: 1
EOF

az ml online-endpoint create -f endpoint.yml
az ml online-deployment create -f blue.yml --all-traffic

# Score one customer
echo '{"input_data": {"columns": ["tenure","support_calls","monthly_charge","contract"], "data": [[34, 2, 79.5, 1]]}}' > sample.json
az ml online-endpoint invoke --name churn-ep --request-file sample.json

# Stop the meter
az ml online-endpoint delete --name churn-ep --yes --no-wait

# Workspace and a cluster that scales to zero. Assumes azurerm provider version 3 or later.
# Managed online endpoints are created with the CLI or SDK tabs (or the azapi provider).
provider "azurerm" {
  features {}
}

variable "location" { default = "centralindia" }

data "azurerm_client_config" "current" {}

resource "azurerm_resource_group" "ml" {
  name     = "rg-ml"
  location = var.location
}

resource "azurerm_storage_account" "ml" {
  name                     = "stmlchurn001"        # 3-24 lowercase letters and digits, unique in Azure
  location                 = azurerm_resource_group.ml.location
  resource_group_name      = azurerm_resource_group.ml.name
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

resource "azurerm_key_vault" "ml" {
  name                = "kv-mlchurn-001"
  location            = azurerm_resource_group.ml.location
  resource_group_name = azurerm_resource_group.ml.name
  tenant_id           = data.azurerm_client_config.current.tenant_id
  sku_name            = "standard"
  purge_protection_enabled = true
}

resource "azurerm_application_insights" "ml" {
  name                = "appi-mlchurn"
  location            = azurerm_resource_group.ml.location
  resource_group_name = azurerm_resource_group.ml.name
  application_type    = "web"
}

resource "azurerm_machine_learning_workspace" "ml" {
  name                    = "mlw-churn"
  location                = azurerm_resource_group.ml.location
  resource_group_name     = azurerm_resource_group.ml.name
  application_insights_id = azurerm_application_insights.ml.id
  key_vault_id            = azurerm_key_vault.ml.id
  storage_account_id      = azurerm_storage_account.ml.id

  identity {
    type = "SystemAssigned"
  }
}

resource "azurerm_machine_learning_compute_cluster" "cpu" {
  name                          = "cpu-cluster"
  location                      = azurerm_resource_group.ml.location
  machine_learning_workspace_id = azurerm_machine_learning_workspace.ml.id
  vm_priority                   = "Dedicated"        # "LowPriority" for interruptible, cheaper nodes
  vm_size                       = "Standard_DS3_v2"

  scale_settings {
    min_node_count                       = 0
    max_node_count                       = 2
    scale_down_nodes_after_idle_duration = "PT2M"    # two minutes, ISO 8601 duration
  }

  identity {
    type = "SystemAssigned"
  }
}

# pip install azure-ai-ml azure-identity
# conda.yml (next to this file) lists python, pip, mlflow, scikit-learn and pandas for the training environment
# ---- run.py: submit the job, register the model, deploy and test ----
from azure.ai.ml import MLClient, Input, command
from azure.ai.ml.constants import AssetTypes
from azure.ai.ml.entities import (AmlCompute, Environment, ManagedOnlineDeployment,
                                  ManagedOnlineEndpoint, Model)
from azure.identity import DefaultAzureCredential

ml = MLClient(DefaultAzureCredential(), "<subscription-id>", "rg-ml", "mlw-churn")

# A cluster that scales to zero when idle
ml.compute.begin_create_or_update(AmlCompute(
    name="cpu-cluster", size="Standard_DS3_v2",
    min_instances=0, max_instances=2, idle_time_before_scale_down=120,
)).result()

job = command(
    code="./src",
    command="python train.py --data ${{inputs.churn}}",
    inputs={"churn": Input(type=AssetTypes.URI_FILE,
                           path="azureml://datastores/workspaceblobstore/paths/churn/train.csv")},
    environment=Environment(image="mcr.microsoft.com/azureml/openmpi4.1.2-ubuntu20.04:latest",
                            conda_file="./conda.yml"),
    compute="cpu-cluster",
    display_name="churn-train",
    experiment_name="churn",
)
run = ml.jobs.create_or_update(job)
ml.jobs.stream(run.name)

# Register the MLflow model the job logged
model = ml.models.create_or_update(Model(
    name="churn-model", type=AssetTypes.MLFLOW_MODEL,
    path=f"azureml://jobs/{run.name}/outputs/artifacts/paths/model/",
))

# One endpoint, one deployment called "blue", all traffic to it
endpoint = ManagedOnlineEndpoint(name="churn-ep", auth_mode="key")
ml.online_endpoints.begin_create_or_update(endpoint).result()
ml.online_deployments.begin_create_or_update(ManagedOnlineDeployment(
    name="blue", endpoint_name="churn-ep", model=model,
    instance_type="Standard_DS3_v2", instance_count=1,
)).result()
endpoint.traffic = {"blue": 100}
ml.online_endpoints.begin_create_or_update(endpoint).result()

# sample.json: {"input_data": {"columns": [...], "data": [[34, 2, 79.5, 1]]}}
print(ml.online_endpoints.invoke(endpoint_name="churn-ep", request_file="sample.json"))

# Delete the endpoint when you've finished testing: it bills while it exists
# ml.online_endpoints.begin_delete(name="churn-ep").result()


# ---- src/train.py: fit a model and log it in MLflow format ----
# import argparse, mlflow, pandas as pd
# from sklearn.ensemble import GradientBoostingClassifier
# from sklearn.model_selection import train_test_split
#
# p = argparse.ArgumentParser(); p.add_argument("--data"); args = p.parse_args()
# df = pd.read_csv(args.data)
# X, y = df.drop(columns=["churned"]), df["churned"]
# X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
# mlflow.sklearn.autolog()
# with mlflow.start_run():
#     model = GradientBoostingClassifier().fit(X_train, y_train)
#     mlflow.log_metric("accuracy", model.score(X_test, y_test))
#     mlflow.sklearn.log_model(model, artifact_path="model")

// Program.cs: call a managed online endpoint (the endpoint itself was created with the CLI or Python tab)
// Set AML_SCORING_URI and AML_KEY from the endpoint's details page, or with:
//   az ml online-endpoint show --name churn-ep --query scoring_uri
//   az ml online-endpoint get-credentials --name churn-ep
using System.Net.Http.Headers;
using System.Text;

var scoringUri = Environment.GetEnvironmentVariable("AML_SCORING_URI")
    ?? throw new InvalidOperationException("Set AML_SCORING_URI");
var key = Environment.GetEnvironmentVariable("AML_KEY")
    ?? throw new InvalidOperationException("Set AML_KEY");

using var http = new HttpClient { Timeout = TimeSpan.FromSeconds(30) };
http.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", key);

// The columns the model was trained on, in the MLflow "input_data" shape
const string body = """
{ "input_data": { "columns": ["tenure","support_calls","monthly_charge","contract"], "data": [[34, 2, 79.5, 1]] } }
""";

// Retry a few times: an endpoint that is updating or scaling can answer 5xx for a moment
for (var attempt = 1; attempt <= 3; attempt++)
{
    using var reply = await http.PostAsync(scoringUri, new StringContent(body, Encoding.UTF8, "application/json"));
    var text = await reply.Content.ReadAsStringAsync();
    if (reply.IsSuccessStatusCode)
    {
        Console.WriteLine($"Prediction: {text}");
        break;
    }
    Console.WriteLine($"Attempt {attempt}: {(int)reply.StatusCode} {text}");
    if (attempt == 3) throw new HttpRequestException($"Scoring failed with {(int)reply.StatusCode}");
    await Task.Delay(TimeSpan.FromSeconds(2 * attempt));
}

Short walkthrough videos for this service are being prepared. They will appear here and load only when you press play. Until then, follow CloudBrum on YouTube for new videos.

Azure ML is for building and operating your own models. To use a hosted large language model you would go to Microsoft Foundry or Azure OpenAI, and for ready-made vision or speech you would use the Azure AI services. Training data often sits in Azure Blob Storage, and a small model can be served from Azure Functions.