Amazon SageMakerAuthorCloudBrum editorial teamLast reviewed28 Oct 2026Reading time~12 minLevelBeginner to advanced

• Notebooks and a studioSageMaker Studio gives you JupyterLab, a code editor and a place to see jobs, experiments and endpoints in one window. The older notebook instances still work, but they bill while they run and have no idle shutdown of their own.
• Training on servers you don’t manageYou point a job at data in S3, pick an instance type and a container, and SageMaker starts the machines, runs the code, saves the model to S3 and shuts everything down. A single job can run for up to 28 days.
• Built-in algorithms and prebuilt containersXGBoost, linear learners and others come ready to use, and there are maintained containers for PyTorch, TensorFlow, Hugging Face and scikit-learn. You can also bring any container that follows the training and serving contract.
• Automatic model tuningGive it a range for each hyperparameter and a metric to improve, and it runs many training jobs and learns which settings work best. It costs a bunch of jobs, so set a limit.
• Four ways to serve a modelReal-time endpoints for steady traffic, serverless endpoints for gaps in traffic, asynchronous endpoints for big payloads and long runs, and batch transform for scoring a whole file at once.
• Pipelines and a model registryChain data prep, training, evaluation and approval into a repeatable pipeline. The registry keeps model versions with their metrics and an approval status that a deployment can wait for.
• JumpStartA catalogue of pretrained models and sample solutions, including open foundation models. You can deploy one in a few clicks or fine-tune it on your own data first.
• Watching models in productionModel Monitor compares live inputs and predictions with a baseline to catch drift, and Clarify reports bias and which features drove a prediction.

Training job

One run of your training code on managed instances. It reads from S3, writes a model artifact back to S3 and stops billing when it ends.

Model artifact

The output of training, usually a model.tar.gz file in S3. SageMaker doesn’t keep the model anywhere else.

Model (SageMaker object)

A record that links a container image to a model artifact and an execution role. Creating it costs nothing, and an endpoint needs one.

Endpoint configuration

The recipe for an endpoint: which models, which instance type, how many instances and what share of traffic goes where.

Endpoint

The running, billed HTTPS service created from an endpoint configuration. You call it with the runtime API and a signed request.

Execution role

The IAM role SageMaker assumes to read your data and pull your container. Its permissions decide what your training code can touch.

Instance type

A name such as ml.m5.xlarge or ml.g5.xlarge. The ml. prefix marks SageMaker instances, and GPU types usually start with a quota of zero.

Managed spot training

Training on spare capacity at a discount. AWS can take the machine back, so the job has to write checkpoints and resume.

Inference component

A way to put several models on one endpoint and give each its own share of CPU, memory and accelerators, with its own scaling.

Multi-model endpoint

One fleet of instances that loads many small models from S3 on demand, so you don’t pay for an endpoint per model.

Pipeline

A directed graph of steps (process, train, evaluate, register) that you define in code and run on demand or on a schedule.

Model Registry

A catalogue of model versions in model groups, each with metrics, lineage and an approval status.

Feature Store

A managed store for features, with an online side for low-latency lookups and an offline side in S3 for training.

HyperPod

A cluster of accelerated instances for very large training runs, kept healthy and resumable by SageMaker.

Train and serve your first model from the AWS console

1. Pick the Region and check quotasChoose a Region close to your data and users (Mumbai or Hyderabad for India). Open Service Quotas and look at the instance types you plan to use. GPU types often start at zero and a request can take a day or two.
2. Create a bucket and an execution rolePut your training data in an S3 bucket in the same Region. Create an IAM role that SageMaker can assume, and give it access to that bucket only. Don’t attach the broad full-access policy except in a sandbox.
3. Open StudioCreate a domain (the quick setup is fine for a first look), then open JupyterLab. Set an idle timeout on the space so it shuts itself down. A notebook left running over a weekend is the classic surprise bill.
4. Prepare the dataClean the file in a notebook and write a training set and a validation set back to S3. Keep the column order the algorithm expects, and keep the label in the first column for the built-in XGBoost.
5. Run a training jobUnder Training jobs choose a built-in algorithm or your own container, point the input channel at S3, and pick a small instance such as ml.m5.xlarge. Turn on managed spot training and set a checkpoint location.
6. Check the resultWhen the job completes, read its metrics in the console or in CloudWatch. The model artifact lands in the output path you gave. If the job failed, the failure reason and the CloudWatch log stream tell you which.
7. Register the modelAdd the model to a Model Registry group with its metrics, and set the status to pending approval. Even for a first project this habit makes later rollbacks and audits easy.
8. Deploy to an endpointCreate a real-time endpoint on one small instance, or a serverless endpoint if traffic will be occasional. Wait for InService, then send a test request.
9. Add scaling, alarms and a stop ruleAttach target tracking on invocations per instance, set a CloudWatch alarm on 5xx errors and latency, and delete test endpoints the same day. Set a budget alert for SageMaker in Billing.

• Pick the instance type from measurements. Inference Recommender can load-test a model across candidates, and a cheaper CPU instance is often enough for tree models.
• Keep training data in the same Region and use Pipe mode or FastFile mode for large datasets so training starts before the whole download finishes.
• Use warm pools when you run many short jobs back to back, so the instance and its container survive between runs.
• Spread large training runs across instances with the distributed training libraries, and use HyperPod when a run is big and long enough that hardware failures are a given.
• Cut cold start on serverless endpoints with Provisioned Concurrency, or keep a real-time endpoint warm when latency matters.
• Batch requests where the model allows it. One call with 32 rows is usually far cheaper than 32 calls.
• Compile or quantise big models, and try Inferentia (ml.inf2) or Trainium instances where your framework supports them.
• Put a multi-model endpoint or inference components in front of many small models instead of one endpoint each.

FinOps is the habit of managing cloud spend together with engineering. These are the FinOps checks that matter most for Amazon SageMaker.

• Use managed spot training with checkpoints for jobs that can restart. AWS quotes savings of up to 90% against on-demand.
• Delete test endpoints and stop idle apps. This is the single biggest saving for most teams.
• Use serverless or asynchronous (scale to zero) endpoints for traffic that comes in bursts, and batch transform for jobs you can run on a schedule.
• Host many small models on a multi-model endpoint or with inference components rather than one instance per model.
• Right-size instances. Check CPU, memory and GPU use in CloudWatch, and move to a smaller type when the graphs are flat.
• Add target-tracking auto scaling to every real-time endpoint, and set the minimum no higher than you need overnight.
• Look at SageMaker AI Savings Plans for steady usage once you know your baseline.
• Cap automatic tuning with a maximum number of jobs and early stopping, so a sweep can’t run away.
• Keep data compact (Parquet, compressed CSV) and delete old training outputs, checkpoints and model versions you won’t use again.
• Tag every job, endpoint and domain with team and project, and give each a budget alert.

• Give each project its own execution role, scoped to the buckets, keys and ECR repositories it needs, and limit who can pass it with iam:PassRole.
• Run Studio, training jobs and endpoints inside a VPC, with private subnets, VPC endpoints for S3, ECR and CloudWatch, and no direct internet where you can avoid it.
• Encrypt with KMS keys you control: S3 data, training volumes, model artifacts and endpoint storage volumes.
• Turn on network isolation for training and inference containers that don’t need to call out.
• Use inter-container traffic encryption for distributed training that handles sensitive data.
• Allow sagemaker:InvokeEndpoint on named endpoints only, and put an API layer in front of any model that outside callers use.
• Block root access and public notebook links where possible, and apply a service control policy that denies creating resources outside a VPC.
• Log with CloudTrail, keep CloudWatch logs for jobs and endpoints, and review who started which job.
• Scan container images in ECR, build them in a pipeline and pin package versions.
• Use Clarify and Model Cards to document how a model was built and tested, and keep that record with the registry entry.

Use AWS CloudTrail to see who started jobs and changed endpoints, AWS Config for configuration history and rules, SageMaker Model Cards and the Model Registry to record what a model is and who approved it, and AWS Artifact for compliance reports. Know who owns what: AWS secures the platform, while you answer for the data, how the model behaves, who can reach it and how releases are reviewed.

• Train less often. Retrain on a schedule that matches how fast the data really changes, not on a habit.
• Start small. A fine-tuned small model or a gradient-boosted tree often does the job of a huge network at a fraction of the energy.
• Use spot capacity and warm pools so fewer machines sit idle between runs.
• Switch off notebooks, Studio apps and test endpoints when you stop work, because an idle accelerator still draws power.
• Pick a Region with a cleaner grid for training jobs that nobody is waiting on, as long as data rules allow it.
• Use efficient hardware (Inferentia, Trainium or Graviton-based instances) and quantised models where they fit.
• The AWS Customer Carbon Footprint Tool gives account-level estimates. Look at what it counts for SageMaker before you quote a number from it.

Purchase options

What you actually pay for

• Instance time for notebooks, Studio apps, processing jobs, training jobs and endpoints, by instance type
• Storage for notebooks and volumes, plus S3 for data and model artifacts
• Serverless inference: compute time and data processed, plus any provisioned concurrency
• Data transfer out and NAT gateway charges if traffic leaves the VPC
• Optional features: Feature Store, Data Wrangler, Canvas sessions, HyperPod clusters and monitoring jobs

The example predicts which customers will leave: train a small classifier on a CSV in S3, serve it from a real-time endpoint and send it one row. The CLI, Python and C# tabs do the work step by step; the Terraform tab builds the serving side (model, endpoint and auto scaling). The built-in XGBoost image has a different address in each Region, so look yours up first. Tear the endpoint down at the end, because billing carries on until you do.

# Placeholders: your account ID, bucket and role. Mumbai is the example Region.
REGION=ap-south-1
BUCKET=acme-ml-123456789012
ROLE=arn:aws:iam::123456789012:role/SageMakerExecutionRole
# Built-in XGBoost image for your Region: see "Docker registry paths" in the SageMaker docs
IMAGE=123456789012.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-xgboost:1.7-1

# 1. Train on spot capacity; checkpoints let an interrupted job pick up where it stopped
aws sagemaker create-training-job --region $REGION \
  --training-job-name churn-001 \
  --role-arn $ROLE \
  --algorithm-specification TrainingImage=$IMAGE,TrainingInputMode=File \
  --hyper-parameters objective=binary:logistic,num_round=100 \
  --input-data-config '[{"ChannelName":"train","ContentType":"text/csv","DataSource":{"S3DataSource":{"S3DataType":"S3Prefix","S3Uri":"s3://'$BUCKET'/churn/train/","S3DataDistributionType":"FullyReplicated"}}}]' \
  --output-data-config S3OutputPath=s3://$BUCKET/churn/output/ \
  --resource-config InstanceType=ml.m5.xlarge,InstanceCount=1,VolumeSizeInGB=20 \
  --stopping-condition MaxRuntimeInSeconds=3600,MaxWaitTimeInSeconds=7200 \
  --enable-managed-spot-training \
  --checkpoint-config S3Uri=s3://$BUCKET/churn/checkpoints/

aws sagemaker wait training-job-completed-or-stopped --region $REGION --training-job-name churn-001

# 2. Turn the artifact into a model, then an endpoint configuration, then the endpoint
aws sagemaker create-model --region $REGION --model-name churn-001 --execution-role-arn $ROLE \
  --primary-container Image=$IMAGE,ModelDataUrl=s3://$BUCKET/churn/output/churn-001/output/model.tar.gz

aws sagemaker create-endpoint-config --region $REGION --endpoint-config-name churn-001 \
  --production-variants VariantName=AllTraffic,ModelName=churn-001,InstanceType=ml.m5.large,InitialInstanceCount=1

aws sagemaker create-endpoint --region $REGION --endpoint-name churn --endpoint-config-name churn-001
aws sagemaker wait endpoint-in-service --region $REGION --endpoint-name churn

# 3. Score one customer (tenure, support calls, monthly charge, contract type)
aws sagemaker-runtime invoke-endpoint --region $REGION --endpoint-name churn \
  --content-type text/csv --cli-binary-format raw-in-base64-out --body "34,2,79.5,1" prediction.json
cat prediction.json

# 4. Stop the meter
aws sagemaker delete-endpoint --region $REGION --endpoint-name churn

# Serving side only: assumes the model.tar.gz already exists in S3 and AWS provider version 4 or later
variable "bucket" { type = string }
variable "image"  { type = string }   # the XGBoost (or your own) image URI for this Region

data "aws_iam_policy_document" "trust" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      type        = "Service"
      identifiers = ["sagemaker.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "sagemaker" {
  name               = "churn-sagemaker-exec"
  assume_role_policy = data.aws_iam_policy_document.trust.json
}

# Read the model artifact only, not the whole account
data "aws_iam_policy_document" "model_read" {
  statement {
    actions   = ["s3:GetObject"]
    resources = ["arn:aws:s3:::${var.bucket}/churn/output/*"]
  }
}

resource "aws_iam_role_policy" "model_read" {
  role   = aws_iam_role.sagemaker.id
  policy = data.aws_iam_policy_document.model_read.json
}

resource "aws_sagemaker_model" "churn" {
  name               = "churn-001"
  execution_role_arn = aws_iam_role.sagemaker.arn

  primary_container {
    image          = var.image
    model_data_url = "s3://${var.bucket}/churn/output/churn-001/output/model.tar.gz"
  }
}

resource "aws_sagemaker_endpoint_configuration" "churn" {
  name = "churn-001"

  production_variants {
    variant_name           = "AllTraffic"
    model_name             = aws_sagemaker_model.churn.name
    initial_instance_count = 1
    instance_type          = "ml.m5.large"
  }
}

resource "aws_sagemaker_endpoint" "churn" {
  name                 = "churn"
  endpoint_config_name = aws_sagemaker_endpoint_configuration.churn.name
}

# One to four instances, aiming for about 100 requests a minute on each
resource "aws_appautoscaling_target" "churn" {
  service_namespace  = "sagemaker"
  resource_id        = "endpoint/${aws_sagemaker_endpoint.churn.name}/variant/AllTraffic"
  scalable_dimension = "sagemaker:variant:DesiredInstanceCount"
  min_capacity       = 1
  max_capacity       = 4
}

resource "aws_appautoscaling_policy" "churn" {
  name               = "invocations-per-instance"
  policy_type        = "TargetTrackingScaling"
  service_namespace  = aws_appautoscaling_target.churn.service_namespace
  resource_id        = aws_appautoscaling_target.churn.resource_id
  scalable_dimension = aws_appautoscaling_target.churn.scalable_dimension

  target_tracking_scaling_policy_configuration {
    target_value = 100
    predefined_metric_specification {
      predefined_metric_type = "SageMakerVariantInvocationsPerInstance"
    }
  }
}

# pip install boto3
import boto3

REGION = "ap-south-1"
BUCKET = "acme-ml-123456789012"
ROLE = "arn:aws:iam::123456789012:role/SageMakerExecutionRole"
IMAGE = "123456789012.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-xgboost:1.7-1"  # look up the one for your Region

sm = boto3.client("sagemaker", region_name=REGION)
runtime = boto3.client("sagemaker-runtime", region_name=REGION)


def train(job_name):
    """Start a spot training job on a CSV in S3 and wait for it to finish. Returns the model artifact's S3 URL."""
    sm.create_training_job(
        TrainingJobName=job_name,
        RoleArn=ROLE,
        AlgorithmSpecification={"TrainingImage": IMAGE, "TrainingInputMode": "File"},
        HyperParameters={"objective": "binary:logistic", "num_round": "100"},
        InputDataConfig=[{
            "ChannelName": "train",
            "ContentType": "text/csv",
            "DataSource": {"S3DataSource": {
                "S3DataType": "S3Prefix",
                "S3Uri": f"s3://{BUCKET}/churn/train/",
                "S3DataDistributionType": "FullyReplicated",
            }},
        }],
        OutputDataConfig={"S3OutputPath": f"s3://{BUCKET}/churn/output/"},
        ResourceConfig={"InstanceType": "ml.m5.xlarge", "InstanceCount": 1, "VolumeSizeInGB": 20},
        StoppingCondition={"MaxRuntimeInSeconds": 3600, "MaxWaitTimeInSeconds": 7200},
        EnableManagedSpotTraining=True,
        CheckpointConfig={"S3Uri": f"s3://{BUCKET}/churn/checkpoints/"},
    )
    sm.get_waiter("training_job_completed_or_stopped").wait(TrainingJobName=job_name)
    job = sm.describe_training_job(TrainingJobName=job_name)
    if job["TrainingJobStatus"] != "Completed":
        raise RuntimeError(job.get("FailureReason", job["TrainingJobStatus"]))
    return job["ModelArtifacts"]["S3ModelArtifacts"]


def deploy(name, model_url):
    """Model, endpoint configuration and endpoint, in that order. Returns when the endpoint is InService."""
    sm.create_model(ModelName=name, ExecutionRoleArn=ROLE,
                    PrimaryContainer={"Image": IMAGE, "ModelDataUrl": model_url})
    sm.create_endpoint_config(EndpointConfigName=name, ProductionVariants=[{
        "VariantName": "AllTraffic", "ModelName": name,
        "InstanceType": "ml.m5.large", "InitialInstanceCount": 1,
    }])
    sm.create_endpoint(EndpointName=name, EndpointConfigName=name)
    sm.get_waiter("endpoint_in_service").wait(EndpointName=name)


def score(endpoint, row):
    """Send one CSV row and return the model's answer as text."""
    reply = runtime.invoke_endpoint(EndpointName=endpoint, ContentType="text/csv", Body=row)
    return reply["Body"].read().decode("utf-8")


def tear_down(name):
    """Delete the endpoint (this is what stops the billing), then its configuration and model."""
    sm.delete_endpoint(EndpointName=name)
    sm.delete_endpoint_config(EndpointConfigName=name)
    sm.delete_model(ModelName=name)


if __name__ == "__main__":
    url = train("churn-001")
    deploy("churn", url)
    try:
        print(score("churn", "34,2,79.5,1"))
    finally:
        tear_down("churn")

// Program.cs: create the endpoint for an existing model, wait for it, score one row, then delete it
// NuGet packages: AWSSDK.SageMaker, AWSSDK.SageMakerRuntime
using System.Text;
using Amazon;
using Amazon.SageMaker;
using Amazon.SageMaker.Model;
using Amazon.SageMakerRuntime;
using Amazon.SageMakerRuntime.Model;

const string name = "churn";

using var sm = new AmazonSageMakerClient(RegionEndpoint.APSouth1);
using var runtime = new AmazonSageMakerRuntimeClient(RegionEndpoint.APSouth1);

// The model "churn-001" must already exist (created by the training steps or by Terraform)
await sm.CreateEndpointConfigAsync(new CreateEndpointConfigRequest
{
    EndpointConfigName = name,
    ProductionVariants = new List<ProductionVariant>
    {
        new ProductionVariant
        {
            VariantName = "AllTraffic",
            ModelName = "churn-001",
            InstanceType = ProductionVariantInstanceType.MlM5Large,
            InitialInstanceCount = 1
        }
    }
});
await sm.CreateEndpointAsync(new CreateEndpointRequest { EndpointName = name, EndpointConfigName = name });

// Poll until InService (the SDK has no built-in waiter here)
while (true)
{
    var status = (await sm.DescribeEndpointAsync(new DescribeEndpointRequest { EndpointName = name })).EndpointStatus;
    Console.WriteLine($"Endpoint status: {status}");
    if (status == EndpointStatus.InService) break;
    if (status == EndpointStatus.Failed) throw new InvalidOperationException("Endpoint creation failed; check its CloudWatch logs.");
    await Task.Delay(TimeSpan.FromSeconds(30));
}

try
{
    var reply = await runtime.InvokeEndpointAsync(new InvokeEndpointRequest
    {
        EndpointName = name,
        ContentType = "text/csv",
        Body = new MemoryStream(Encoding.UTF8.GetBytes("34,2,79.5,1"))
    });
    using var reader = new StreamReader(reply.Body);
    Console.WriteLine($"Prediction: {await reader.ReadToEndAsync()}");
}
finally
{
    // Deleting the endpoint is what stops the billing
    await sm.DeleteEndpointAsync(new DeleteEndpointRequest { EndpointName = name });
}

Short walkthrough videos for this service are being prepared. They will appear here and load only when you press play. Until then, follow CloudBrum on YouTube for new videos.

Not every model needs this much machinery. If you only want to call a hosted language model, Amazon Bedrock does that with no instances to manage, and ready-made vision or text APIs are quicker still. Small models can live inside AWS Lambda, and the training data usually starts life in Amazon S3.