Amazon RekognitionAuthorCloudBrum editorial teamLast reviewed31 Oct 2026Reading time~12 minLevelBeginner to advanced

• LabelsDetectLabels names the objects, scenes and activities in a picture, groups them into categories and can add the dominant colours and a rough quality reading (brightness, sharpness). It is the call behind most photo search and auto-tagging.
• ModerationDetectModerationLabels scores a picture against a two-level list of unsafe topics. A child label always comes with its parent, so you can write one rule for the parent. Custom moderation adapters let you correct its mistakes on your own content.
• Text in the sceneDetectText reads signs, packaging, number plates and captions in a photo and returns words and lines with positions. It is made for scene text. For a scanned form or invoice, Amazon Textract is the better tool.
• Face detection and comparisonDetectFaces returns where each face is, its landmarks, pose and quality, and attributes such as an open mouth or glasses. CompareFaces measures how alike a face in one picture is to faces in another.
• Face collectionsIndexFaces stores a mathematical vector for each face, not the photo. SearchFacesByImage then finds the closest stored faces. Several vectors of one person can be grouped under a user to make matching steadier.
• Face LivenessA short session in which the user’s phone or browser records a face while a coloured light plays over it. The service answers with a live-or-not score and a few audit images. It targets photo, screen and mask spoofing at sign-up.
• Stored videoStart a job on a video in S3 for labels, faces, moderation, text, people tracking or shot detection. The job runs in the background, tells an SNS topic when it ends, and you read the results page by page.
• Custom LabelsTrain a model on your own labelled pictures to find things Rekognition has no name for, such as your own parts or defects. You pay for training time and for every hour a trained model is running.

Label

A name Rekognition gives to something it sees, with a confidence between 0 and 100. Labels have parents, so "Sedan" sits under "Car" under "Vehicle".

Confidence

How sure the model is that its answer is right. It is not the share of the picture covered or a probability you can calibrate for your own data.

Moderation label

A name from the unsafe-content taxonomy. A top-level category such as Violence has child labels, and every child returns its parent in ParentName.

Bounding box

A rectangle given as fractions of the image width and height, so it works at any resolution. Multiply by the pixel size to draw it.

Face collection

A server-side store of face vectors in one Region. It holds no pictures, and one collection should serve one purpose.

Face vector

The numbers Rekognition derives from a face. Each one gets a FaceId, and the ExternalImageId you pass with the picture is attached to every face found in it, so index one person per picture.

User vector

A person in a collection, made by associating several face vectors with a user id. A search by user compares against all of that person’s faces.

Similarity

The score from CompareFaces and SearchFaces. It says how close two faces are. It does not say that they belong to the same person.

Face model version

The version of the matching model a collection was built with. Vectors from different versions can’t be compared, so a new model means re-indexing from the pictures.

Quality filter

A setting on IndexFaces and SearchFacesByImage that drops faces that are blurred, dark, tiny or turned away. AUTO is the usual choice.

Face Liveness session

A one-time session you create on the server, run in the user’s app or browser, then read results from by session id. It returns a score and up to four audit images.

Audit image

A frame saved during liveness, kept so a person can look at a doubtful case later.

Job

An asynchronous video analysis. You start it, get a JobId, wait for the SNS message and then page through results with the matching Get call.

Custom Labels project

A container for the datasets and trained model versions of one custom detection task.

Inference unit

The capacity a Custom Labels model runs on while it is started. It bills for every hour it runs, used or not.

Screen an upload and search a face collection with the AWS CLI

1. Write down the decision first"Block unsafe pictures", "tag photos for search" and "let staff in" are very different jobs. Say what a wrong answer costs, because that decides thresholds and whether a person reviews the result.
2. Pick the Region and keep the pictures in itRekognition reads from S3 in the same Region as the call. The pages here assume Mumbai (ap-south-1); check that each feature you need is offered there, Face Liveness included.
3. Create a private bucketBlock public access, turn on encryption, and give uploads a lifecycle rule. The application’s role reads from it; nobody else needs to.
4. Try the calls by handRun detect-labels, detect-moderation-labels and detect-text on twenty real pictures from your own users, not stock photos. Note where the scores are wrong.
5. Choose thresholds from that samplePick a confidence for reject, a lower one for human review, and keep the rest. Write the numbers down next to the date you tested.
6. Add faces only with consentCreate a collection per purpose, set ExternalImageId to your own person id, use the quality filter, and write a way to delete a person’s vectors on request.
7. Put the calls behind one small moduleModeration first, then the other calls only for pictures that pass. One place to change thresholds, retries and logging.
8. Run videos as jobsStart the job, publish to an SNS topic (its name must start with AmazonRekognition to work with the AWS managed role), and read all pages of the result when the message arrives.
9. Watch the bill and the errorsAlarm on throttling and on spend. Re-run your twenty-picture sample every few months, because models change.

• Send less pixels. A 12-megapixel phone photo takes longer to upload than it takes to analyse, and labels rarely need more than about 1,500 pixels on the long side.
• Run moderation first and the other calls only for pictures that pass. A rejected picture then costs one call.
• Ask for the features you use. DetectLabels with image properties adds work you may not need.
• Use asynchronous jobs for video and read results by page with MaxResults, not by waiting on a long request.
• Process uploads from an S3 event through a queue, so a spike waits in the queue instead of hitting a throttle.
• Use adaptive retries and a cap on parallel calls. Hammering a throttled API makes it slower.
• Remember results by a hash of the picture; an identical upload then costs nothing.
• Keep collections small and purposeful. A search that only needs staff should not look through visitors.

FinOps is the habit of managing cloud spend together with engineering. These are the FinOps checks that matter most for Amazon Rekognition.

• Call moderation first and stop there for rejected pictures.
• Resize and deduplicate pictures before calling. Identical uploads cost the same each time.
• Sample video frames or use a job with a sensible minimum confidence, instead of analysing every second at maximum detail.
• Stop Custom Labels models when nobody is calling them. A started model bills by the hour.
• Delete unused face vectors. Stored face metadata bills every month.
• Use the free tier for testing only, and set a budget alert so a loop in your code can’t run up a bill overnight.
• Choose the cheapest API that answers the question. Labels, moderation and text are priced in different groups.
• Tag the account or project and watch spend by API in Cost Explorer.

• Give the application role only the calls it uses, and limit face calls to one collection by its ARN.
• Keep the bucket private and encrypted, and add a lifecycle rule so pictures of people don’t stay for ever.
• Use a VPC endpoint for Rekognition and S3 where the workload runs in private subnets.
• Decide on the AI services opt-out policy in AWS Organizations before sending customer content.
• Log calls with CloudTrail and alert on DeleteCollection, DeleteFaces and changes to collections.
• Store no secret in the picture metadata and no personal data in ExternalImageId; use an opaque id.
• Protect Face Liveness sessions: create them on the server, bind them to the signed-in user, and use each only once.
• Delete face vectors when consent is withdrawn or the person leaves, and record that you did.
• Limit who can start Custom Labels models and who can read training images.
• Publish a short notice to users explaining what is analysed, why and for how long.

AWS CloudTrail logs who called which API, AWS Config keeps the history of related resources such as the bucket, and AWS Artifact has the compliance reports. AWS runs the models and the infrastructure. What is analysed, how long it is kept, the thresholds and the consequences of a wrong answer are yours.

• Analyse each picture once and cache the answer.
• Resize before upload so less data moves and less is processed.
• Sample video frames instead of scanning everything.
• Stop Custom Labels models when they are idle, since a started model keeps its capacity busy.
• Delete pictures and vectors you no longer need.
• The AWS carbon tool reports per account and doesn’t list Rekognition on its own, so quote no Rekognition figure.

Purchase options

What you actually pay for

• Images processed, by API group and monthly volume tier
• Minutes of stored video analysed
• Face and user vectors stored each month
• Face Liveness checks
• Custom Labels training and inference hours
• Data transfer, S3 storage and requests, and any Lambda or SNS you use around it

An image-intake setup: a private encrypted bucket, a face collection, an SNS topic and a role for video jobs, and an application policy that can only use the calls it needs. The CLI tab runs labels, moderation, text, face enrolment and search, a stored-video job, a liveness session and the clean-up. Terraform describes the bucket, collection, topic, role and policy. The Python tab screens an upload (moderation first), enrols and identifies faces and reads every page of a video job, and the C# tab audits the face collections in a Region. Replace the Region, bucket and account with your own.

# Placeholders: replace the Region, the bucket and the keys. Rekognition reads each image from S3 in the same Region as the call.
REGION=ap-south-1
BUCKET=my-image-intake
KEY=uploads/shopfront.jpg
ACCOUNT_ID=111122223333

# 1. What is in the picture? Labels above 80 percent confidence, plus the three dominant colours.
aws rekognition detect-labels --region $REGION \
  --image "S3Object={Bucket=$BUCKET,Name=$KEY}" \
  --features GENERAL_LABELS IMAGE_PROPERTIES --settings "ImageProperties={MaxDominantColors=3}" \
  --max-labels 15 --min-confidence 80

# 2. Is it safe to publish? Moderation labels come with a parent category, so you can block on the parent.
aws rekognition detect-moderation-labels --region $REGION \
  --image "S3Object={Bucket=$BUCKET,Name=$KEY}" --min-confidence 60

# 3. Text in the picture (signs, labels, plates). Keep only words detected at 85 percent or better.
aws rekognition detect-text --region $REGION \
  --image "S3Object={Bucket=$BUCKET,Name=$KEY}" --filters "WordFilter={MinConfidence=85}"

# 4. A face collection: create it, add one face for a known person, then search it with a new photo. 99 percent is the bar for decisions that matter.
aws rekognition create-collection --region $REGION --collection-id staff-faces --tags team=security,owner=platform
aws rekognition index-faces --region $REGION --collection-id staff-faces \
  --image "S3Object={Bucket=$BUCKET,Name=enrol/employee-1042.jpg}" \
  --external-image-id employee-1042 --max-faces 1 --quality-filter AUTO --detection-attributes DEFAULT
aws rekognition search-faces-by-image --region $REGION --collection-id staff-faces \
  --image "S3Object={Bucket=$BUCKET,Name=$KEY}" --face-match-threshold 99 --max-faces 1 --quality-filter AUTO
aws rekognition describe-collection --region $REGION --collection-id staff-faces
aws rekognition list-faces --region $REGION --collection-id staff-faces --max-results 10

# 5. Compare two photos directly, with no collection.
aws rekognition compare-faces --region $REGION --similarity-threshold 90 \
  --source-image "S3Object={Bucket=$BUCKET,Name=enrol/employee-1042.jpg}" \
  --target-image "S3Object={Bucket=$BUCKET,Name=$KEY}"

# 6. A stored video is analysed asynchronously: start the job, let Rekognition publish to SNS when it ends, then read the pages.
JOB_ID=$(aws rekognition start-label-detection --region $REGION --min-confidence 80 --job-tag gate-camera-1 \
  --video "S3Object={Bucket=$BUCKET,Name=videos/gate.mp4}" \
  --notification-channel "SNSTopicArn=arn:aws:sns:$REGION:$ACCOUNT_ID:AmazonRekognitionVideoDone,RoleArn=arn:aws:iam::$ACCOUNT_ID:role/rekognition-video-publish" \
  --query JobId --output text)
aws rekognition get-label-detection --region $REGION --job-id $JOB_ID --sort-by TIMESTAMP --max-results 100

# 7. Liveness check for a sign-up flow. Check first that your Region offers Face Liveness.
SESSION_ID=$(aws rekognition create-face-liveness-session --region $REGION \
  --settings "OutputConfig={S3Bucket=$BUCKET,S3KeyPrefix=liveness/},AuditImagesLimit=2" --query SessionId --output text)
aws rekognition get-face-liveness-session-results --region $REGION --session-id $SESSION_ID

# 8. Delete a person's faces when they leave, and the whole collection when the project ends.
aws rekognition delete-faces --region $REGION --collection-id staff-faces --face-ids $FACE_ID
aws rekognition delete-collection --region $REGION --collection-id staff-faces

# Private image intake: an encrypted bucket, a face collection, an SNS topic and a role for video jobs, and a least-privilege policy for the application.
terraform {
  required_providers {
    aws = { source = "hashicorp/aws", version = ">= 6.0" }
  }
}

provider "aws" {
  region = "ap-south-1"
  default_tags {
    tags = { project = "image-intake", owner = "platform-team" }
  }
}

data "aws_caller_identity" "me" {}

resource "aws_s3_bucket" "images" {
  bucket = "my-image-intake-${data.aws_caller_identity.me.account_id}"
}

resource "aws_s3_bucket_public_access_block" "images" {
  bucket                  = aws_s3_bucket.images.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

resource "aws_s3_bucket_server_side_encryption_configuration" "images" {
  bucket = aws_s3_bucket.images.id
  rule {
    apply_server_side_encryption_by_default { sse_algorithm = "AES256" }
  }
}

# Uploads expire after 30 days; pictures of people should not stay for ever.
resource "aws_s3_bucket_lifecycle_configuration" "images" {
  bucket = aws_s3_bucket.images.id
  rule {
    id     = "expire-uploads"
    status = "Enabled"
    filter { prefix = "uploads/" }
    expiration { days = 30 }
  }
}

# The face collection. It stores face vectors, not the pictures.
resource "aws_rekognition_collection" "staff" {
  collection_id = "staff-faces"
}

# Video jobs tell this topic when they finish; Rekognition publishes through the role below. The managed policy only allows topics whose name starts with AmazonRekognition.
resource "aws_sns_topic" "video_done" {
  name = "AmazonRekognitionVideoDone"
}

resource "aws_iam_role" "video_publish" {
  name = "rekognition-video-publish"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Service = "rekognition.amazonaws.com" }
      Action    = "sts:AssumeRole"
      Condition = { StringEquals = { "aws:SourceAccount" = data.aws_caller_identity.me.account_id } }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "video_publish" {
  role       = aws_iam_role.video_publish.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonRekognitionServiceRole"
}

# What the application may do: call the image APIs, search and add faces in one collection, and read uploads.
data "aws_iam_policy_document" "app" {
  statement {
    sid       = "ImageApis"
    actions   = ["rekognition:DetectLabels", "rekognition:DetectModerationLabels", "rekognition:DetectText", "rekognition:CompareFaces"]
    resources = ["*"]
  }
  statement {
    sid       = "OneCollection"
    actions   = ["rekognition:IndexFaces", "rekognition:SearchFacesByImage", "rekognition:ListFaces", "rekognition:DeleteFaces"]
    resources = [aws_rekognition_collection.staff.arn]
  }
  statement {
    sid       = "ReadUploads"
    actions   = ["s3:GetObject"]
    resources = ["${aws_s3_bucket.images.arn}/*"]
  }
}

resource "aws_iam_policy" "app" {
  name   = "image-intake-app"
  policy = data.aws_iam_policy_document.app.json
}

"""Image intake with Amazon Rekognition: screen an upload, read its labels and text, enrol and recognise faces, and page through a video job."""
import boto3
from botocore.config import Config
from botocore.exceptions import ClientError

# Adaptive retries slow the client down when Rekognition throttles it, which is what you want for a burst of uploads.
rek = boto3.client("rekognition", region_name="ap-south-1", config=Config(retries={"mode": "adaptive", "max_attempts": 8}))

BLOCK = {"Explicit", "Violence", "Hate Symbols", "Drugs & Tobacco"}   # top-level moderation categories this site refuses
REVIEW_FROM, BLOCK_FROM = 60.0, 90.0                                      # confidence bands: queue for a person, reject outright


def _s3(bucket, key):
    return {"S3Object": {"Bucket": bucket, "Name": key}}


def moderation_decision(labels):
    """'reject', 'review' or 'allow' from the labels DetectModerationLabels returned. A child label is judged through its parent."""
    decision = "allow"
    for label in labels:
        top = label["ParentName"] or label["Name"]
        score = label["Confidence"]
        if top in BLOCK and score >= BLOCK_FROM:
            return "reject"
        if score >= REVIEW_FROM:
            decision = "review"
    return decision


def screen_upload(bucket, key):
    """Moderation first. Labels and text are read only for pictures that are not rejected, so you do not pay for them."""
    mod = rek.detect_moderation_labels(Image=_s3(bucket, key), MinConfidence=REVIEW_FROM)
    decision = moderation_decision(mod["ModerationLabels"])
    result = {"key": key, "decision": decision, "moderation": [m["Name"] for m in mod["ModerationLabels"]]}
    if decision == "reject":
        return result
    labels = rek.detect_labels(Image=_s3(bucket, key), Features=["GENERAL_LABELS"], MaxLabels=15, MinConfidence=80)
    text = rek.detect_text(Image=_s3(bucket, key), Filters={"WordFilter": {"MinConfidence": 85}})
    result["labels"] = [lab["Name"] for lab in labels["Labels"]]
    result["words"] = [t["DetectedText"] for t in text["TextDetections"] if t["Type"] == "WORD"]
    return result


def enrol(collection, bucket, key, person_id):
    """Add one face to a collection. Blurred, tiny or dark faces are refused by the quality filter, and the caller hears why."""
    out = rek.index_faces(CollectionId=collection, Image=_s3(bucket, key), ExternalImageId=person_id,
                          MaxFaces=1, QualityFilter="AUTO", DetectionAttributes=["DEFAULT"])
    if not out["FaceRecords"]:
        reasons = [r for u in out.get("UnindexedFaces", []) for r in u["Reasons"]] or ["NO_FACE_FOUND"]
        raise ValueError("face not enrolled: " + ", ".join(reasons))
    return out["FaceRecords"][0]["Face"]["FaceId"]


def identify(collection, bucket, key, threshold=99.0):
    """The person_id of the best match above the threshold, or None. A picture with no usable face is a 'no', not an error."""
    try:
        out = rek.search_faces_by_image(CollectionId=collection, Image=_s3(bucket, key), FaceMatchThreshold=threshold,
                                        MaxFaces=1, QualityFilter="AUTO")
    except ClientError as err:
        if err.response["Error"]["Code"] == "InvalidParameterException":
            return None
        raise
    matches = out["FaceMatches"]
    return matches[0]["Face"]["ExternalImageId"] if matches else None


def forget(collection, face_ids):
    """Delete face vectors (for example when a person leaves or withdraws consent). Returns the ids that were removed."""
    return rek.delete_faces(CollectionId=collection, FaceIds=list(face_ids))["DeletedFaces"]


def video_labels(job_id):
    """Read every page of a finished label-detection job. Returns {label: first time it appears, in milliseconds}."""
    first, token = {}, None
    while True:
        args = {"JobId": job_id, "SortBy": "TIMESTAMP", "MaxResults": 100}
        if token:
            args["NextToken"] = token
        page = rek.get_label_detection(**args)
        if page["JobStatus"] != "SUCCEEDED":
            raise RuntimeError("job is %s: %s" % (page["JobStatus"], page.get("StatusMessage", "")))
        for item in page["Labels"]:
            first.setdefault(item["Label"]["Name"], item["Timestamp"])
        token = page.get("NextToken")
        if not token:
            return first

// Audit every face collection in one Region: how many faces it holds, which model made them, and whether anyone owns it.
// Needs rekognition:ListCollections, rekognition:DescribeCollection and rekognition:ListTagsForResource.
using Amazon;
using Amazon.Rekognition;
using Amazon.Rekognition.Model;

var region = RegionEndpoint.APSouth1;
using var rek = new AmazonRekognitionClient(region);

var ids = new List<string>();
string? token = null;
do
{
    var page = await rek.ListCollectionsAsync(new ListCollectionsRequest { MaxResults = 100, NextToken = token });
    ids.AddRange(page.CollectionIds);
    token = page.NextToken;
} while (token != null);

Console.WriteLine($"{ids.Count} collection(s) in {region.SystemName}");
var findings = 0;
foreach (var id in ids.OrderBy(i => i, StringComparer.Ordinal))
{
    var info = await rek.DescribeCollectionAsync(new DescribeCollectionRequest { CollectionId = id });
    var tags = (await rek.ListTagsForResourceAsync(new ListTagsForResourceRequest { ResourceArn = info.CollectionARN })).Tags;

    var notes = new List<string>();
    if (info.FaceCount == 0) notes.Add("empty, delete it if nobody uses it");
    if (!tags.ContainsKey("owner")) notes.Add("no owner tag");
    if (info.FaceCount > 100_000) notes.Add("large, check that you still need every face");

    findings += notes.Count;
    Console.WriteLine($"{id,-30} faces={info.FaceCount,-8} model={info.FaceModelVersion,-5} created={info.CreationTimestamp:yyyy-MM-dd}  {(notes.Count == 0 ? "ok" : string.Join("; ", notes))}");
}
Environment.ExitCode = findings == 0 ? 0 : 1;   // lets a scheduled job fail when something needs a look

Videos picked for this service. Each one is labelled with its source, opens on YouTube if you prefer, and loads only when you press play.

Typical callers are AWS Lambda functions started by uploads to Amazon S3, with results saved to Amazon DynamoDB and routed on through Amazon EventBridge. Spend, throttling and errors show up in Amazon CloudWatch.