Gemini API and Model Garden on Vertex AIAuthorCloudBrum editorial teamLast reviewed30 Oct 2026Reading time~12 minLevelBeginner to advanced

• A catalogue with three kinds of modelModel Garden lists Google’s Gemini and other Google models, partner models such as Claude, Llama and Mistral, and open models. You try them in the console, call them, tune some of them or deploy some to your own endpoint.
• The Gemini API on Google CloudSend text, images, audio, video or documents to generateContent, stream the answer, and count tokens before you send. It looks like the Gemini Developer API, but you sign in with IAM instead of an API key and you choose the Region.
• Managed calls or your own machinesPartner and some open models run as managed APIs that bill per token. You can also deploy an open model to an endpoint on GPUs you choose, which bills by the node-hour and makes you the operator.
• Grounding in search or in your dataGround an answer in Google Search, or in your own documents through Vertex AI Search or RAG Engine. Each grounded request is metered, and the answer can carry its sources.
• Safety settings and Model ArmorBuilt-in safety settings block harmful output by category. Model Armor screens prompts and answers for injection, jailbreak attempts, harmful content and sensitive data, and works with the Gemini API.
• Agents with ADK and Agent EngineThe Agent Development Kit is the open framework for writing agents with tools. Agent Engine is the managed runtime that hosts them, with sessions and memory.
• Shared capacity or reserved capacityPay-as-you-go calls draw on a shared pool with no fixed limit, so a busy pool means 429 errors. Provisioned Throughput reserves capacity in GSUs for a week up to a year, and Batch and Flex trade speed for a lower price.
• Caching and tuningContext caching bills a repeated long prompt at a fraction of the normal input rate. Supervised tuning adapts a Gemini model to your examples without running training machines yourself.

Gemini API on Vertex AI

The generateContent family of calls on aiplatform.googleapis.com, used with IAM credentials and a Region. The Gemini Developer API is a separate service with API keys.

Model Garden

The catalogue of Google, partner and open models, with a card for each that shows how to call, tune or deploy it.

Publisher model

A model offered by Google or a partner, named in the URL, as in publishers/google/models/gemini-2.5-flash.

Managed API (model as a service)

A partner or open model that Google runs for you and bills per token, with nothing to deploy.

Self-deployed model

An open model you deploy from Model Garden to your own endpoint on chosen machines, billed by the node-hour.

Google Gen AI SDK

The google-genai library. Setting vertexai=True with a project and location points it at Vertex AI instead of the Developer API.

Global and regional endpoints

A regional endpoint such as asia-south1 keeps processing in that Region. The global endpoint gives more capacity but doesn’t say where the request runs.

Dynamic shared quota

Pay-as-you-go capacity shared by all customers. There is no per-project cap, and a 429 means the pool is busy.

Provisioned Throughput

Reserved capacity measured in GSUs, bought for a period from a week to a year.

Context caching

Storing a long, repeated part of the prompt once, so later calls pay a lower rate for it and finish sooner.

Grounding

Tying an answer to Google Search results or to your own data, and returning the sources.

Safety settings

Per-category thresholds that block harmful prompts or answers. A blocked answer ends with the finish reason SAFETY.

Model Armor

A screening service for prompts and answers, set up as a template and attached to calls.

Agent Engine

The managed runtime for agents built with the Agent Development Kit or other frameworks.

Call Gemini on Vertex AI as a service account, then add safeguards, with gcloud and curl

1. Pick a project and turn on the APIUse a project with billing attached and enable aiplatform.googleapis.com. Add modelarmor.googleapis.com if you plan to screen prompts.
2. Choose where requests runA regional endpoint such as Mumbai (asia-south1) keeps processing in that Region, if the model is offered there. Delhi (asia-south2) is a listed Region for machine learning services, but check the model’s page before relying on it for Gemini. The global endpoint has more capacity and no location promise.
3. Create an identity with the least it needsMake a service account and give it roles/aiplatform.user. The application runs as that account, so there is no key to store.
4. Enable partner models if you want themModels such as Claude are switched on for a project from their Model Garden card, which accepts the provider’s terms. The person who enables one needs a role that can purchase from Marketplace, and callers need the Vertex AI user role. Gemini needs no such step.
5. Make a first callUse curl or the Python SDK with a small maxOutputTokens. Read the finish reason and the usage numbers, so you see why it ended and what it used.
6. Add safeguardsSet safety thresholds for your use, create a Model Armor template for injection and sensitive data, and add grounding only where answers must cite sources.
7. Plan for 429 responsesRetry with truncated exponential backoff and jitter, ramp traffic up gradually, try the global endpoint, ask for a quota increase where the model has quotas, and buy Provisioned Throughput if you need a firm level.
8. Watch and cap spendSet a budget alert, chart invocation counts and errors in Cloud Monitoring, and turn on request-response logging only if you need it and have somewhere safe to keep it.
9. Add your data and agents lastBring in Vertex AI Search or RAG Engine and the Agent Development Kit after the plain call works. Both bring extra cost, new permissions and new ways to break.

• Start from a Flash-class model and move up only when your own tests demand it. It answers sooner and costs less.
• Stream the answer so people see text at once.
• Put the long, unchanging part of a prompt first and use context caching for it.
• Set maxOutputTokens to what you need, and allow for thinking tokens on models that think.
• Use the global endpoint when capacity matters more than location, and a regional one when it doesn’t.
• Send offline work to Batch.
• Spread traffic evenly, since sharp rises can trigger limits that a steady ramp avoids.
• Ground with fewer, better sources, and test retrieval on its own.

FinOps is the habit of managing cloud spend together with engineering. These are the FinOps checks that matter most for Gemini API and Model Garden.

• Try a Flash-class model before a larger one, and measure cost per correct answer.
• Use Batch or Flex for work that can wait, at about half the standard price.
• Turn on context caching for long repeated prompts, where cached input costs a fraction of normal input.
• Shorten chat history and grounded context. Whatever you send again is billed again.
• Cap output length, and count tokens before sending large inputs.
• Count grounded requests. Each search query used for grounding is metered separately.
• Compare Provisioned Throughput with your measured tokens before committing, since it bills whether or not you call.
• Undeploy self-deployed endpoints the same day, because they bill by the node-hour.
• Watch model price changes and introductory rates, which can end on a published date.
• Set a budget alert, label resources by team and review spend weekly.

• Run apps as service accounts with roles/aiplatform.user, and avoid keys.
• Use Private Service Connect or Private Google Access so calls stay off the public internet, and add VPC Service Controls where data rules demand it.
• Screen prompts and answers with Model Armor, and set safety thresholds that fit your use.
• Use regional endpoints for regulated data, and read the data residency page for each model and feature.
• Use customer-managed encryption keys for tuning data and stored artifacts where you need them.
• Keep request-response logging off unless needed, and protect and expire what it writes.
• Send Cloud Audit Logs to a protected project and alert on changes to IAM, endpoints and Model Armor templates.
• Run each agent and tool under a narrow service account of its own, and validate the inputs a tool gets.
• Use organisation policy to limit locations and who may enable partner models.
• Read the license and data terms of any partner or open model before a project turns it on.

Cloud Audit Logs show who changed a project and, once you turn on Data Access logs, who called the APIs. Cloud Monitoring holds the invocation metrics, and Assured Workloads and the compliance reports describe the controls Google offers. Google runs the hosting of its models. What stays with you: the model you pick, what goes into prompts, who has access, the safety thresholds and the testing.

• Try a Flash-class model before a larger one: it needs less hardware for each answer.
• Cache a long repeated prefix so later calls skip that work.
• Run bulk work through Batch, so each request is not a separate call.
• Undeploy idle endpoints and delete unused Provisioned Throughput.
• Google publishes carbon data for its Regions, so use it to choose when location rules leave you a choice.
• Google publishes no per-request energy figure for hosted models that we could verify, so none appears here.

Purchase options

What you actually pay for

• Input and output tokens for every call, including thinking tokens on models that think
• Provisioned Throughput for its whole term
• Grounding queries, caching storage and tuning jobs
• Node-hours for self-deployed endpoints, and a partner model’s own fee
• Cloud Logging, BigQuery for request logs, Model Armor and data transfer

A first Gemini setup on Vertex AI: turn on the API, create a service account with one role, list models in Model Garden, send a question and count tokens with curl, create a Model Armor template and set a budget alert. The gcloud tab walks through it in order. Terraform builds the same resources plus an alert on failed calls. The Python tab uses the Google Gen AI SDK with retries for 429 responses, streaming and token counting, and the C# tab sends a counted, limited question and exits with 1 when a safety setting blocks the answer. Replace the project, the Region and the model with your own.

# Placeholders: use your own project, and a Gemini model that the Region offers (check the Model Garden card for the exact ID).
PROJECT_ID=my-project
LOCATION=asia-south1                # Mumbai. Use "global" for the most capacity, but then Google picks where the request runs.
MODEL=gemini-2.5-flash
API=https://$LOCATION-aiplatform.googleapis.com
[ "$LOCATION" = global ] && API=https://aiplatform.googleapis.com

# 1. Turn on the API, and Model Armor if you want prompt screening.
gcloud services enable aiplatform.googleapis.com modelarmor.googleapis.com --project $PROJECT_ID

# 2. An identity for the app with only the Vertex AI user role. No keys: the app runs as this service account.
gcloud iam service-accounts create gemini-app --project $PROJECT_ID --display-name "Gemini caller"
gcloud projects add-iam-policy-binding $PROJECT_ID --member "serviceAccount:gemini-app@$PROJECT_ID.iam.gserviceaccount.com" --role roles/aiplatform.user

# 3. What does Model Garden offer? Narrow the list with a filter on the name.
gcloud ai model-garden models list --model-filter gemini --limit 20

# 4. Ask Gemini a question. The model name is part of the URL, and maxOutputTokens limits the answer.
curl -s -X POST "$API/v1/projects/$PROJECT_ID/locations/$LOCATION/publishers/google/models/$MODEL:generateContent" \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/json" \
  -d '{"systemInstruction":{"parts":[{"text":"Answer in at most three sentences."}]},
       "contents":[{"role":"user","parts":[{"text":"What is the difference between the Gemini API and Model Garden?"}]}],
       "generationConfig":{"maxOutputTokens":400,"temperature":0.2}}' \
  | jq '{answer: .candidates[0].content.parts[0].text, finish: .candidates[0].finishReason, tokens: .usageMetadata}'

# 5. Count the tokens of a prompt before you send it, to check cost and the context window.
curl -s -X POST "$API/v1/projects/$PROJECT_ID/locations/$LOCATION/publishers/google/models/$MODEL:countTokens" \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/json" \
  -d '{"contents":[{"role":"user","parts":[{"text":"A long prompt you want to measure."}]}]}' | jq .totalTokens

# 6. A Model Armor template that screens for prompt injection and jailbreaks, and for harmful content. Attach it to calls from your app.
gcloud model-armor templates create gemini-guard --project $PROJECT_ID --location $LOCATION \
  --pi-and-jailbreak-filter-settings-enforcement enabled --pi-and-jailbreak-filter-settings-confidence-level medium-and-above \
  --rai-settings-filters '[{"filterType":"HATE_SPEECH","confidenceLevel":"MEDIUM_AND_ABOVE"},{"filterType":"DANGEROUS","confidenceLevel":"MEDIUM_AND_ABOVE"}]' \
  --basic-config-filter-enforcement enabled

# 7. A budget alert on the billing account, so a loop or a leaked credential shows up at 50 and 90 percent.
gcloud billing budgets create --billing-account 000000-AAAAAA-BBBBBB --display-name "Gemini spend" \
  --filter-projects "projects/$PROJECT_ID" --budget-amount 100USD --threshold-rule percent=0.5 --threshold-rule percent=0.9

# Placeholders: set your project and billing account. This enables the APIs, creates an identity for the app with only the Vertex AI user role,
# a Model Armor template, a budget alert and an alert on failed model calls.
variable "project_id" {
  type = string
}

variable "billing_account" {
  type        = string
  description = "Billing account ID, for example 000000-AAAAAA-BBBBBB"
}

variable "region" {
  type    = string
  default = "asia-south1"
}

provider "google" {
  project = var.project_id
  region  = var.region
}

resource "google_project_service" "apis" {
  for_each           = toset(["aiplatform.googleapis.com", "modelarmor.googleapis.com", "monitoring.googleapis.com"])
  service            = each.key
  disable_on_destroy = false
}

# The app runs as this service account. There are no keys to leak.
resource "google_service_account" "gemini_app" {
  account_id   = "gemini-app"
  display_name = "Gemini caller"
}

resource "google_project_iam_member" "can_call_models" {
  project = var.project_id
  role    = "roles/aiplatform.user"
  member  = "serviceAccount:${google_service_account.gemini_app.email}"
}

# Screens prompts and answers for injection and jailbreak attempts and for harmful content. The app names this template in its calls.
resource "google_model_armor_template" "guard" {
  location    = var.region
  template_id = "gemini-guard"

  filter_config {
    pi_and_jailbreak_filter_settings {
      filter_enforcement = "ENABLED"
      confidence_level   = "MEDIUM_AND_ABOVE"
    }
    rai_settings {
      rai_filters {
        filter_type      = "HATE_SPEECH"
        confidence_level = "MEDIUM_AND_ABOVE"
      }
      rai_filters {
        filter_type      = "DANGEROUS"
        confidence_level = "MEDIUM_AND_ABOVE"
      }
    }
  }

  depends_on = [google_project_service.apis]
}

data "google_project" "this" {
  project_id = var.project_id
}

resource "google_billing_budget" "gemini" {
  billing_account = var.billing_account
  display_name    = "Gemini spend"

  budget_filter {
    projects = ["projects/${data.google_project.this.number}"]
  }

  amount {
    specified_amount {
      currency_code = "USD"
      units         = "100"
    }
  }

  threshold_rules {
    threshold_percent = 0.5
  }

  threshold_rules {
    threshold_percent = 0.9
  }
}

# Model calls that did not return 200. The metric and its labels are as in Metrics Explorer; check them there for your model.
resource "google_monitoring_alert_policy" "failed_calls" {
  display_name = "Gemini calls failing"
  combiner     = "OR"

  conditions {
    display_name = "More than 20 failed calls in 5 minutes"

    condition_threshold {
      filter          = "resource.type = \"aiplatform.googleapis.com/PublisherModel\" AND metric.type = \"aiplatform.googleapis.com/publisher/online_serving/model_invocation_count\" AND metric.labels.response_code != \"200\""
      comparison      = "COMPARISON_GT"
      threshold_value = 20
      duration        = "300s"

      aggregations {
        alignment_period   = "300s"
        per_series_aligner = "ALIGN_SUM"
      }
    }
  }

  depends_on = [google_project_service.apis]
}

output "service_account" {
  value = google_service_account.gemini_app.email
}

# pip install google-genai
# Calls Gemini on Vertex AI as the identity you are logged in with (no API key), retries when the shared pool is busy,
# streams an answer, and counts tokens before sending. Run `gcloud auth application-default login` first, or run on Google Cloud with a service account.
import os
import random
import time

from google import genai
from google.genai import errors, types

PROJECT = os.environ["GOOGLE_CLOUD_PROJECT"]
LOCATION = os.environ.get("GOOGLE_CLOUD_LOCATION", "asia-south1")    # "global" gives the most capacity but no promise about where requests run
MODEL = os.environ.get("GEMINI_MODEL", "gemini-2.5-flash")

client = genai.Client(vertexai=True, project=PROJECT, location=LOCATION)


def ask(question, system="Answer in at most three sentences.", max_output_tokens=400, attempts=6):
    """Returns (text, finish_reason, usage). Retries 429, 500 and 503 with truncated exponential backoff and jitter, as Google advises for a busy shared pool."""
    config = types.GenerateContentConfig(system_instruction=system, max_output_tokens=max_output_tokens, temperature=0.2)
    for attempt in range(attempts):
        try:
            reply = client.models.generate_content(model=MODEL, contents=question, config=config)
            finish = reply.candidates[0].finish_reason if reply.candidates else None
            return reply.text, finish, reply.usage_metadata
        except errors.APIError as error:
            if error.code not in (429, 500, 503) or attempt == attempts - 1:
                raise
            time.sleep(min(2 ** attempt, 32) + random.random())


def ask_streaming(question, max_output_tokens=400):
    """Streams the answer to the console and returns it as one string."""
    pieces = []
    config = types.GenerateContentConfig(max_output_tokens=max_output_tokens)
    for chunk in client.models.generate_content_stream(model=MODEL, contents=question, config=config):
        if chunk.text:
            print(chunk.text, end="", flush=True)
            pieces.append(chunk.text)
    print()
    return "".join(pieces)


def tokens(question):
    """How many tokens the prompt uses, before you pay for an answer."""
    return client.models.count_tokens(model=MODEL, contents=question).total_tokens


if __name__ == "__main__":
    text, finish, usage = ask("What is the difference between the Gemini API and Model Garden?")
    print(text)
    print("finish:", finish, "| tokens in/out:", usage.prompt_token_count, usage.candidates_token_count)
    if str(finish).endswith("MAX_TOKENS"):
        print("The answer was cut off. Models that think first spend part of the limit on thinking, so raise max_output_tokens.")
    elif str(finish).endswith("SAFETY"):
        print("A safety setting stopped this answer.")

// dotnet add package Google.Cloud.AIPlatform.V1
// Sends one question to Gemini on Vertex AI as the identity from Application Default Credentials, counts the tokens first,
// and reports why the answer ended. Exits with 1 when a safety setting blocked it, so a pipeline can notice.
using Google.Cloud.AIPlatform.V1;

string project = Environment.GetEnvironmentVariable("GOOGLE_CLOUD_PROJECT") ?? throw new InvalidOperationException("Set GOOGLE_CLOUD_PROJECT");
string location = Environment.GetEnvironmentVariable("GOOGLE_CLOUD_LOCATION") ?? "asia-south1";
string model = Environment.GetEnvironmentVariable("GEMINI_MODEL") ?? "gemini-2.5-flash";
string modelName = $"projects/{project}/locations/{location}/publishers/google/models/{model}";

// A regional endpoint keeps the call in that Region. For the "global" location the host is plain aiplatform.googleapis.com.
string host = location == "global" ? "aiplatform.googleapis.com" : $"{location}-aiplatform.googleapis.com";
PredictionServiceClient client = new PredictionServiceClientBuilder { Endpoint = host }.Build();
LlmUtilityServiceClient utility = new LlmUtilityServiceClientBuilder { Endpoint = host }.Build();

var contents = new List<Content>
{
    new Content { Role = "USER", Parts = { new Part { Text = "What is the difference between the Gemini API and Model Garden?" } } }
};

CountTokensResponse counted = await utility.CountTokensAsync(new CountTokensRequest { Endpoint = modelName, Model = modelName, Contents = { contents } });
Console.WriteLine($"prompt tokens: {counted.TotalTokens}");

GenerateContentResponse reply = await client.GenerateContentAsync(new GenerateContentRequest
{
    Model = modelName,
    SystemInstruction = new Content { Parts = { new Part { Text = "Answer in at most three sentences." } } },
    Contents = { contents },
    GenerationConfig = new GenerationConfig { MaxOutputTokens = 400, Temperature = 0.2f }
});

Candidate? first = reply.Candidates.FirstOrDefault();
Console.WriteLine(first?.Content?.Parts.FirstOrDefault()?.Text ?? "(no text)");
Console.WriteLine($"finish: {first?.FinishReason} | tokens in/out: {reply.UsageMetadata?.PromptTokenCount}/{reply.UsageMetadata?.CandidatesTokenCount}");

if (first?.FinishReason == Candidate.Types.FinishReason.Safety)
{
    Console.WriteLine("A safety setting blocked this answer.");
    return 1;
}
if (first?.FinishReason == Candidate.Types.FinishReason.MaxTokens)
{
    Console.WriteLine("The answer was cut off. Models that think first spend part of the limit on thinking, so raise MaxOutputTokens.");
}
return 0;

Videos picked for this service. Each one is labelled with its source, opens on YouTube if you prefer, and loads only when you press play.

Typical companions are Cloud Storage for documents, Cloud Run functions for glue code, Cloud Monitoring for metrics and alerts and Cloud KMS for keys. To train, tune with your own code or host a model of your own, look at the Gemini Enterprise Agent Platform (Vertex AI) machine learning page.