Google Kubernetes EngineAuthorCloudBrum editorial teamLast reviewed28 Oct 2026Reading time~12 minLevelBeginner to advanced

• Autopilot modeYou write pods with CPU and memory requests, and Google provisions, scales, patches and secures the nodes. Billing follows what your pods request, not the machines underneath, so empty capacity isn’t yours to pay for.
• Standard modeYou create node pools, choose machine types, set autoscaling limits and decide how nodes upgrade. It gives the most control, including privileged workloads and unusual hardware, and it leaves the packing and the idle capacity to you.
• Release channelsRapid, Regular and Stable trade speed for caution, and an Extended channel on Standard clusters keeps a minor version supported for longer. GKE upgrades the control plane and nodes inside the channel’s rules, and you pick the maintenance windows.
• Workload Identity FederationPods call Google APIs as their own Kubernetes service account. You grant an IAM role directly to that identity, so there are no key files to create, mount or rotate. It’s always on in Autopilot.
• VPC-native networkingPods and services get addresses from secondary ranges in your subnet, so they route like any other VPC address. Dataplane V2, built on Cilium, enforces network policy and adds logging.
• Gateway APIGoogle Cloud load balancers are configured with the Kubernetes Gateway and HTTPRoute objects, with global or regional options, and Cloud Armor and managed certificates attach to them.
• Private clusters and postureKeep nodes on private addresses, limit who can reach the control plane, scan workloads for vulnerabilities and watch configuration drift from a security posture dashboard.
• FleetsThe Enterprise edition manages many clusters as a fleet, with shared policy, configuration sync from Git and multi-cluster services across projects.

Autopilot

A GKE mode where Google owns the nodes. Pods are billed by their requests, and some node-level settings (privileged containers, host access) are off limits.

Standard

A GKE mode where you manage node pools, machine types and upgrade strategy, and pay for the virtual machines.

Zonal and regional cluster

A zonal cluster has its control plane in one zone. A regional cluster spreads the control plane, and by default the nodes, over three zones, and has the higher availability target.

Node pool

A group of nodes in a Standard cluster that share a machine type and settings. Autoscaling and upgrades act on the pool.

Compute class

A named hardware profile that a pod asks for with a node selector, such as Balanced or Scale-Out. Autopilot uses them to choose the machine family.

Release channel

A subscription to Kubernetes versions: Rapid, Regular, Stable or, for Standard clusters, Extended. The channel decides which version you get and when.

Maintenance window

A recurring time slot when GKE may do disruptive upgrades. Exclusions block upgrades for a chosen period, such as a sales peak.

VPC-native cluster

A cluster that gets its pod and service ranges from alias IP ranges in a subnet, so no routes need to be created per node.

Dataplane V2

GKE’s networking layer built on Cilium. It enforces Kubernetes network policy and gives flow logging, and it’s the default on Autopilot.

Workload Identity Federation for GKE

Maps a Kubernetes service account to an identity inside a workload identity pool, PROJECT_ID.svc.id.goog, which IAM can grant roles to.

Private cluster

A cluster whose nodes have only internal addresses. The control plane can be reached privately, through authorised networks, or through a DNS-based endpoint.

Gateway

A Kubernetes API object that describes a load balancer. GKE turns a Gateway and its routes into a Google Cloud load balancer.

Binary Authorization

A deploy-time check that admits only images signed or attested by your pipeline.

Fleet

A logical group of clusters registered together so that policy, configuration and services can be managed as one.

Create an Autopilot cluster with gcloud

1. Choose the project, region and modePick a project that belongs to a billing account, and a region such as asia-south1 (Mumbai) or asia-south2 (Delhi). Use Autopilot unless you know you need node-level control. Autopilot clusters are regional.
2. Enable the servicesTurn on the Kubernetes Engine and Artifact Registry APIs. The first cluster in a project can take a few extra minutes while GKE sets itself up.
3. Prepare the networkCreate a VPC subnet with two secondary ranges, one for pods and one for services, and with Private Google Access. Private nodes have no external address, so add Cloud Router and Cloud NAT if they need to reach the internet.
4. Create the clusterRun gcloud container clusters create-auto with the network, the secondary ranges, private nodes, authorised networks for the control plane and a release channel.
5. Set the maintenance scheduleAdd a recurring maintenance window outside business hours, and an exclusion for any freeze period.
6. ConnectRun gcloud container clusters get-credentials. With --dns-endpoint you can reach the control plane from anywhere you’re allowed by IAM, without an authorised network entry.
7. Grant access to the workloadGive the service account’s workload identity the IAM role it needs on the bucket. The principal is named after the project, namespace and service account, and no annotation is required.
8. DeployApply the namespace, service account, deployment, service, gateway, route, autoscaler, disruption budget and network policy, and wait for the rollout. Autopilot may take a minute to add nodes for the first pods.
9. Plan the upgradesCheck which versions your channel offers, set maintenance windows, and test a Regular or Stable upgrade on a staging cluster first.

• Set requests from observed usage. Autopilot bills them and the scheduler uses them, so they decide both cost and placement.
• Use the Horizontal Pod Autoscaler for replicas, and Vertical Pod Autoscaler in recommendation mode to find the right size.
• Choose a compute class that fits: Balanced for general services, Scale-Out for cheap parallel work, accelerator classes for GPUs.
• Spread replicas over zones with topology spread constraints, and use regional clusters for production.
• Use NEGs through the Gateway API so the load balancer sends traffic straight to pods, not through a node hop.
• Turn on image streaming and keep images small, since pod start-up time is dominated by the pull on a fresh node.
• Keep a small buffer of spare capacity with a low-priority placeholder deployment if scale-up delay hurts your latency target.
• Place stateful workloads and their Persistent Disks in the same zone, and use regional disks when a pod must survive a zone failure.

FinOps is the habit of managing cloud spend together with engineering. These are the FinOps checks that matter most for Google Kubernetes Engine.

• Remember the cluster management fee of $0.10 per cluster-hour, and use the monthly free credit, which covers one Autopilot or zonal cluster per billing account.
• On Autopilot, right-size requests. Unused requests are billed at the same rate as used ones.
• On Standard, keep node utilisation high with the cluster autoscaler or node auto-provisioning, because you pay for idle VMs.
• Compare modes by node utilisation: Autopilot usually wins for bursty or small workloads, and Standard for dense, steady ones. Measure before you move.
• Use Spot pods or Spot VMs for fault-tolerant batch work.
• Buy committed use discounts for the steady base of compute, and let spikes run at the normal rate.
• Share gateways and load balancers between services, since each forwarding rule bills by the hour.
• Watch inter-zone and egress traffic. Chatty services across zones and large downloads add to the bill.
• Send only the logs you need to Cloud Logging, and exclude noisy namespaces from ingestion.
• Label clusters and namespaces, export billing data to BigQuery and set budget alerts.

• Use private nodes, limit the control plane with authorised networks or the DNS-based endpoint, and keep the endpoint off the public internet where possible.
• Give pods Workload Identity Federation instead of key files, and use a dedicated, minimal service account for nodes in Standard clusters.
• Grant people IAM roles at project level and Kubernetes RBAC at namespace level, with groups instead of individual accounts.
• Enable Shielded GKE Nodes (the default) and use Container-Optimized OS images, which GKE patches through upgrades.
• Enforce Pod Security Standards, run containers as non-root and drop capabilities. Autopilot already blocks several risky settings.
• Write network policies, beginning with default deny, and let Dataplane V2 enforce them.
• Encrypt secrets with a Cloud KMS key (application-layer secrets encryption), and prefer Secret Manager for real secrets.
• Turn on Binary Authorization to admit only attested images, and scan images in Artifact Registry.
• Use the security posture dashboard and workload vulnerability scanning to catch misconfigurations and known flaws.
• Keep Cloud Audit Logs for the Kubernetes API on, and alert on exec, role bindings and secret reads.

Cloud Audit Logs record who created, changed or deleted clusters and what was done through the Kubernetes API, Security Command Center reports posture findings, Policy Controller (in the Enterprise edition, or open source Gatekeeper) enforces admission rules, and Compliance Manager and the Cloud compliance pages hold certificates. The shared duties are the standard managed-Kubernetes ones: Google runs and patches the control plane, and on Autopilot the nodes too, while you answer for workload permissions, network policy, images and how soon you adopt upgrades.

• Right-size requests. Pods that ask for more than they use keep hardware running for nothing.
• Let Autopilot or the autoscaler remove idle capacity, and delete test clusters.
• Choose a Region with a low carbon share when data rules allow. Google publishes the carbon-free energy percentage per region.
• Batch non-urgent jobs onto Spot capacity that would otherwise sit idle.
• Share clusters across teams with namespaces instead of duplicating whole stacks.
• Compress logs and keep traffic in-zone to reduce stored and moved data.
• Cloud Carbon Footprint in the console estimates emissions per project and service. Check what it shows for GKE before you quote a figure.

Purchase options

What you actually pay for

• The cluster management fee, less the monthly free credit where it applies
• Pod requests (Autopilot) or node VMs (Standard)
• Persistent disks, Filestore and snapshots
• Load balancers, Cloud NAT and data leaving Google Cloud or crossing zones
• Logging, monitoring, the Enterprise edition and extended support if you use them

The orders-api service once more, this time on a regional, private Autopilot cluster in Mumbai. The gcloud tab prepares the network, creates the cluster, grants a bucket role directly to the pod’s identity and applies the manifest (service account, deployment, Gateway, route, autoscaler, disruption budget and network policy). Terraform builds the same cluster. The Python tab runs inside the pod and writes a receipt to Cloud Storage with the identity GKE hands it. The C# tab runs from a pipeline and lists pods that keep restarting or were killed for memory. Replace the project, project number, bucket and ranges with your own. A cluster bills from the moment it exists, so delete the test one when you’re done.

# Placeholders: replace the project, project number, bucket and IP ranges with your own.
PROJECT=my-orders-project
PROJECT_NUMBER=123456789012
REGION=asia-south1
CLUSTER=orders-prod
BUCKET=orders-receipts-example

gcloud config set project $PROJECT
gcloud services enable container.googleapis.com artifactregistry.googleapis.com compute.googleapis.com

# The network: one subnet with secondary ranges for pods and services, and NAT for private nodes.
gcloud compute networks create orders-net --subnet-mode=custom
gcloud compute networks subnets create orders-nodes --network=orders-net --region=$REGION --range=10.10.0.0/20 \
  --secondary-range=pods=10.64.0.0/14,services=10.68.0.0/20 --enable-private-ip-google-access
gcloud compute routers create orders-router --network=orders-net --region=$REGION
gcloud compute routers nats create orders-nat --router=orders-router --region=$REGION --auto-allocate-nat-external-ips --nat-all-subnet-ip-ranges

# The cluster: Autopilot, private nodes, the control plane limited to the office range, and the Regular release channel.
gcloud container clusters create-auto $CLUSTER --region=$REGION \
  --network=orders-net --subnetwork=orders-nodes \
  --cluster-secondary-range-name=pods --services-secondary-range-name=services \
  --enable-private-nodes --enable-master-authorized-networks --master-authorized-networks=203.0.113.0/24 \
  --release-channel=regular --binauthz-evaluation-mode=PROJECT_SINGLETON_POLICY_ENFORCE \
  --security-posture=standard --workload-vulnerability-scanning=standard

# Upgrades only on Saturday nights (UTC), and a freeze around the end-of-quarter sale.
gcloud container clusters update $CLUSTER --region=$REGION \
  --maintenance-window-start=2026-11-07T21:00:00Z --maintenance-window-end=2026-11-08T01:00:00Z \
  --maintenance-window-recurrence="FREQ=WEEKLY;BYDAY=SA"

# Workload Identity Federation needs no service account: grant the role straight to the pod's principal.
gcloud storage buckets add-iam-policy-binding gs://$BUCKET \
  --member="principal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$PROJECT.svc.id.goog/subject/ns/orders/sa/orders-api" \
  --role=roles/storage.objectCreator

# Connect through the DNS-based endpoint, then deploy.
gcloud container clusters get-credentials $CLUSTER --region=$REGION --dns-endpoint
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: orders
  labels:
    pod-security.kubernetes.io/enforce: restricted
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: orders-api
  namespace: orders
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-api
  namespace: orders
spec:
  replicas: 3
  selector:
    matchLabels:
      app: orders-api
  template:
    metadata:
      labels:
        app: orders-api
    spec:
      serviceAccountName: orders-api
      nodeSelector:
        cloud.google.com/compute-class: Balanced
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: api
          image: $REGION-docker.pkg.dev/$PROJECT/apps/orders-api:1.4.2
          ports:
            - containerPort: 8080
          env:
            - name: RECEIPT_BUCKET
              value: $BUCKET
          resources:
            requests:
              cpu: 250m
              memory: 512Mi
              ephemeral-storage: 1Gi
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
---
apiVersion: v1
kind: Service
metadata:
  name: orders-api
  namespace: orders
spec:
  selector:
    app: orders-api
  ports:
    - port: 80
      targetPort: 8080
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: orders-gw
  namespace: orders
spec:
  gatewayClassName: gke-l7-global-external-managed
  listeners:
    - name: http
      protocol: HTTP
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: orders-api
  namespace: orders
spec:
  parentRefs:
    - name: orders-gw
  rules:
    - backendRefs:
        - name: orders-api
          port: 80
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: orders-api
  namespace: orders
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: orders-api
  minReplicas: 3
  maxReplicas: 12
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: orders-api
  namespace: orders
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: orders-api
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: orders-api-from-load-balancer
  namespace: orders
spec:
  podSelector:
    matchLabels:
      app: orders-api
  policyTypes: ["Ingress"]
  ingress:
    - from:
        - ipBlock:
            cidr: 130.211.0.0/22
        - ipBlock:
            cidr: 35.191.0.0/16
      ports:
        - port: 8080
EOF
kubectl -n orders rollout status deployment/orders-api

# Later: which versions does the Regular channel offer for this cluster?
gcloud container get-server-config --region=$REGION --format="yaml(channels)"

# Placeholders: pass in your project, project number and the range that may reach the control plane.
variable "project_id" {
  type = string
}

variable "project_number" {
  type = string
}

variable "office_cidr" {
  type    = string
  default = "203.0.113.0/24"
}

variable "region" {
  type    = string
  default = "asia-south1"
}

variable "receipt_bucket" {
  type    = string
  default = "orders-receipts-example"
}

resource "google_compute_network" "orders" {
  name                    = "orders-net"
  auto_create_subnetworks = false
}

resource "google_compute_subnetwork" "nodes" {
  name                     = "orders-nodes"
  region                   = var.region
  network                  = google_compute_network.orders.id
  ip_cidr_range            = "10.10.0.0/20"
  private_ip_google_access = true

  secondary_ip_range {
    range_name    = "pods"
    ip_cidr_range = "10.64.0.0/14"
  }

  secondary_ip_range {
    range_name    = "services"
    ip_cidr_range = "10.68.0.0/20"
  }
}

resource "google_compute_router" "orders" {
  name    = "orders-router"
  region  = var.region
  network = google_compute_network.orders.id
}

resource "google_compute_router_nat" "orders" {
  name                               = "orders-nat"
  router                             = google_compute_router.orders.name
  region                             = var.region
  nat_ip_allocate_option             = "AUTO_ONLY"
  source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES"
}

resource "google_container_cluster" "orders" {
  name                = "orders-prod"
  location            = var.region
  enable_autopilot    = true
  deletion_protection = true

  network    = google_compute_network.orders.id
  subnetwork = google_compute_subnetwork.nodes.id

  ip_allocation_policy {
    cluster_secondary_range_name  = "pods"
    services_secondary_range_name = "services"
  }

  private_cluster_config {
    enable_private_nodes    = true
    enable_private_endpoint = false
  }

  master_authorized_networks_config {
    cidr_blocks {
      cidr_block   = var.office_cidr
      display_name = "office"
    }
  }

  release_channel {
    channel = "REGULAR"
  }

  binary_authorization {
    evaluation_mode = "PROJECT_SINGLETON_POLICY_ENFORCE"
  }

  maintenance_policy {
    recurring_window {
      start_time = "2026-11-07T21:00:00Z"
      end_time   = "2026-11-08T01:00:00Z"
      recurrence = "FREQ=WEEKLY;BYDAY=SA"
    }
  }
}

# Workload Identity Federation: the role goes to the pod's principal, no service account and no key.
resource "google_storage_bucket_iam_member" "orders_api" {
  bucket = var.receipt_bucket
  role   = "roles/storage.objectCreator"
  member = "principal://iam.googleapis.com/projects/${var.project_number}/locations/global/workloadIdentityPools/${var.project_id}.svc.id.goog/subject/ns/orders/sa/orders-api"
}

output "cluster_name" {
  value = google_container_cluster.orders.name
}

# Runs inside the orders-api pod. With Workload Identity Federation the metadata server hands the pod short-lived
# credentials, and Application Default Credentials find them. There is no key file and nothing to mount.
import json
import os
import time

from google.api_core import retry
from google.api_core.exceptions import PreconditionFailed
from google.cloud import storage

_client = storage.Client()    # picks the pod's identity up on its own
_bucket = _client.bucket(os.environ["RECEIPT_BUCKET"])


def write_receipt(order_id, total):
    blob = _bucket.blob("receipts/%s.json" % order_id)
    doc = json.dumps({"order": order_id, "total": total, "saved_at": int(time.time())})
    try:
        # if_generation_match=0 means "only if it doesn't exist yet", so a retry can't overwrite a receipt
        blob.upload_from_string(doc, content_type="application/json", if_generation_match=0,
                                retry=retry.Retry(initial=0.5, maximum=8.0, timeout=30.0))
    except PreconditionFailed:
        return "gs://%s/%s (already there)" % (_bucket.name, blob.name)
    return "gs://%s/%s" % (_bucket.name, blob.name)


if __name__ == "__main__":
    print(write_receipt("A-1042", 1249.50))

// dotnet add package KubernetesClient
// Lists pods that keep restarting or were killed for running out of memory, across all namespaces.
// The kubeconfig written by "gcloud container clusters get-credentials" calls gke-gcloud-auth-plugin for the token.
using k8s;
using k8s.Models;

var client = new Kubernetes(KubernetesClientConfiguration.BuildConfigFromConfigFile());
V1PodList all = await client.CoreV1.ListPodForAllNamespacesAsync();

var noisy = from pod in all.Items
            from cs in pod.Status.ContainerStatuses ?? new List<V1ContainerStatus>()
            let oom = cs.LastState?.Terminated?.Reason == "OOMKilled"
            where cs.RestartCount >= 3 || oom
            orderby cs.RestartCount descending
            select new { Ns = pod.Metadata.NamespaceProperty, Pod = pod.Metadata.Name, Container = cs.Name, cs.RestartCount, oom };

int count = 0;
foreach (var row in noisy)
{
    count++;
    string hint = row.oom ? "  <- raise the memory request" : "";
    Console.WriteLine($"{row.Ns}/{row.Pod} [{row.Container}] restarts={row.RestartCount}{hint}");
}

Console.WriteLine(count == 0 ? "No pod has restarted three times or been OOM killed." : $"{count} container(s) need a look.");
return count == 0 ? 0 : 1;

Short walkthrough videos for this service are being prepared. They will appear here and load only when you press play. Until then, follow CloudBrum on YouTube for new videos.

If you only need to run a stateless container, look at Cloud Run before you build a cluster. For event-driven code with no servers use Cloud Run functions. The data for a cluster's applications usually sits outside it, in Cloud SQL or Cloud Storage.