Skip to main content
On GCP, Tembo runs on GKE and launches agent hosts as Compute Engine instances in your VPC. Prepare the projects, cluster and network first, then apply the tembo-agent-host module for Google Cloud, and use its outputs to configure the chart.

Before you run Terraform

Starting from an empty project? The module’s complete example creates everything below: a VPC with a subnetwork for hosts, Cloud NAT with dynamic port allocation, and a private GKE Autopilot cluster that hosts can reach.

Projects and APIs

  • A project for agent hosts. The host controller can create instances anywhere in the hosts’ project, and change or delete only its own. We recommend a project of its own for agent hosts: set it as the module’s host_project_id. Otherwise hosts run in the cluster’s project.
  • APIs enabled in those projects: Compute Engine, IAM, IAM Service Account Credentials and Cloud Storage (compute.googleapis.com, iam.googleapis.com, iamcredentials.googleapis.com, storage.googleapis.com).
If gcloud reports RESOURCE_EXHAUSTED for Resource Manager reads against a shared consumer project, use a project you control for CLI quota. Enable the APIs there before setting the quota project:
You need serviceusage.services.use on that project. This changes API request quota attribution, not Compute Engine CPU or disk quotas. Newly enabled APIs can take a short time to become usable; retry a SERVICE_DISABLED response after propagation.

Cluster

  • A VPC-native GKE cluster (Autopilot or Standard) with Workload Identity. Autopilot clusters always have it.
  • Your access to the control plane. Check authorized networks on a new cluster: an empty allowlist can leave kubectl hanging, and enforcement on the private endpoint can also block agent hosts. Use the DNS endpoint for administrators and authorize the hosts’ subnet for the private endpoint (see below).
  • Ingress: the chart serves Tembo through GKE’s Ingress with a Google-managed certificate (ingress.gcp.enabled), so you don’t need a certificate in advance. GKE issues it once your hostname resolves to the Ingress.

VPC and subnetwork

Agent hosts launch into a subnetwork you choose, in the cluster’s VPC:
  • Cloud NAT and Private Google Access. Hosts have no external addresses. They need NAT for egress to your Tembo URL, Tembo’s API, LLM providers and code hosts such as GitHub, and Private Google Access for Cloud Storage and IAM.
  • Dynamic port allocation on the NAT, with at least 1,024 ports per VM. A host carries the egress of every session on it, and Cloud NAT’s default of 64 ports per VM runs out with a couple of sessions:
  • The cluster’s control plane reachable from it. Hosts read their configuration through the Kubernetes API, at the cluster’s private endpoint. If the cluster enforces master authorized networks on its private endpoint, add the subnetwork’s range to them. Hosts in a different region than the cluster need the control plane’s global access.
  • Your own Postgres or Redis, if you use them instead of the chart’s, must accept connections from the subnetwork. Hosts reach the chart’s bundled Postgres and Redis through internal load balancers.
For an existing cluster, enable DNS access for administrators and fetch credentials for that endpoint:
Your administrator needs container.clusters.connect permission and the gke-gcloud-auth-plugin installed. Agent hosts still use the private IP endpoint. If it enforces authorized networks, add their subnet’s CIDR:
This flag replaces the allowlist: include all existing node, pod and administrator ranges that must retain access. If there are no existing ranges, supply just the hosts’ subnet CIDR. Use the cluster’s zone for a zonal cluster or region for a regional cluster. Keep these settings in your Terraform configuration if it manages the cluster. The module reads the subnetwork and doesn’t change it. It creates the firewall rule that lets the cluster’s pods and nodes reach the hosts’ manager on port 8743; for a Shared VPC subnetwork, it creates it in the network’s host project.

Organization policies

  • compute.disableNestedVirtualization not enforced on the hosts’ project: hosts run agents in VMs.
  • compute.requireShieldedVm not enforced: the agent host image boots with legacy BIOS.
  • If you restrict compute.trustedImageProjects, allow Tembo’s image project, projects/tembo-prod.

Quotas

Each host needs 32 vCPUs and 1,500 GB of local SSD, times the pool’s maximum hosts (three by default). Free Trial billing accounts can’t run a single host or request more quota: upgrade to a paid account first.
  • N2 CPUs and local SSD in the hosts’ region. The defaults often cover a few hosts.
  • Persistent Disk SSD in the hosts’ region: each host’s 100 GB boot disk counts toward it, as do GKE’s node disks. New-project quotas can be as low as 250 GB.
  • CPUs (all regions), a project-wide limit that also counts the cluster’s nodes. New-project quotas can be as low as 12 vCPUs, which isn’t enough for a single host.
Check your actual limits and request increases in Quotas & System Limits. Approval is not guaranteed even after upgrading billing; check the console for the reason if a CLI request only says Quota request denied. Start with one host on a quota-limited project. Add this to the values.yaml you pass after the module’s values:
Reserve quota for both GKE and agent hosts. For example, a 48-vCPU project can fit one 32-vCPU host plus a Standard zonal cluster capped at three e2-standard-4 nodes (12 vCPUs). Allow headroom for node upgrades too. Autopilot scales its nodes automatically and also consumes project quota; use Standard with a capped node pool when you need a predictable CPU budget. A regional node pool’s size applies per zone. Raise maxHosts only after quota increases are approved. Hosts beyond your quota don’t launch; replacing a host without interrupting sessions also needs room for the replacement (see Apply changes).

Permissions to apply

Apply the module as the projects’ Owner, or with these roles, besides read access to the cluster (roles/container.clusterViewer): Editor isn’t enough: it can’t create the controller’s custom roles or set IAM on the subnetwork.

Install the module

It needs Google provider 6.0 or later. The module creates:
  • The host controller’s service account, bound to the chart’s service account with Workload Identity. It can change and delete only instances and disks named with your install’s prefix, launch them only into your subnetwork, and attach only the hosts’ service account.
  • The agent hosts’ service account, with access to the install’s bucket.
  • The firewall rule for the hosts’ manager.
  • A GCS bucket for project images, session snapshots and file uploads, and the control plane’s service account to use it. To use a bucket you have, set object_storage_bucket_name.
Set kubernetes_namespace and release_name to the namespace and release you’ll install the chart with: the service accounts trust only the chart’s service accounts in that namespace. See the module’s README for the other options, such as machine types. New IAM bindings can take a few minutes to take effect; pods that start before then retry.

Configure the chart with its outputs

Save the Helm values, including the source ranges derived from your host subnet in the output above:
They configure the chart for the resources the module created:
  • GKE’s Ingress for the chart (ingress.gcp).
  • The host controller, with its service account, the hosts’ project and your install’s instance name prefix.
  • A host class named default, with your subnetwork, the hosts’ service account, n2-custom-32-65536 with four local SSDs, and Tembo’s stable agent host image.
  • A pool named default, of one to three hosts.
  • Object storage, with the bucket and the control plane’s service account.
Deploy with tembo.json first and your own values.yaml after it, so yours override them. To change the agent hosts, such as their machine type, see Agent hosts; to use managed Postgres or Redis, see Postgres and Redis. Next, set up GitHub.

DNS

Use Cloud DNS private zones attached to the hosts’ VPC for postgres.<your-host>, redis.<your-host> and sandbar.<your-host>. A zone for each internal hostname avoids shadowing the public app and preview names. With Shared VPC, attach the zones to the shared network and grant the DNS administrator access in the zone’s project. Configure external-dns for Google Cloud DNS to watch Services and manage these zones. The chart already annotates each internal Service with its hostname, so external-dns creates and updates the records when GKE assigns load balancer addresses. Keep public and private zone filters separate. The Terraform module currently creates neither these zones nor external-dns; use your existing DNS automation where available. Without external-dns, create private records from the Service addresses shown by kubectl -n tembo get svc. Public app and preview records can stay with your existing DNS provider.

Ingress and certificates

We use GKE’s built-in Ingress, external-dns and cert-manager, with a Let’s Encrypt ClusterIssuer that solves DNS-01 in Cloud DNS.
  • Load balancers: ingress.gcp sets up both Ingresses for GKE, with health checks and the HTTPS redirect. To serve on a reserved IP, add a kubernetes.io/ingress.global-static-ip-name annotation.
  • Certificates: GKE gives tembo.example.com a Google-managed certificate, and issues one for ingress.public.host when you enable the public Ingress. Those can’t be wildcards, so cert-manager issues *.previews.tembo.example.com into tembo-previews-tls. Use your ClusterIssuer’s name in the annotation.

Sign in with Google

Google sign-in works on any cloud. You need your Tembo hostname, permission to create a Google OAuth client, and access to the tembo Kubernetes namespace. Create the namespace first if you have not deployed yet.
  1. In the Google Cloud console, open Google Auth Platform and set up your app. Choose Internal under Audience to allow only your Google Workspace’s users.
  2. Under Clients, create a client of type Web application, with the authorized redirect URI https://<your-host>/api/auth/callback/google.
  3. Save the client secret when Google first shows it, then store it with the client ID in a Secret:
  4. Add it to your values.yaml, then install, or upgrade if Tembo is already installed (see Apply changes):

Restrict sign-in to Google

To use only Google sign-in, first verify Google login works, then set:
This disables email/password login and other OAuth providers, including GitHub account linking. For a Google Workspace-only install, also choose an Internal Google OAuth audience. See Sign-in and registration for shared settings and account-linking troubleshooting.

Need Help?

If you run into any issues, contact support@tembo.io.