Skip to main content
Your values.yaml holds your overrides to the chart and Terraform module’s defaults. Keep it alongside the module’s values file; store credentials in Kubernetes Secrets.

Apply changes

Upgrade with the same files:
Agent hosts read their configuration and image when they start, and the controller replaces them for you. Within about five minutes of a change that reaches them, such as the GitHub App, your hostname, the database or a new agent host image, it marks the running hosts Drifted and replaces them, as many at a time as the pool’s disruption budget allows (one by default):
  1. The pool launches a replacement while the old host keeps taking sessions.
  2. Once the replacement is ready, the old host is cordoned: it keeps serving the sessions already on it, but new sessions go to other hosts.
  3. Once the old host has been idle for the pool’s idle period (10 minutes by default), the controller drains and terminates it.
The replacement launches alongside the old host only if the pool’s maxHosts leaves room; otherwise the old host is cordoned right away and replaced once it’s gone. The controller drains each host before terminating it: running agent turns finish, and session snapshots upload to object storage. Stopped sessions then resume on another host from object storage, without the processes that were running. A drain still going after the pool’s drainTimeout (105 minutes by default) is marked DrainStalled. To keep a host from being replaced, annotate it with hosts.tembo.io/do-not-disrupt=true. To replace a host sooner, delete it:
To terminate a host without draining it, annotate it as well. This ends the sessions running on it:

License key

The license key also pulls Tembo’s images from registry.tembo.io, and gets your AWS account access to the agent host image. To keep the key out of your values, put it in a Kubernetes Secret and reference it:
If you render the chart with helm template (for example with Argo CD or Flux), the chart can’t read that Secret: add a docker-registry Secret for registry.tembo.io, with any username and the license key as the password, to image.pullSecrets.

Ingress

The chart serves Tembo through one Ingress at ingress.host, over HTTPS. On AWS, the load balancer uses your ACM certificate for the host. On GKE, ingress.gcp.enabled gets a Google-managed certificate instead; to serve on a reserved IP, add a kubernetes.io/ingress.global-static-ip-name annotation. With another Ingress controller, set ingress.tls.secretName to a TLS Secret. Computer previews get an Ingress of their own; see Networking. ingress.aws and ingress.gcp share these settings (chart 1.0.0 and later): The load balancer is internet-facing, so GitHub and other providers can deliver webhooks. To keep Tembo on your network, make it internal:
On EKS Auto Mode, set scheme: internal in the IngressClassParams instead. On GKE, set ingress.gcp.internal. An internal GKE Ingress can’t use a Google-managed certificate or the HTTPS redirect, so give it a TLS Secret for your hostname:
An internal GKE Ingress also needs a proxy-only subnet in the cluster’s region (purpose REGIONAL_MANAGED_PROXY) and a firewall rule letting that subnet reach the cluster’s pods.

Public API host

A second, internet-facing Ingress can serve only the routes providers call, so the rest of Tembo stays on your network:
It serves webhooks, Slack interactivity, OIDC discovery for AWS integrations, MCP client metadata and the images in pull request comments, and returns 404 for anything else. Give its host DNS and a certificate; see Networking.
  • Provider settings: Tembo registers webhooks at the public host. Where a provider’s page asks for a webhook or event URL (the GitHub App, Slack, Linear, Sentry, Teams), use https://api.tembo.example.com/api/.... OAuth callback URLs stay on ingress.host. For an existing GitHub App, change its webhook URL and check Active.
  • EKS Auto Mode: the IngressClassParams decide the scheme, so set ingress.public.className to an internet-facing class.
  • GKE: the public Ingress is external, with a Google-managed certificate for its host.
  • API and MCP from outside your network: for CI or hosted MCP clients, add /api/public-api/ and /mcp to ingress.public.paths.
  • AWS integrations with OIDC: the token issuer moves to the public host. Recreate the IAM OIDC provider, and the roles’ trust policies, for it.
Without it, on an internal network, uncheck Active on the GitHub App: comments that mention Tembo, issue and pull request triggers, and pull request status updates then don’t work.

Postgres and Redis

By default, the chart runs Postgres and Redis in your cluster. Agent hosts reach them through internal load balancers at postgres.<your-host> and redis.<your-host>; set postgresql.hostAccess.hostname or redis.hostAccess.hostname to use other names. Keep these load balancers internal and use private DNS. Configure source restrictions in Terraform using the selected host subnets, rather than copying CIDRs into values.yaml; the GCP module example derives them from the module’s host_subnetwork_cidr output. Older charts shipped Redis without authentication; current charts generate a Redis password. The bundled Postgres also uses a generated password. Passwords do not replace network restrictions, and the bundled services do not provide TLS. The bundled data stores are for evaluation. For production, use managed services, such as Amazon RDS and ElastiCache or Cloud SQL and Memorystore, for backups and high availability:
  • Agent hosts connect to them directly. On AWS, add host_access_rules to the Terraform module for their security groups.
  • Each agent host holds about 50 database connections. Set the database’s max_connections to about 50 per host, plus 50.
  • If Redis needs a password, set redis.password under config.
  • redis.external.tls is off by default. The bundled Redis never uses TLS.

Bundled Postgres credentials

The chart stores both postgres-password and DATABASE_URL in <release>-conn. Changing the Secret does not change the password of an existing Postgres database. Changing only postgres-password leaves the URL stale, and the pre-upgrade migrations job can fail before the platform rolls out. There is no chart value to rotate the bundled password. Before a manual rotation, back up the database and plan a maintenance window: the database role password, both Secret keys, and all clients must agree. Contact support for help with an existing install, or use a managed database’s rotation procedure for production. Treat connection URLs as credentials when sharing logs.

Agent hosts

The module’s Helm values create a host class and a pool, both named default. The pool keeps one host running, adds hosts when less than 32 GiB of memory is free, and runs up to three. To change it:
To scale to zero, set both minHosts and minIdleMemory to zero:
With the default minIdleMemory: 32Gi, the pool keeps capacity running even when minHosts is zero. Hosts are removed after their session VMs stop and the idle period passes; the next session waits for a host to start. See Agent hosts for how pools scale. Scaling hosts to zero does not stop the cluster, data stores, NAT or load balancers. Those resources can still incur charges while no sessions run.

Instance types

Set the module’s instance_types on AWS, or machine_types on GCP. Agent hosts need nested virtualization, and must fit the largest sandbox size you use: its vCPUs plus 1, and its memory plus about 8 GiB.
  • Use local NVMe (the d in c8id; local SSDs on GCP, local_ssd_count). Hosts keep VM disks and snapshots on it; on network disks, stopping and forking sessions is much slower.
  • On GCP, use Intel machine types (N2, C3): Compute Engine supports nested virtualization only on Intel CPUs.
  • Use about 1 vCPU per 2 GiB. Pools add hosts based on memory, so hosts should run out of memory before CPU.
  • Use one processor generation, such as only c8id and m8id. Sessions resume from saved memory only on a matching CPU.

Overallocation

By default, hosts admit sandbox VMs without overallocating CPU or memory. To allow more allocated vCPUs on a host whose sessions spend much of their time idle:
These settings do not add physical resources or change a VM’s size. Increase CPU overallocation gradually while watching CPU pressure and session latency. Keep memory at 1 until you have validated your workloads under load; raising it increases memory pressure. Each VM must still fit the host’s physical capacity. Hosts read these settings at boot. Apply the values and let the controller replace existing hosts before expecting them to take effect (see Apply changes).

Warm VMs

For projects with Warm pool on, each host keeps up to 2 warm VMs per size, and 1 for large, xl and xxl. To change a size, or turn it off with 0:

Sign-in and registration

Email and password registration and login are enabled by default. Configure Google sign-in or GitHub sign-in before inviting users. These settings apply to your whole install, on any cloud. Set these keys under config in your Helm values: These settings are not an email-domain allowlist or invite-only mode. Restrict access at your network or identity provider before making the install available to untrusted users. For a Google Workspace-only install, see Restrict sign-in to Google. If existing email/password users get account_not_linked when switching to OAuth, keep their working login available and contact support. Self-hosted email/password accounts are not email-verified automatically, which can prevent automatic linking; do not disable their login until they can sign in with the new provider.

Model providers

Each organization connects its own model providers in the Tembo app. There’s no key to get from Tembo and nothing to set in the chart.
  1. As an organization admin, open Settings → Models → API Keys.
  2. Add credentials for a provider. When you save them, Tembo checks them with the provider and tells you if they don’t work.
  3. In Settings → Models, pick the provider each model runs through.
Before the first session, configure credentials for its selected agent and model. An Anthropic key does not authorize an OpenAI model, or vice versa. Missing credentials can surface as a provider 401 after a session starts. Internal prompts, such as session titles and fast helpers, also need credentials. In Tembo 0.2.895, these use Claude models through the organization’s Bedrock, Vertex AI, OpenRouter or Anthropic API credentials. An OpenAI key alone does not cover them. Verify that a first session completes a turn and gets a generated title. Your cluster needs outbound HTTPS to the providers you use.

LiteLLM

Requests that use AWS Bedrock, GCP Vertex AI or OpenRouter go through LiteLLM, which the chart runs in your cluster. That includes internal prompts when they use one of those providers. LiteLLM holds no provider keys: each request carries the organization’s credentials. Requests using an organization’s direct Anthropic, OpenAI or xAI credentials go to the provider directly. Keep LiteLLM enabled unless you have verified that all session models and internal prompts use direct providers. Disabling it does not supply missing credentials or make other providers take over. Older releases can also send internal model aliases, including tembo-fast, to LiteLLM without a direct fallback; after changing this setting, check session titles and helper features as well as agent turns.
The API authenticates to LiteLLM with a master key. The chart generates it on install and keeps it in the <release>-litellm Secret, so you don’t need to create one. To choose it yourself, set litellm.masterKey under config, or in a Secret listed under configSecrets. If you render the chart with helm template (for example with Argo CD), the chart can’t read your cluster: a generated key would change on every render, and configSecrets can’t be read. Set the key under config instead, from a values file you keep encrypted:

Cluster resources

The chart sets CPU and memory requests for its components. Raise <component>.resources.requests for heavier use; for example, api.resources and litellm.resources configure those pods. Size cluster nodes for the sum of requests plus Kubernetes system pods and upgrade headroom. These cluster resources are separate from the agent-host capacity described above. Keep the API and bundled data stores on reliable capacity. Spot-only clusters can lose the whole install during a preemption, and an on-demand baseline plus Spot overflow does not itself guarantee which pods run on which nodes.

Node autoscalers

Each Tembo component runs as a single pod. A node autoscaler that evicts one to consolidate nodes takes that component down until its replacement starts, which can take a few minutes for the API. Annotate the pods so your autoscaler leaves them in place; upgrades and node drains still replace them. On EKS Auto Mode or Karpenter:
On GKE, or with the Cluster Autoscaler:
Nodes running these pods aren’t consolidated, so on EKS your cluster may keep a node it could otherwise remove. GKE Autopilot bills by pod requests, so it doesn’t cost more there.

Object storage

The module creates a bucket (S3 on AWS, GCS on GCP) and sets objectStorage in its Helm values. To use a bucket you already have, set the module’s object_storage_bucket_name, and allow your Tembo URL in its CORS. On GCP, Tembo signs upload and download links with its service account through IAM, which needs roles/iam.serviceAccountTokenCreator on that service account; the module grants it. Without object storage (object_storage_enabled = false), project builds and file uploads are off.

Integrations

  • GitHub: see GitHub.
  • Others, such as Slack or Linear: set their config keys under config, or keep them in Secrets listed under configSecrets.

Usage telemetry

From chart 1.1.0, Tembo sends usage counts to Tembo’s PostHog at us.i.posthog.com: pull requests opened and merged, sessions, automation runs and integrations, tagged with your install’s hostname. Events carry actions, statuses, counts and IDs only, never repository names, URLs, titles, code or prompts. It’s on by default. To turn it off, or if your cluster has no outbound access to us.i.posthog.com:

Need Help?

If you run into any issues, contact support@tembo.io.