values.yaml holds your overrides to the chart and Terraform module’s defaults. Keep it alongside the module’s values file; store credentials in Kubernetes Secrets.
Apply changes
Upgrade with the same files:Drifted and replaces them, as many at a time as the pool’s disruption budget allows (one by default):
- The pool launches a replacement while the old host keeps taking sessions.
- Once the replacement is ready, the old host is cordoned: it keeps serving the sessions already on it, but new sessions go to other hosts.
- Once the old host has been idle for the pool’s idle period (10 minutes by default), the controller drains and terminates it.
maxHosts leaves room; otherwise the old host is cordoned right away and replaced once it’s gone.
The controller drains each host before terminating it: running agent turns finish, and session snapshots upload to object storage. Stopped sessions then resume on another host from object storage, without the processes that were running. A drain still going after the pool’s drainTimeout (105 minutes by default) is marked DrainStalled.
To keep a host from being replaced, annotate it with hosts.tembo.io/do-not-disrupt=true. To replace a host sooner, delete it:
License key
The license key also pulls Tembo’s images fromregistry.tembo.io, and gets your AWS account access to the agent host image. To keep the key out of your values, put it in a Kubernetes Secret and reference it:
helm template (for example with Argo CD or Flux), the chart can’t read that Secret: add a docker-registry Secret for registry.tembo.io, with any username and the license key as the password, to image.pullSecrets.
Ingress
The chart serves Tembo through one Ingress atingress.host, over HTTPS. On AWS, the load balancer uses your ACM certificate for the host. On GKE, ingress.gcp.enabled gets a Google-managed certificate instead; to serve on a reserved IP, add a kubernetes.io/ingress.global-static-ip-name annotation. With another Ingress controller, set ingress.tls.secretName to a TLS Secret. Computer previews get an Ingress of their own; see Networking.
ingress.aws and ingress.gcp share these settings (chart 1.0.0 and later):
The load balancer is internet-facing, so GitHub and other providers can deliver webhooks. To keep Tembo on your network, make it internal:
scheme: internal in the IngressClassParams instead.
On GKE, set ingress.gcp.internal. An internal GKE Ingress can’t use a Google-managed certificate or the HTTPS redirect, so give it a TLS Secret for your hostname:
REGIONAL_MANAGED_PROXY) and a firewall rule letting that subnet reach the cluster’s pods.
Public API host
A second, internet-facing Ingress can serve only the routes providers call, so the rest of Tembo stays on your network:- Provider settings: Tembo registers webhooks at the public host. Where a provider’s page asks for a webhook or event URL (the GitHub App, Slack, Linear, Sentry, Teams), use
https://api.tembo.example.com/api/.... OAuth callback URLs stay oningress.host. For an existing GitHub App, change its webhook URL and check Active. - EKS Auto Mode: the IngressClassParams decide the scheme, so set
ingress.public.classNameto an internet-facing class. - GKE: the public Ingress is external, with a Google-managed certificate for its host.
- API and MCP from outside your network: for CI or hosted MCP clients, add
/api/public-api/and/mcptoingress.public.paths. - AWS integrations with OIDC: the token issuer moves to the public host. Recreate the IAM OIDC provider, and the roles’ trust policies, for it.
Postgres and Redis
By default, the chart runs Postgres and Redis in your cluster. Agent hosts reach them through internal load balancers atpostgres.<your-host> and redis.<your-host>; set postgresql.hostAccess.hostname or redis.hostAccess.hostname to use other names.
Keep these load balancers internal and use private DNS. Configure source restrictions in Terraform using the selected host subnets, rather than copying CIDRs into values.yaml; the GCP module example derives them from the module’s host_subnetwork_cidr output.
Older charts shipped Redis without authentication; current charts generate a Redis password. The bundled Postgres also uses a generated password. Passwords do not replace network restrictions, and the bundled services do not provide TLS.
The bundled data stores are for evaluation. For production, use managed services, such as Amazon RDS and ElastiCache or Cloud SQL and Memorystore, for backups and high availability:
- Agent hosts connect to them directly. On AWS, add
host_access_rulesto the Terraform module for their security groups. - Each agent host holds about 50 database connections. Set the database’s
max_connectionsto about 50 per host, plus 50. - If Redis needs a password, set
redis.passwordunderconfig. redis.external.tlsis off by default. The bundled Redis never uses TLS.
Bundled Postgres credentials
The chart stores bothpostgres-password and DATABASE_URL in <release>-conn. Changing the Secret does not change the password of an existing Postgres database. Changing only postgres-password leaves the URL stale, and the pre-upgrade migrations job can fail before the platform rolls out.
There is no chart value to rotate the bundled password. Before a manual rotation, back up the database and plan a maintenance window: the database role password, both Secret keys, and all clients must agree. Contact support for help with an existing install, or use a managed database’s rotation procedure for production. Treat connection URLs as credentials when sharing logs.
Agent hosts
The module’s Helm values create a host class and a pool, both nameddefault. The pool keeps one host running, adds hosts when less than 32 GiB of memory is free, and runs up to three. To change it:
minHosts and minIdleMemory to zero:
minIdleMemory: 32Gi, the pool keeps capacity running even when minHosts is zero. Hosts are removed after their session VMs stop and the idle period passes; the next session waits for a host to start. See Agent hosts for how pools scale.
Scaling hosts to zero does not stop the cluster, data stores, NAT or load balancers. Those resources can still incur charges while no sessions run.
Instance types
Set the module’sinstance_types on AWS, or machine_types on GCP. Agent hosts need nested virtualization, and must fit the largest sandbox size you use: its vCPUs plus 1, and its memory plus about 8 GiB.
- Use local NVMe (the
dinc8id; local SSDs on GCP,local_ssd_count). Hosts keep VM disks and snapshots on it; on network disks, stopping and forking sessions is much slower. - On GCP, use Intel machine types (N2, C3): Compute Engine supports nested virtualization only on Intel CPUs.
- Use about 1 vCPU per 2 GiB. Pools add hosts based on memory, so hosts should run out of memory before CPU.
- Use one processor generation, such as only
c8idandm8id. Sessions resume from saved memory only on a matching CPU.
Overallocation
By default, hosts admit sandbox VMs without overallocating CPU or memory. To allow more allocated vCPUs on a host whose sessions spend much of their time idle:
These settings do not add physical resources or change a VM’s size. Increase CPU overallocation gradually while watching CPU pressure and session latency. Keep memory at
1 until you have validated your workloads under load; raising it increases memory pressure. Each VM must still fit the host’s physical capacity.
Hosts read these settings at boot. Apply the values and let the controller replace existing hosts before expecting them to take effect (see Apply changes).
Warm VMs
For projects with Warm pool on, each host keeps up to 2 warm VMs per size, and 1 forlarge, xl and xxl. To change a size, or turn it off with 0:
Sign-in and registration
Email and password registration and login are enabled by default. Configure Google sign-in or GitHub sign-in before inviting users. These settings apply to your whole install, on any cloud. Set these keys underconfig in your Helm values:
These settings are not an email-domain allowlist or invite-only mode. Restrict access at your network or identity provider before making the install available to untrusted users. For a Google Workspace-only install, see Restrict sign-in to Google.
If existing email/password users get
account_not_linked when switching to OAuth, keep their working login available and contact support. Self-hosted email/password accounts are not email-verified automatically, which can prevent automatic linking; do not disable their login until they can sign in with the new provider.
Model providers
Each organization connects its own model providers in the Tembo app. There’s no key to get from Tembo and nothing to set in the chart.- As an organization admin, open Settings → Models → API Keys.
-
Add credentials for a provider. When you save them, Tembo checks them with the provider and tells you if they don’t work.
- In Settings → Models, pick the provider each model runs through.
401 after a session starts.
Internal prompts, such as session titles and fast helpers, also need credentials. In Tembo 0.2.895, these use Claude models through the organization’s Bedrock, Vertex AI, OpenRouter or Anthropic API credentials. An OpenAI key alone does not cover them. Verify that a first session completes a turn and gets a generated title.
Your cluster needs outbound HTTPS to the providers you use.
LiteLLM
Requests that use AWS Bedrock, GCP Vertex AI or OpenRouter go through LiteLLM, which the chart runs in your cluster. That includes internal prompts when they use one of those providers. LiteLLM holds no provider keys: each request carries the organization’s credentials. Requests using an organization’s direct Anthropic, OpenAI or xAI credentials go to the provider directly. Keep LiteLLM enabled unless you have verified that all session models and internal prompts use direct providers. Disabling it does not supply missing credentials or make other providers take over. Older releases can also send internal model aliases, includingtembo-fast, to LiteLLM without a direct fallback; after changing this setting, check session titles and helper features as well as agent turns.
<release>-litellm Secret, so you don’t need to create one. To choose it yourself, set litellm.masterKey under config, or in a Secret listed under configSecrets.
If you render the chart with helm template (for example with Argo CD), the chart can’t read your cluster: a generated key would change on every render, and configSecrets can’t be read. Set the key under config instead, from a values file you keep encrypted:
Cluster resources
The chart sets CPU and memory requests for its components. Raise<component>.resources.requests for heavier use; for example, api.resources and litellm.resources configure those pods. Size cluster nodes for the sum of requests plus Kubernetes system pods and upgrade headroom. These cluster resources are separate from the agent-host capacity described above.
Keep the API and bundled data stores on reliable capacity. Spot-only clusters can lose the whole install during a preemption, and an on-demand baseline plus Spot overflow does not itself guarantee which pods run on which nodes.
Node autoscalers
Each Tembo component runs as a single pod. A node autoscaler that evicts one to consolidate nodes takes that component down until its replacement starts, which can take a few minutes for the API. Annotate the pods so your autoscaler leaves them in place; upgrades and node drains still replace them. On EKS Auto Mode or Karpenter:Object storage
The module creates a bucket (S3 on AWS, GCS on GCP) and setsobjectStorage in its Helm values. To use a bucket you already have, set the module’s object_storage_bucket_name, and allow your Tembo URL in its CORS. On GCP, Tembo signs upload and download links with its service account through IAM, which needs roles/iam.serviceAccountTokenCreator on that service account; the module grants it. Without object storage (object_storage_enabled = false), project builds and file uploads are off.
Integrations
- GitHub: see GitHub.
- Others, such as Slack or Linear: set their config keys under
config, or keep them in Secrets listed underconfigSecrets.
Usage telemetry
From chart 1.1.0, Tembo sends usage counts to Tembo’s PostHog atus.i.posthog.com: pull requests opened and merged, sessions, automation runs and integrations, tagged with your install’s hostname. Events carry actions, statuses, counts and IDs only, never repository names, URLs, titles, code or prompts.
It’s on by default. To turn it off, or if your cluster has no outbound access to us.i.posthog.com: