writing-skills
writing-skills
Use when creating new skills, editing existing skills, or verifying skills work before deployment
mirrord-operatorHelp users install and configure the mirrord Operator for team/enterprise environments. Use when users ask about operator setup, Helm installation, cloud API key or license configuration, air-gapped/offline licensing, enabling features (queue splitting, DB branching, preview environments, multi-cluster), internal registries, OpenShift/GKE Autopilot, RBAC, o…
Independent GitHub snapshot · Automated classification · Original documentation governs
Help users install and operate the mirrord Operator — the persistent Kubernetes control plane that enables all mirrord Team / Enterprise features:
In open-source mirrord each session is standalone (the CLI creates a privileged agent pod directly). The Operator centralizes this:
Trigger on questions like:
Feature X requires using mirrord operator or Operator not found on a cluster that has no mirrord for Teams licenseInstalling the operator modifies a shared cluster. Treat every operation as high-impact.
values.yaml. No cloud API keys, license keys, tokens, or PEM contents via --set or inline in committed values. Create a Kubernetes Secret and reference it (cloud.apiKey.keyRef, license.keyRef, license.pemRef), or use Google Secret Manager refs (cloud.apiKey.gsmRef, license.keyGsmRef, license.pemGsmRef).helm install/upgrade, kubectl create/apply, and RBAC commands for review and get explicit approval before running any of them. Do not run them automatically.-f values.yaml) for all structured input.references/troubleshooting.md — common operator issues and solutionsreferences/helm-values.md — curated digest of the important chart values (auth, feature flags, agent, TLS, registry, platform)references/values.yaml — verbatim upstream chart values.yaml, kept in sync automatically. Read this for exact current defaults or any value the digest doesn't list.Authoritative upstream sources:
values.yaml: https://raw.githubusercontent.com/metalbear-co/charts/main/mirrord-operator/values.yamlkubectl cluster-info # cluster reachable
helm version # Helm 3.x
kubectl auth can-i create deployments -n mirrord # sufficient RBAC (usually cluster-admin to install)
You also need a mirrord for Teams license. Register at app.metalbear.com. If there's no license yet, an agent can start a trial for the user once they agree — see No License Yet? Agent-Started Trials.
helm repo add metalbear https://metalbear-co.github.io/charts
helm repo update
curl https://raw.githubusercontent.com/metalbear-co/charts/main/mirrord-operator/values.yaml --output values.yaml
The operator needs credentials to obtain its license. Pick one path:
A. Cloud API key (default, recommended). The operator authenticates to the mirrord cloud with a cloud API key and obtains its license over the API. Generate the key in the dashboard under Settings at app.metalbear.com — it's shown only once. When generating it, an org admin also chooses identity sharing (ticked by default): with it on, usage metrics sent to the mirrord cloud include developer usernames and session targets so the usage dashboard can show them by name; with it off, metrics stay anonymized. Set cloud.anonymizeData: true in Helm values to keep metrics anonymized regardless of the key's setting. Provide the key one of three ways:
values.yaml. Point the chart at the secret name now (this doesn't require the secret to exist yet):
cloud:
apiKey:
keyRef: mirrord-operator-cloud-api-key
Install the chart first (Step 3 — this also creates the mirrord namespace), then have the user create the Secret in it. Write the key to a file (its exact contents, no trailing newline) and create the Secret with --from-file, so the key isn't exposed in the shell's process arguments (ps) or history — then delete the file. (Avoid --from-literal=apiKey=..., which places the key in argv.)
# apikey.txt holds only the key, with no trailing newline
kubectl create secret generic mirrord-operator-cloud-api-key \
--namespace mirrord --from-file=apiKey=./apikey.txt
rm apikey.txt
The Secret's data key must be apiKey (what cloud.apiKey.keyRef expects). A stray trailing newline in the file becomes part of the key and breaks authentication. The operator pod waits for the Secret to appear and starts automatically once it's created — no restart needed. (If the user wants to create the Secret before installing instead, the mirrord namespace must already exist and be Helm-managed, or helm install will fail to adopt it.)cloud.apiKey.gsmRef: projects/PROJECT_ID/secrets/SECRET_NAME/versions/latest (read via Application Default Credentials; see sa.gcpSa).cloud.apiKey.key: <YOUR_API_KEY> (lands in the pod spec as plaintext; avoid for real clusters).Rotate/revoke from the dashboard (revocation supports a grace window so you can roll the operator to a new key).
B. License key (deprecated for cloud auth). Still valid for existing installs and required when using your own license server. Set license.key or reference a Secret via license.keyRef (Secret data key OPERATOR_LICENSE_KEY). New cloud installs should prefer the cloud API key.
C. Air-gapped / offline. See Air-gapped / offline.
helm install -f values.yaml mirrord-operator metalbear/mirrord-operator
(For upgrades, use helm upgrade --install ... -f values.yaml with the same release name.)
mirrord operator status # preferred — confirms clients can reach the operator
kubectl get pods -n mirrord # operator pod should be Running
Once installed, all mirrord clients use the Operator automatically when running against the cluster.
Most features are off by default and gated behind a Helm value under operator.*. Enable only what you need, then helm upgrade. See references/helm-values.md for the full list. Highlights:
| Feature | Helm value(s) |
|---|---|
| SQS queue splitting | operator.sqsSplitting: true |
| Kafka queue splitting | operator.kafkaSplitting: true (+ operator.kafkaSplittingSidecar.enabled for Kafka Streams / JVM clients) |
| RabbitMQ / GCP Pub/Sub / Azure Service Bus / Redis Pub/Sub / Temporal / BullMQ splitting | operator.rmqSplitting / gcpPubsubSplitting / azureServiceBusSplitting / redisPubsubSplitting / temporalSplitting / bullmqSplitting |
| DB branching (per engine) | operator.mysqlBranching, pgBranching, mariadbBranching, mongodbBranching, mssqlBranching, redisBranching, dynamodbBranching, clickhouseBranching, cockroachdbBranching, spannerBranching |
| Generic DB branching (user-supplied images) | operator.genericBranching: true (⚠️ lets branch creators run arbitrary images; restrict with genericBranchConfig.dbPod.allowedImages) |
| Preview environments | operator.previewEnv: true — default on recent chart versions, so only needs setting explicitly on older ones (+ operator.shareIngress.shareDomain and the mirrord-share-ingress chart for link sharing) |
| Multi-cluster orchestration | operator.multiCluster.enabled: true on the primary cluster; operator.multiClusterMember: true on members |
| Preview environments as multi-cluster replicas | operator.multiCluster.preview.mode: replicas on every cluster (default default-cluster; needs operator/chart 3.193.0+ and mirrord 3.247.0+) |
| Prometheus metrics | operator.metrics: true |
For the specific minimum operator/CLI/chart versions each feature needs, see the corresponding feature skill (e.g.
mirrord-db-branching,mirrord-kafka,mirrord-prev-env).
If the cluster has no mirrord for Teams license, an agent can offer the user a seven-day Enterprise trial instead of stopping. Run mirrord operator status first: if it succeeds, the cluster already runs the Operator (possibly from a trial an earlier session started), so don't sign up again. Otherwise, once the user agrees:
mirrord operator install --no-browser
This runs against the cluster of the current kubecontext, or the one named with --context. No authentication, no credit card, and no API key for the agent to handle. In a terminal, the command first asks you to confirm the kubecontext and namespace before it starts the trial or changes the cluster — --yes skips the question; an agent usually runs without a terminal, and then it isn't asked either way.
The command starts the trial, installs the Operator from the default Helm chart (no Helm needed), and prints the trial's end date, a claim_url for the user to open to take ownership of the organization, and the trial's API key. --no-browser keeps it from opening the claim page itself (it's skipped automatically whenever the command isn't running in a terminal). It sends the cluster's ID (the UID of its default namespace) as a hint so the user can recognize the cluster on the claim page; --cluster-hint <name> sends a different name, --no-hint sends none. If an Operator is already installed, or an earlier install left objects behind, the command stops and tells you rather than touching anything.
If the install fails or is stopped half-way, don't start another trial — remove what was installed and retry with the trial's key (the command prints both commands, filled in, when it fails):
mirrord operator uninstall
mirrord operator install --api-key <key>
Give both commands the same --context if the first attempt used one.
Treat the returned API key like any other secret material (see Security Boundaries) — don't echo it; if you move the install to Helm later, pass it via a Secret reference, not --set.
Confirm with the user before running mirrord operator install — it creates an external organization and installs into the cluster; treat it with the same care as any other cluster-modifying action.
Because the trial is an Enterprise license, it also covers Preview Environments. The Helm chart enables operator.previewEnv by default, so an Operator installed this way supports them as-is — no extra flag needed.
To remove an Operator that mirrord operator install set up (also after a half-finished install), run mirrord operator uninstall. It first lets the Operator end its sessions so the workloads they changed are restored, then deletes everything the install created, including the mirrord CRDs (so it also deletes the cluster's mirrord policies and profiles). Like install, it takes --context and asks for confirmation in a terminal. If the install was later taken over by Helm (via the helm install command mirrord operator install prints for anything beyond the default setup), use helm uninstall instead — mirrord operator uninstall detects this and stops rather than acting.
An agent that can't run the CLI can post to the signup endpoint directly instead:
curl -fsS -X POST https://app.metalbear.com/api/v1/agent/signup \
-H 'content-type: application/json' \
-d '{"agent": "claude-code", "developer_email": "<user email, optional>", "cluster_hint": "<optional>"}'
This endpoint needs no authentication and no credit card. The response describes the provisional organization: an api_key to use as the cloud API key in Step 2 above, and a claim_url the user has to open to take ownership of the organization — until they do, nobody can administer billing or seats, and the org expires with the trial. Treat the returned api_key like any other secret material — put it in a Kubernetes Secret, don't echo it. Confirm with the user before making this request — it creates an external organization and, if developer_email is set, sends mail to that address.
Air-gapped clusters can't reach the cloud to exchange an API key for a license, so they use an offline license certificate or a self-hosted license server.
license.file.secret.data:
license:
file:
secret:
data:
license.pem: |
-----BEGIN CERTIFICATE-----
<contents of your license.pem>
-----END CERTIFICATE-----
license.pemRef (doesn't require the secret to exist yet). Install the chart first (creates the mirrord namespace), then create the Secret in it:
kubectl create secret generic mirrord-operator-license-pem \
--namespace mirrord --from-file=license.pem=/path/to/license.pem
The operator pod waits for the Secret to appear and starts automatically once it's created — no restart needed. (To create the Secret first instead, the mirrord namespace must already exist and be Helm-managed, or helm install will fail to adopt it.)license.licenseServer: http://mirrord-operator-license-server.mirrord.svc. Here the license key is the shared secret the operator uses to authenticate to your server (a value you choose), and remains required.Air-gapped is Enterprise-only. Don't suggest the free/self-serve trial for offline clusters — the trial license needs connectivity to mirrord's telemetry endpoints.
Reduces startup time and removes the dependency on GitHub's registry. Copy the operator + agent images to your registry (multi-arch copy with regctl):
IMAGE_VERSION=$(helm show chart metalbear/mirrord-operator | grep 'appVersion:' | awk '{print $2}')
regctl image copy ghcr.io/metalbear-co/operator:$IMAGE_VERSION your-registry/operator:$IMAGE_VERSION
AGENT_IMAGE_VERSION=$(regctl image config ghcr.io/metalbear-co/operator:$IMAGE_VERSION | jq -r '.config.Labels."metalbear.mirrord.version"')
regctl image copy ghcr.io/metalbear-co/mirrord:$AGENT_IMAGE_VERSION your-registry/mirrord:$AGENT_IMAGE_VERSION
operator:
image: your-registry/operator
agent:
image:
registry: your-registry/mirrord
Feature images are pulled only when the feature is enabled (Kafka sidecar, MSSQL tools, Flyway, Liquibase) and DB branch pods pull a per-engine database image — both have registry overrides under operator.<engine>BranchConfig.dbPod.image and imagePullSecrets. See references/helm-values.md.
openshift: true in values (renders a SecurityContextConstraints), covering the mirrord-operator and default service accounts in the mirrord namespace.AllowlistSynchronizer (not a manually-written WorkloadAllowlist — Autopilot rejects a manual one with an admission error on standard clusters):
apiVersion: auto.gke.io/v1
kind: AllowlistSynchronizer
metadata:
name: mirrord-allowlist
spec:
allowlistPaths:
- "mirrord/mirrord-agent/*"
If some configs still produce non-matching agent pods, merge agent.annotations.cloud.google.com/generate-allowlist: "true" into values to get the exact WorkloadAllowlist embedded in the operator's error logs.operator.port (e.g. 3000 / 8443) and ensure nodes can reach it.The chart creates the roles users need — you don't hand-write mirrord CRD roles. For each namespace where developers run mirrord, add it to roleNamespaces so a namespaced role is created there; then bind users to it with your own RoleBinding:
roleNamespaces:
- staging
- development
The chart also ships cluster roles: mirrord-operator-user-basic, mirrord-operator-user, and mirrord-operator-ci (scoped for CI — creating/deleting preview sessions plus the operator APIs the CLI needs; used by preview-environment CI, see the mirrord-prev-env skill). Bind the appropriate role to your users, groups, or service accounts. roleNamespaces: [] (empty) creates no namespaced roles.
Clients use the operator automatically when it's present. To force operator mode explicitly:
{ "operator": true, "target": "pod/my-app" }
mirrord exec --target pod/my-app -- node app.js
# Upgrade (reuse the same release name + values file)
helm repo update
helm upgrade --install -f values.yaml mirrord-operator metalbear/mirrord-operator
# Uninstall
helm uninstall mirrord-operator --namespace mirrord
kubectl delete namespace mirrord
Coordinate upgrades: in-flight sessions can break. Check with kubectl get sessions.operator.metalbear.co -A (or mirrord operator status) and upgrade when quiet.
operator.* flag; call out the generic-branching and preview security implications.helm/kubectl commands and wait for approval before running.mirrord operator status after install.