Multi-Environment Management

Goal

Set up and manage multiple deployment environments (development, staging, production) for the Keymate platform. At the end of this guide, you have isolated environments with independent configuration, a clear promotion workflow for moving changes between stages, and confidence that production changes are validated before deployment.

Audience

Platform engineers and operators responsible for managing multiple environments across the deployment lifecycle.

Prerequisites

• A running Keymate platform deployment
• At least one Tenant onboarded
• Understanding of the Customization Overview (platform → Tenant → environment override model)

Before You Start

• Define your environment strategy. Most organizations use three stages: development, staging, and production. Some add additional stages (QA, performance testing, canary). Decide your stage count before setting up environments.
• Choose your isolation model. You can run environments on separate clusters (strongest isolation) or as separate configurations within the same cluster (shared infrastructure, logical isolation). The right choice depends on your compliance requirements and infrastructure budget.
• Plan your promotion workflow. Decide how changes move from one stage to the next. GitOps workflows handle this naturally through Git branch or directory-based promotion. Helm-based workflows require explicit upgrade commands per environment.

Steps

1. Define the environment structure

Determine how many environments you need and what purpose each one serves.

Environment	Purpose	Typical users
Development	Feature development, rapid iteration, debugging	Developers, platform engineers
Staging	Pre-production validation, integration testing, release candidate verification	QA, platform engineers, product
Production	Live customer-facing environment	End users, operators

2. Configure environment-specific overrides

Each environment needs its own configuration overrides. Use the customization model to set environment-level values.

Common environment differences:

Setting	Development	Staging	Production
Replica count	1	2	3+
Resource limits	Low	Medium	High
Debug logging	Enabled	Enabled	Disabled
TLS certificates	Self-signed or Let's Encrypt staging	Let's Encrypt staging	Production certificates
Domain	dev.acme.example.com	staging.acme.example.com	acme.example.com
External integrations	Mock or sandbox	Sandbox	Production endpoints
Autoscaling	Disabled	Enabled (conservative)	Enabled (production tuned)

3. Set up environment isolation

Ensure environments cannot affect each other.

Cluster-level isolation (recommended for production):

• Separate Kubernetes clusters per environment
• Independent infrastructure, networking, and credentials
• Strongest guarantee that a development issue cannot impact production

Namespace-level isolation (acceptable for dev/staging):

• Separate namespaces within the same cluster
• Network policies to prevent cross-environment traffic
• Resource quotas to prevent one environment from consuming all cluster resources

4. Establish the promotion workflow

Define how configuration changes move through environments.

For GitOps deployments:

Changes are promoted through Git:

1. Engineer makes a configuration change and commits to the development environment directory
2. After validation in dev, a pull request promotes the change to the staging directory
3. After staging validation, another pull request promotes to the production directory
4. ArgoCD applies each change to the respective environment automatically

For Helm deployments:

Changes are promoted through explicit upgrade commands:

1. Engineer updates the dev values file and runs helm upgrade on the dev environment
2. After validation, the same change is applied to the staging values file
3. After staging validation, applied to the production values file

5. Validate environment parity

Keep environments as similar as possible in structure, differing only in scale and external configuration. Validate that:

• All environments deploy the same set of platform components
• Component versions are consistent (promote the same version through stages)
• You have intentionally documented all configuration differences
• No environment contains manual changes that you have not tracked in configuration

6. Monitor each environment independently

Set up observability per environment so you can detect issues at each stage.

• Configure each environment to emit telemetry to its own monitoring context
• Make alerts environment-aware (a staging alert does not page the on-call team)
• Set up production monitoring as the most comprehensive

See Observability Overview for monitoring setup details.

Validation Scenario

Scenario

A platform team manages three environments for a Tenant. They need to verify that a configuration change can be safely promoted from development to production.

Expected Result

• A change applied in development does not affect staging or production
• The same change can be promoted to staging through a controlled process
• After staging validation, the change reaches production with no surprises
• Each environment's monitoring shows the change taking effect independently

How to Verify

• Apply a configuration change (e.g., adjust a replica count) in dev only
• Confirm staging and production are unchanged
• Promote to staging and verify the change applies correctly
• Promote to production and verify the final result

Troubleshooting

• Configuration drift between environments. Compare the configuration for each environment. In GitOps, use git diff between environment directories. In Helm, compare values files. Drift typically occurs when manual changes bypass the promotion workflow.
• Promotion causes unexpected failures. The staging environment may not fully replicate production conditions (different scale, different external services). Ensure staging uses production-like settings for critical integrations.
• Environment-specific secrets missing. Each environment needs its own credentials. A common failure is promoting a change that references a secret that exists in dev but not in staging. Verify secrets exist in the target environment before promoting.

tip

Treat your promotion workflow as a pipeline, not a manual process. Automated promotion with gates (tests pass, approval received) reduces human error and speeds up delivery.

Next Steps

• Multi-Stage Environments — Understand the multi-stage deployment model
• Customization Overview — Deep dive into the override model
• GitOps-Based Installation — GitOps-native environment promotion

Related Docs

Multi-Stage Environments

Deployment model for multiple stages

Customization Overview

Configuration override model

Deployment Best Practices

Production readiness recommendations

Observability Overview

Per-environment monitoring setup