DevGuard CloudNativePG: Deploy with a Highly Available Database

The Helm chart ships a single-instance PostgreSQL StatefulSet with no replicas and no automatic failover. For a highly available database, run PostgreSQL with the CloudNativePG operator and connect DevGuard to it as an external database.

This guide starts with an empty Kubernetes cluster and ends with a running DevGuard instance on a three-node PostgreSQL cluster.

Prerequisites

  • A Kubernetes cluster (1.24+) with a default StorageClass, or kind and Docker to create a local one
  • kubectl and Helm 3.x
  • Optional: an S3-compatible bucket (AWS S3, MinIO, Garage, …) for backups

All commands use the devguard namespace. The database and DevGuard must run in the same namespace.

Create a local cluster (optional)

To try the setup locally, create a cluster with kind. Skip this step if you install into an existing cluster.

kind switches your current kubectl context to kind-devguard, so all following commands run against this cluster. It also comes with a default StorageClass, which the PostgreSQL volumes need.

To remove the cluster again:

Install the CloudNativePG operator

Skip this step if the operator is already installed. See the CloudNativePG installation guide for Helm-based installs and newer versions.

Create the namespace

Create the database secrets

DevGuard uses two databases: devguard for the API and kratos for authentication. Each has its own role and password.

The Helm chart and CloudNativePG expect credentials in different formats, so you create two secrets per role, holding the same password:

SecretRead byKeysName
db-secretDevGuard APIpostgres-passwordFixed. The chart only looks for this name.
kratos-db-secretKratospasswordFixed. The chart only looks for this name.
devguard-pg-appCloudNativePGusername, passwordYour choice. Referenced in the Cluster.
devguard-pg-kratosCloudNativePGusername, passwordYour choice. Referenced in the Cluster.

The CloudNativePG secrets must be of type kubernetes.io/basic-auth, and their username must be devguard and kratos. DevGuard and Kratos always connect with these role names.

Create the PostgreSQL cluster

Use DevGuard's PostgreSQL image, ghcr.io/l3montree-dev/devguard/postgresql. It includes the semver extension DevGuard needs. The upstream CloudNativePG images don't.

Check that both databases exist with the right owners and that semver is installed:

Expected output:

devguard|devguard
kratos|kratos
semver

Create the values file

Turn off the bundled database and point the chart at the cluster's read/write service. CloudNativePG names it <cluster-name>-rw and always routes it to the current primary, so DevGuard follows a failover without any change. DevGuard connects to a single host and doesn't send reads to replicas, so you don't need the -ro or -r services.

sslMode applies to every connection from DevGuard and Kratos. It takes the standard PostgreSQL values (disable, require, verify-ca, verify-full, …) and defaults to disable. CloudNativePG enables TLS on every cluster, so require works without extra setup. For verify-ca or verify-full, the clients also need to trust the cluster's CA certificate, which CloudNativePG stores in the devguard-pg-ca secret.

For ingress, mail and authentication settings, see Deploy with Helm.

Install DevGuard

This installs the latest chart release. Add --version <version> to pin a specific one.

The chart generates the remaining secrets, including the In-Toto signing key, on the first install.

Check that the database migrations ran

Before the API starts, its devguard-migrate init container creates or updates the database schema. Wait for the API to roll out, then check the migration log:

Expected output:

migrations completed successfully
schema migrations completed successfully

If the rollout times out and the API pod stays in Init, the logs of devguard-migrate show the database error. See Troubleshooting.

Verify the deployment

All pods should be Running: three PostgreSQL instances (devguard-pg-1 to -3), the API, the web frontend and Kratos.

NAME                                       READY   STATUS    RESTARTS   AGE
devguard-api-deployment-55c8488865-jdv4c   1/1     Running   0          3m
devguard-pg-1                              1/1     Running   0          9m
devguard-pg-2                              1/1     Running   0          8m
devguard-pg-3                              1/1     Running   0          6m
devguard-web-deployment-ff55b888d-j2q9n    1/1     Running   0          3m
kratos-5cbdd649c5-jfvsj                    1/1     Running   0          3m

To confirm that DevGuard and Kratos connect over TLS:

Every row should show t in the second column.

Enable backups

CloudNativePG can continuously archive WAL to an S3-compatible bucket and take base backups from it. Together they let you restore the database to any point in time.

Create a secret with the bucket credentials:

Add a backup section to the Cluster in cluster.yaml and apply it again:

Check that WAL archiving works:

Schedule a daily base backup:

The first backup should reach the completed phase within a few minutes. To restore, see Recovery in the CloudNativePG documentation.

Troubleshooting

SymptomCauseFix
initdb job fails with could not look up effective user ID 26CloudNativePG's default UID doesn't match the imageSet postgresUID: 999 and postgresGID: 999
API or Kratos pods fail with secret "db-secret" not found (or kratos-db-secret)The chart's secrets are missing or named differentlyCreate them with exactly these names, see step 3
devguard-migrate fails with password authentication failed for user "devguard"db-secret and devguard-pg-app hold different passwordsMake both secrets hold the same password
devguard-migrate fails with extension "semver" is not availableThe cluster runs an image without semverUse ghcr.io/l3montree-dev/devguard/postgresql as imageName
ContinuousArchiving is False, logs show HeadBucket operation: Bad RequestRegion is missing or wrongSet s3Credentials.region to your object store's region
ContinuousArchiving is False, logs show Unable to locate credentialsCredential secret or key names are wrongCheck the secret referenced in s3Credentials

Have feedback? We want to hear from you!

Fields marked with * are required