Skip to main content

Bootstrap an Instance

dcctl bootstrap stands up a complete DeviceChain instance — infrastructure, the operator, and all the service workloads — with a single command:

dcctl bootstrap local my-instance

dcctl carries its own content: the OpenTofu infrastructure config, the Helm chart, and the operator manifests are all embedded inside it, so you never need a checkout of the source tree or git to deploy.

It does not carry its own tools. bootstrap drives docker, kubectl, helm and tofu (or terraform) as binaries on your PATH, and on the local provider kind as well. It checks for all of them before it starts and stops if one is missing, so install them first — the full list, and what each is used for, is under Prerequisites.

Status

DeviceChain is pre-release. dcctl bootstrap local is implemented and validated end-to-end on local Kubernetes (kind), and it creates the kind cluster for you if none is running — it asks first, unless you pass --yes. The gcp provider is a planned follow-up.

What it does

The bootstrap runs as an ordered pipeline that builds an instance, and tells you which step failed if one does.

It is a create verb. Every credential the instance has is minted here — the database passwords, the broker's authority and logins, the cross-service secret, the secret-store root key — because none of them exists yet. Point it at an instance that is already running and it stops at step 3 — before the operator, the infrastructure or the chart are touched — and names the command that does move a live instance: dcctl upgrade, covered in Releases & Upgrades.

A run that failed partway through is a different case, and re-running it is still how you repair it. What step 3 refuses is a live instance, which it recognises by the configuration document written in step 9 — so everything short of that is a half-built instance, and running the bootstrap again is the supported way to finish it.

One-time exception: instances created before the database moved to CloudNativePG

The relational database changed from a StatefulSet to a CloudNativePG cluster, and there is no in-place upgrade — a StatefulSet's data directory cannot be adopted by the operator. On an instance created before that change, the bootstrap refuses and tells you how to dump the data or discard it deliberately. That refusal is the point: without it the old database would be removed and a new, empty one would take over the same hostname, leaving an instance that looks perfectly healthy and has no data in it.

This is the one documented reason to run dcctl bootstrap against an instance that is already live, so --allow-legacy-db-removal is carved out of the refusal in step 3 as well as this one. Nothing else is.

One instance per cluster. dcctl installs one DeviceChain instance into a cluster, and step 4 is what says so. Almost everything a bootstrap applies is a cluster-wide singleton — the operator's own Deployment and the infrastructure releases behind the ingress controller, cert-manager and the CloudNativePG operator — so a second instance does not sit beside the first one. Its own Helm release is named after the instance and would not collide; everything around that release would. It installs this run's operator over the one already running, adopts the shared infrastructure into a second OpenTofu state, and mints database, broker and root-key credentials over the ones the instance that is there is authenticating with. Step 4 asks the cluster what it already holds and stops before any of that, naming the instance it found and the artifact it read that from. There are three things to do instead:

  • Move the instance that is there onto a new versiondcctl upgrade is the verb for an instance that already exists, and it mints nothing.
  • Build this one in a cluster of its own — bootstrap it into a new local cluster, or point it at a different one with --kube-context.
  • Replace what is there with this onedcctl destroy the instance that is holding the cluster, then bootstrap again. That takes its data with it.

The refusal prints all three as commands, with your own instance and provider names filled in, so it can be acted on without coming back here.

It is not the step 3 refusal and the two do not overlap. That one recognises this instance and stops a rebuild of it; this one asks whether anything else is here, so a re-run aimed at the instance that is already there keeps meeting the message written for it. A cluster that cannot say what it holds is an error rather than an empty cluster. And because this refusal can only fire against a cluster dcctl did not create, it is the one bootstrap failure that leaves nothing behind at all: the local record the run wrote before it started is put back the way it was, so dcctl instances list does not grow an entry for an instance that was never installed.

The steps below are the ones the run prints as it goes ([5/12] Install core components), so a failure names a step you can find here:

  1. Ensure local registry — the developer --build path only: provision a local registry and build every image into it. On the published-image path it does nothing and says so. It goes first because the operator installed four steps later names an image, and on the --build path this is the step that produces it.
  2. Claim the cluster — create the operator's namespace and take the cluster lock, before anything is applied. While it is held, a second dcctl bootstrap against the same cluster is refused rather than quietly applying over this one. See The Cluster Lock — that page also covers what to do when the cluster turns out to be claimed by somebody else.
  3. Refuse a rebuild — ask the cluster whether this instance is already live, and stop if it is. Its position is deliberate on both sides: after the lock, because a concurrent bootstrap is exactly what would make the answer stale between reading it and acting on it, and before anything of the instance is applied, because every step below this one writes to a cluster that may already be running the instance it would be writing over. A dry run says what a real run would refuse rather than hiding it.
  4. Refuse a second instance — ask the cluster whether it is already holding a different instance, and stop if it is. It is the other half of the same edge as the step above, asking the opposite question: not "is this instance already here" but "is anything else here". The two read different artifacts and cannot both fire — one keys on finding this instance, the other on finding another. This one takes the first answer it gets from the instance declarations in the cluster, then the credentials dcctl minted into dc-system, then the DeviceChain Helm releases the cluster holds, asked in that order because it is the order a bootstrap writes them: a run that died partway through is answered by whatever it did get to. A dry run says what a real run would refuse, and says so even when it could not reach the cluster to ask. See One instance per cluster above.
  5. Install core components — render the operator (CRDs + RBAC + controller) and apply it with the Kubernetes API directly. It runs ahead of the infrastructure apply because the definition of an instance has to exist in the cluster before anything can describe one to it — and describing one is the very next step.
  6. Declare the instance — write the instance's declaration into the cluster: the provider and cluster it belongs to, the profile, the image version, whether its databases are being recovered from an archive. It is then read back, and every step below works from what came back rather than from the flags that produced it — so the cluster, not your laptop, is the record of what this instance is. See the instance declaration.
  7. Render configuration — resolve the instance id, namespace, profile, and every generated credential: the broker-auth material (the shared service password and the callout issuer key), the certificate authority that signs the broker's own TLS certificate, the cross-service auth secret, and the secret-store root key. All of them are minted here, because step 3 has established there is no live instance to take them from. Finishing a half-built instance is the exception: there the step reads back what an earlier run already put in the cluster rather than generating a second set. It also records the broker's credentials on the machine you run it from, before the broker is configured with them, so that a run interrupted partway through can be resumed by simply running it again — the broker is configured before the instance is, and its credentials cannot be recovered from the cluster once they are in it. The root key is additionally escrowed to an encrypted file you keep; see Disaster Recovery.
  8. Apply infrastructuretofu apply the embedded OpenTofu config (NATS, PostgreSQL, TimescaleDB, NGINX ingress, cert-manager, the CloudNativePG operator and its Barman Cloud backup plugin, and the object store the backup plugin archives to) via terraform-exec. State is kept in ~/.devicechain/instances/<instance>/infra, so subsequent runs are incremental.
  9. Install instance (Helm) — write the instance's configuration document — the one every service reads its credentials and endpoints from — and then deploy the Helm chart via the Helm Go SDK, blocking until the workloads are ready. That document is what makes the instance live, and what step 3 looks for on any later run.
  10. Seed admin credential — the superuser credential is seeded by the user-management service on first start; this step settles the values the final report prints.
  11. Wait for readiness — poll each enabled area's Deployment until it has finished rolling onto the configuration this run produced, as an explicit confirmation gate rather than trusting the Helm step's own wait. Having replicas available is not enough: where pods are being replaced that is already true of the ones on their way out, so the step also waits for the new template to be observed, for every replica to be recreated on it, and for no old replica to still be running. dcctl upgrade uses the same gate for the same reason.
  12. Report access info — print the namespace, the superuser credential, and how to reach the instance.
Ctrl+C stops a run cleanly

An interrupted run stops the infrastructure tool gracefully — it finishes what it is doing and writes its state — and hands the cluster lock back, so re-running is all that is needed. A second Ctrl+C exits immediately and gives up both of those. See Interrupting a run.

If the run had already reached step 9, the instance exists and the bootstrap will refuse the next time you run it. That is not a dead end — the instance is built, and dcctl upgrade is how you move it from there.

Because the embedded artifacts are the same ones the platform ships, a bootstrapped instance exercises the real deployment — it cannot drift from a production deploy.

The default backup destination is an AGPL component

Database backups need somewhere to go, and by default that somewhere is a single-replica MinIO in your instance's namespace, so that a stock bootstrap produces an instance whose write-ahead log is genuinely being archived rather than one carrying a backup plugin with nowhere to put anything.

Two things to know before you accept that default. MinIO is licensed AGPL-3.0, and community MinIO entered maintenance mode in December 2025 and was archived in April 2026, so the pinned image receives no further security patches. Neither affects DeviceChain's own Apache-2.0 licensing — the image is referenced, never built, modified or redistributed, and the platform reaches it over the S3 HTTP API — but the component does run in your cluster, and many organisations do not permit AGPL software regardless of how it is used.

Point the backup destination at storage outside the cluster to avoid both. That is the recommended production configuration anyway, for a reason that has nothing to do with licensing: an in-cluster bucket shares the cluster's failure domain, so it cannot be disaster recovery. See Disaster Recovery and the OpenTofu configuration's backup_destination.

Prerequisites

  • A Kubernetes cluster, version 1.29 or newer, and a kube-context pointing at it. The floor comes from the CloudNativePG charts, which refuse to install below it; dcctl preflight checks it up front, because otherwise the failure lands part-way through a bootstrap that has already written your root-key escrow file. For the local provider this is a local cluster (kind / minikube / k3d / docker-desktop). dcctl auto-detects a local context; pass --kube-context <name> to choose one explicitly. (Today the local provider selects an existing context; creating the cluster for you is a planned addition.)
  • OpenTofu (the tofu binary; terraform also works) on your PATH. dcctl drives it to provision infrastructure. Install it from opentofu.org. Run dcctl preflight local to check this and the rest of your environment up front.

Image source

By default, bootstrap deploys the published images from ghcr.io/devicechain-io — nothing to build:

dcctl bootstrap local my-instance

Developers working from a source checkout can build the images from source and deploy those instead with --build, which builds each service and the operator with ko — plus the web console with docker build — into a local registry and deploys by reference:

# from a source checkout; requires Docker + ko
dcctl bootstrap local my-instance --build

The only difference between the two paths is the registry the pods pull from — the pipeline, chart, and operator are identical.

Useful flags

FlagPurpose
--kube-context <name>Target a specific kube-context (default: auto-detect a local one).
--profile <profile>Functional-area profile: default (the standard system, used when omitted), full (everything — adds AI inference, outbound connectors, and MCP), telemetry, or ingest-only.
--buildBuild images from source into a local registry (developer path; needs the source tree + Docker + ko).
--registry / --versionOverride the image registry / tag (defaults: published ghcr.io/devicechain-io, or localhost:5000 + dev with --build).
--host <name>Ingress host to expose the instance on (default devicechain.local). Use localhost on a local cluster to reach the console with no /etc/hosts edit.
--no-tlsServe plain HTTP instead of a self-signed cert. With --host localhost, a zero-config http://localhost/ (no cert warning).
--compactSmall-footprint preset — see below.
--haMessaging high availability — see below. Needs at least 3 schedulable nodes.
--no-cnpgSkip the CloudNativePG operator and the database backup plugin. For a cluster that already runs CloudNativePG: Helm cannot adopt objects another installer created, so the infra apply fails without this.
--dry-runPrint what each step would do without changing anything. A dry run creates no cluster and takes no cluster lock, so checks that need to read one — the --ha node-capacity check in particular — report what they could not see rather than failing the rehearsal; it does still report whether another operator is holding the cluster. What such a check does see is still fatal: a cluster that answers and cannot host --ha fails a dry run too.
--skip-preflightSkip the environment checks.
--escrow-passphrase-file <path>Read the root-key escrow passphrase from a file instead of prompting. See below.
--escrow-file <path>Write the escrow artifact somewhere other than ~/.devicechain/escrow/.
--no-escrowDo not escrow the root key. For throwaway instances only; implied by --dev. An instance created this way can be given an escrow later — see the escrow reconcile.
--restore-root-key <path>Disaster recovery: seed this instance's root key from an escrow artifact instead of minting one.

The root-key escrow

Bootstrap writes an encrypted copy of the instance's secret-store root key to ~/.devicechain/escrow/<instance>-rootkey.escrow, sealed under a passphrase you choose. It will prompt for that passphrase, or take it from --escrow-passphrase-file or DCCTL_ESCROW_PASSPHRASE.

This is on by default, and a non-interactive run with no passphrase fails rather than proceeding without one:

# automation
DCCTL_ESCROW_PASSPHRASE="$(pass show devicechain/prod-escrow)" \
dcctl bootstrap local prod --yes

# a throwaway instance
dcctl bootstrap local scratch --dev
This file is not optional for anything you care about

The root key encrypts every secret the instance stores, it lives only in the cluster's etcd, and no DeviceChain backup contains etcd. Without this file, a database backup restored to a new cluster rehydrates secrets that nothing can decrypt. The areas that store secrets refuse to start rather than serve credentials they cannot open, so you find out immediately — and there is nothing to be done about it by then. Disaster Recovery explains the whole procedure; read it before you need it.

An instance that has no escrow — one created with --no-escrow, or with --dev — can be given one later without being rebuilt: dcctl upgrade writes the missing artifact when you pass it a passphrase, and checks an existing one every time it runs. See the escrow reconcile.

--compact

A preset for small clusters. It composes levers that already exist rather than adding a tuning axis of its own:

  • lower JetStream and KV per-stream ceilings, and the smaller volumes those permit (2Gi JetStream, 2Gi relational Postgres, 4Gi TimescaleDB);
  • lower scheduling requests (25m / 64Mi), so pods fit a small node — limits are untouched, since lowering the memory limit converts pressure into OOMKills and lowering the CPU limit throttles, neither of which shrinks anything;
  • no monitoring stack, the single largest consumer;
  • no cert-manager, since with TLS off nothing needs a certificate issued (keep TLS and cert-manager stays — see below), and consequently no database backup plugin.

It does not change which services run — that stays on --profile, where it is named and visible. A profile larger than default — today only full — is rejected: the published compact numbers are measured on default, so they would not describe an instance running three more services. The smaller profiles (telemetry, ingest-only) are accepted.

Both TLS and monitoring can be kept: an explicit --no-tls=false or --no-monitoring=false is honoured, and every other compact lever still applies. Keeping TLS also keeps cert-manager, which is what issues the certificate. --grafana-sso needs the monitoring stack Grafana lives in, so it is rejected unless you keep it with --no-monitoring=false.

Why --compact --no-tls drops the backup plugin

The Barman Cloud plugin issues its own certificates through cert-manager, so dropping cert-manager drops the plugin with it. Turning TLS back on (--no-tls=false) restores both. Note it takes both flags: --no-tls on its own — as in the local-URL example below — keeps cert-manager and therefore keeps the plugin.

The CloudNativePG operator itself is installed on every bring-up, compact included — one Deployment requesting 100m/128Mi, plus its CRDs. That is a footprint cost compact does not avoid, and it is deliberate: backup is not a high-availability feature, so the storage tier has one shape everywhere.

Both databases now run on the operator — the relational store and the event store alike.

Volume sizes are a time budget, not a capacity budget

The JetStream volume is derived: the per-stream ceilings are reserved up front, so it is sized to hold their sum. The two database volumes are not. Nothing prunes the command or alarm tables, and retentionDays defaults to 0 — keep data forever — so on a compact instance meant to run indefinitely, set a retention window rather than relying on the volume size.

Apply it to a fresh cluster

Lowering a ceiling below what a stream or KV bucket already holds succeeds silently, truncates nothing, and refuses writes until the data ages out. --compact is safe on a first bring-up; it is not the same operation applied to a running instance.

Zero-config local URL

dcctl bootstrap local my-instance --build --host localhost --no-tls exposes the console at http://localhost/ — no hosts-file entry and no certificate warning.

--ha

Runs the message broker as a 3-node RAFT cluster, one server per node, with every JetStream stream and KV bucket replicated across it. The instance then survives the loss of any one node without losing messages, device sessions, or live state.

dcctl bootstrap local my-instance --ha

Both halves are set from that one flag, and that is the point of it. The broker's size is infrastructure (OpenTofu); the per-stream replica factor is instance configuration (Helm). They live in different tools, neither of which can see the other, and raising only the first is the failure mode this flag exists to prevent: a three-node cluster whose every stream is still single-replica costs three times the compute, reports three healthy peers, and survives nothing.

It survives exactly ONE node loss

Three servers commit on a majority, so two remain a quorum and one does not. Losing a second node — including losing one to a rolling node upgrade while another is already down — stops writes until a node returns. Plan maintenance one node at a time. Surviving two concurrent losses needs a 5-server cluster, which is not a supported topology today.

Three schedulable nodes, not three nodes. The servers carry a hard anti-affinity constraint, so if the cluster cannot place one per node the surplus stays Pending rather than doubling up — co-located replicas would cost what replication costs and protect against nothing. dcctl counts schedulable nodes and refuses before provisioning anything. On a local kind cluster this means three workers: kind only removes the control-plane's taint on a single-node cluster, so a control plane plus two workers is a three-node cluster with two usable nodes.

What it also does. It runs the relational database as three instances with synchronous replication, behind the same dc-postgresql hostname clients already use — that hostname is maintained by the operator and follows the primary across a failover, so no service configuration changes.

Synchronous replication is what forces three instances rather than two. One standby must confirm every commit, so with only two instances the loss of either one stalls every write: worse availability than a single node, in exchange for better durability. A third instance means a standby can be lost without the cluster losing its confirming replica.

The event store is replicated to three instances too, but with a deliberate difference: it does not hold a write waiting for a standby. If no standby is available it falls back to asynchronous replication and catches up when one returns. That trade is right for this store and wrong for the other one. Events are already held durably upstream in the messaging layer until they are persisted, so a failover's worth of writes can be replayed; the audit journal in the relational store has no such upstream, which is why it stalls instead. The cost is that the event store's recovery point is bounded by replication lag rather than being zero.

What it does not do. The number of service replicas is unchanged, and nothing here survives a node loss on its own — replication is what makes recovery possible, not what performs it.

A stalled write is committed, not rejected

This applies to the relational store, which is the one that stalls.

When no standby is available, a write does not fail — it waits, and the row is already committed locally. A client that gives up and retries will write twice unless the operation is idempotent. Note also that statement_timeout does not bound this wait, because the wait happens after the commit rather than during the statement.

Verifying it

An HA claim is only worth what the broker actually holds, so check it there rather than in the rendered configuration:

dcctl ha verify --instance my-instance

This reads the live broker and asserts that every stream, KV bucket and durable consumer carries the declared replica factor with all peers current, and that the three servers are on three distinct nodes. It exits non-zero if anything falls short, and prints what it examined so a pass over an empty set is not mistaken for a pass.

After bootstrap

The command prints the namespace, the superuser credential, and how to reach the instance through the cluster ingress. The superuser is seeded with a default password — change it immediately.

The instance includes the web console: the ingress serves it at the host root (https://<host>/) and routes https://<host>/api/<area>/graphql to each functional-area service. Open the console in a browser and sign in with the superuser's email and password. A fresh instance is tenant-less, so you land in the admin console (/admin) to create your first tenant and assign memberships; switch into a tenant to reach the tenant console. (For a headless/ingest-only instance, deploy with the console disabled — see the chart's frontend.enabled value.)

To inspect the running instance:

kubectl --context <kube-context> get pods -n my-instance

To explore the console against a moving fleet rather than an empty one, run a simulation. sim create mints a scoped identity and tenant on the instance and writes the handshake file the dc-simulator process reads to come up:

dcctl sim create demo --instance my-instance --server localhost

The simulator then drives telemetry and alarms in over the same device wire real hardware uses — see Trying it with simulated data.