Bootstrap an Instance
dcctl bootstrap stands up a complete DeviceChain instance — infrastructure, the
operator, and all the service workloads — with a single command:
dcctl bootstrap local my-instance
dcctl carries its own content: the OpenTofu infrastructure config, the Helm
chart, and the operator manifests are all embedded inside it, so you never need a
checkout of the source tree or git to deploy.
It does not carry its own tools. bootstrap drives docker, kubectl,
helm and tofu (or terraform) as binaries on your PATH, and on the local
provider kind as well. It checks for all of them before it starts and stops if
one is missing, so install them first — the full list, and what each is used for,
is under Prerequisites.
DeviceChain is pre-release. dcctl bootstrap local is implemented and validated
end-to-end on local Kubernetes (kind), and it creates the kind cluster for you if
none is running — it asks first, unless you pass --yes. The gcp provider is a
planned follow-up.
What it does
The bootstrap runs as an ordered pipeline that builds an instance, and tells you which step failed if one does.
It is a create verb. Every credential the instance has is minted here — the database
passwords, the broker's authority and logins, the cross-service secret, the secret-store
root key — because none of them exists yet. Point it at an instance that is already
running and it stops at step 3 — before the operator, the infrastructure or the chart are
touched — and names the command that does move a live instance: dcctl upgrade, covered in
Releases & Upgrades.
A run that failed partway through is a different case, and re-running it is still how you repair it. What step 3 refuses is a live instance, which it recognises by the configuration document written in step 9 — so everything short of that is a half-built instance, and running the bootstrap again is the supported way to finish it.
The relational database changed from a StatefulSet to a CloudNativePG cluster, and there is no in-place upgrade — a StatefulSet's data directory cannot be adopted by the operator. On an instance created before that change, the bootstrap refuses and tells you how to dump the data or discard it deliberately. That refusal is the point: without it the old database would be removed and a new, empty one would take over the same hostname, leaving an instance that looks perfectly healthy and has no data in it.
This is the one documented reason to run dcctl bootstrap against an instance that is
already live, so --allow-legacy-db-removal is carved out of the refusal in step 3 as well
as this one. Nothing else is.
One instance per cluster. dcctl installs one DeviceChain instance into a cluster,
and step 4 is what says so. Almost everything a bootstrap applies is a cluster-wide
singleton — the operator's own Deployment and the infrastructure releases behind the
ingress controller, cert-manager and the CloudNativePG operator — so a second instance
does not sit beside the first one. Its own Helm release is named after the instance and
would not collide; everything around that release would. It installs this run's operator
over the one already running, adopts the shared infrastructure into a second OpenTofu
state, and mints database, broker and root-key credentials over the ones the instance
that is there is authenticating with. Step 4 asks the cluster what it already holds and
stops before any of that, naming the instance it found and the artifact it read that from.
There are three things to do instead:
- Move the instance that is there onto a new version —
dcctl upgradeis the verb for an instance that already exists, and it mints nothing. - Build this one in a cluster of its own — bootstrap it into a new local cluster, or
point it at a different one with
--kube-context. - Replace what is there with this one —
dcctl destroythe instance that is holding the cluster, then bootstrap again. That takes its data with it.
The refusal prints all three as commands, with your own instance and provider names filled in, so it can be acted on without coming back here.
It is not the step 3 refusal and the two do not overlap. That one recognises this
instance and stops a rebuild of it; this one asks whether anything else is here, so a
re-run aimed at the instance that is already there keeps meeting the message written for
it. A cluster that cannot say what it holds is an error rather than an empty cluster.
And because this refusal can only fire against a cluster dcctl did not create, it is
the one bootstrap failure that leaves nothing behind at all: the local record the run
wrote before it started is put back the way it was, so dcctl instances list does not
grow an entry for an instance that was never installed.
The steps below are the ones the run prints as it goes ([5/12] Install core components), so a failure names a step you can find here:
- Ensure local registry — the developer
--buildpath only: provision a local registry and build every image into it. On the published-image path it does nothing and says so. It goes first because the operator installed four steps later names an image, and on the--buildpath this is the step that produces it. - Claim the cluster — create the operator's namespace and take the cluster
lock, before anything is applied. While it is held, a second
dcctl bootstrapagainst the same cluster is refused rather than quietly applying over this one. See The Cluster Lock — that page also covers what to do when the cluster turns out to be claimed by somebody else. - Refuse a rebuild — ask the cluster whether this instance is already live, and stop if it is. Its position is deliberate on both sides: after the lock, because a concurrent bootstrap is exactly what would make the answer stale between reading it and acting on it, and before anything of the instance is applied, because every step below this one writes to a cluster that may already be running the instance it would be writing over. A dry run says what a real run would refuse rather than hiding it.
- Refuse a second instance — ask the cluster whether it is already holding a
different instance, and stop if it is. It is the other half of the same edge as the
step above, asking the opposite question: not "is this instance already here" but "is
anything else here". The two read different artifacts and cannot both fire — one keys
on finding this instance, the other on finding another. This one takes the first
answer it gets from the instance declarations in the cluster, then the credentials
dcctlminted intodc-system, then the DeviceChain Helm releases the cluster holds, asked in that order because it is the order a bootstrap writes them: a run that died partway through is answered by whatever it did get to. A dry run says what a real run would refuse, and says so even when it could not reach the cluster to ask. See One instance per cluster above. - Install core components — render the operator (CRDs + RBAC + controller) and apply it with the Kubernetes API directly. It runs ahead of the infrastructure apply because the definition of an instance has to exist in the cluster before anything can describe one to it — and describing one is the very next step.
- Declare the instance — write the instance's declaration into the cluster: the provider and cluster it belongs to, the profile, the image version, whether its databases are being recovered from an archive. It is then read back, and every step below works from what came back rather than from the flags that produced it — so the cluster, not your laptop, is the record of what this instance is. See the instance declaration.
- Render configuration — resolve the instance id, namespace, profile, and every generated credential: the broker-auth material (the shared service password and the callout issuer key), the certificate authority that signs the broker's own TLS certificate, the cross-service auth secret, and the secret-store root key. All of them are minted here, because step 3 has established there is no live instance to take them from. Finishing a half-built instance is the exception: there the step reads back what an earlier run already put in the cluster rather than generating a second set. It also records the broker's credentials on the machine you run it from, before the broker is configured with them, so that a run interrupted partway through can be resumed by simply running it again — the broker is configured before the instance is, and its credentials cannot be recovered from the cluster once they are in it. The root key is additionally escrowed to an encrypted file you keep; see Disaster Recovery.
- Apply infrastructure —
tofu applythe embedded OpenTofu config (NATS, PostgreSQL, TimescaleDB, NGINX ingress, cert-manager, the CloudNativePG operator and its Barman Cloud backup plugin, and the object store the backup plugin archives to) via terraform-exec. State is kept in~/.devicechain/instances/<instance>/infra, so subsequent runs are incremental. - Install instance (Helm) — write the instance's configuration document — the one every service reads its credentials and endpoints from — and then deploy the Helm chart via the Helm Go SDK, blocking until the workloads are ready. That document is what makes the instance live, and what step 3 looks for on any later run.
- Seed admin credential — the superuser credential is seeded by the user-management service on first start; this step settles the values the final report prints.
- Wait for readiness — poll each enabled area's Deployment until it has finished
rolling onto the configuration this run produced, as an explicit confirmation gate
rather than trusting the Helm step's own wait. Having replicas available is not
enough: where pods are being replaced that is already true of the ones on their way
out, so the step also waits for the new template to be observed, for every replica to
be recreated on it, and for no old replica to still be running.
dcctl upgradeuses the same gate for the same reason. - Report access info — print the namespace, the superuser credential, and how to reach the instance.
Ctrl+C stops a run cleanlyAn interrupted run stops the infrastructure tool gracefully — it finishes what it is
doing and writes its state — and hands the cluster lock back, so re-running is all that
is needed. A second Ctrl+C exits immediately and gives up both of those. See
Interrupting a run.
If the run had already reached step 9, the instance exists and the bootstrap will refuse
the next time you run it. That is not a dead end — the instance is built, and
dcctl upgrade is how you move it from there.
Because the embedded artifacts are the same ones the platform ships, a bootstrapped instance exercises the real deployment — it cannot drift from a production deploy.
Database backups need somewhere to go, and by default that somewhere is a single-replica MinIO in your instance's namespace, so that a stock bootstrap produces an instance whose write-ahead log is genuinely being archived rather than one carrying a backup plugin with nowhere to put anything.
Two things to know before you accept that default. MinIO is licensed AGPL-3.0, and community MinIO entered maintenance mode in December 2025 and was archived in April 2026, so the pinned image receives no further security patches. Neither affects DeviceChain's own Apache-2.0 licensing — the image is referenced, never built, modified or redistributed, and the platform reaches it over the S3 HTTP API — but the component does run in your cluster, and many organisations do not permit AGPL software regardless of how it is used.
Point the backup destination at storage outside the cluster to avoid both. That is the
recommended production configuration anyway, for a reason that has nothing to do with
licensing: an in-cluster bucket shares the cluster's failure domain, so it cannot be
disaster recovery. See Disaster Recovery and the OpenTofu
configuration's backup_destination.
Prerequisites
- A Kubernetes cluster, version 1.29 or newer, and a kube-context pointing at
it. The floor comes from the CloudNativePG charts, which refuse to install below
it;
dcctl preflightchecks it up front, because otherwise the failure lands part-way through a bootstrap that has already written your root-key escrow file. For thelocalprovider this is a local cluster (kind / minikube / k3d / docker-desktop).dcctlauto-detects a local context; pass--kube-context <name>to choose one explicitly. (Today thelocalprovider selects an existing context; creating the cluster for you is a planned addition.) - OpenTofu (the
tofubinary;terraformalso works) on yourPATH.dcctldrives it to provision infrastructure. Install it from opentofu.org. Rundcctl preflight localto check this and the rest of your environment up front.
Image source
By default, bootstrap deploys the published images from
ghcr.io/devicechain-io — nothing to build:
dcctl bootstrap local my-instance
Developers working from a source checkout can build the images from source and
deploy those instead with --build, which builds each service and the operator
with ko — plus the web console with docker build — into a
local registry and deploys by reference:
# from a source checkout; requires Docker + ko
dcctl bootstrap local my-instance --build
The only difference between the two paths is the registry the pods pull from — the pipeline, chart, and operator are identical.
Useful flags
| Flag | Purpose |
|---|---|
--kube-context <name> | Target a specific kube-context (default: auto-detect a local one). |
--profile <profile> | Functional-area profile: default (the standard system, used when omitted), full (everything — adds AI inference, outbound connectors, and MCP), telemetry, or ingest-only. |
--build | Build images from source into a local registry (developer path; needs the source tree + Docker + ko). |
--registry / --version | Override the image registry / tag (defaults: published ghcr.io/devicechain-io, or localhost:5000 + dev with --build). |
--host <name> | Ingress host to expose the instance on (default devicechain.local). Use localhost on a local cluster to reach the console with no /etc/hosts edit. |
--no-tls | Serve plain HTTP instead of a self-signed cert. With --host localhost, a zero-config http://localhost/ (no cert warning). |
--compact | Small-footprint preset — see below. |
--ha | Messaging high availability — see below. Needs at least 3 schedulable nodes. |
--no-cnpg | Skip the CloudNativePG operator and the database backup plugin. For a cluster that already runs CloudNativePG: Helm cannot adopt objects another installer created, so the infra apply fails without this. |
--dry-run | Print what each step would do without changing anything. A dry run creates no cluster and takes no cluster lock, so checks that need to read one — the --ha node-capacity check in particular — report what they could not see rather than failing the rehearsal; it does still report whether another operator is holding the cluster. What such a check does see is still fatal: a cluster that answers and cannot host --ha fails a dry run too. |
--skip-preflight | Skip the environment checks. |
--escrow-passphrase-file <path> | Read the root-key escrow passphrase from a file instead of prompting. See below. |
--escrow-file <path> | Write the escrow artifact somewhere other than ~/.devicechain/escrow/. |
--no-escrow | Do not escrow the root key. For throwaway instances only; implied by --dev. An instance created this way can be given an escrow later — see the escrow reconcile. |
--restore-root-key <path> | Disaster recovery: seed this instance's root key from an escrow artifact instead of minting one. |
The root-key escrow
Bootstrap writes an encrypted copy of the instance's secret-store root key to
~/.devicechain/escrow/<instance>-rootkey.escrow, sealed under a passphrase you
choose. It will prompt for that passphrase, or take it from
--escrow-passphrase-file or DCCTL_ESCROW_PASSPHRASE.
This is on by default, and a non-interactive run with no passphrase fails rather than proceeding without one:
# automation
DCCTL_ESCROW_PASSPHRASE="$(pass show devicechain/prod-escrow)" \
dcctl bootstrap local prod --yes
# a throwaway instance
dcctl bootstrap local scratch --dev
The root key encrypts every secret the instance stores, it lives only in the cluster's etcd, and no DeviceChain backup contains etcd. Without this file, a database backup restored to a new cluster rehydrates secrets that nothing can decrypt. The areas that store secrets refuse to start rather than serve credentials they cannot open, so you find out immediately — and there is nothing to be done about it by then. Disaster Recovery explains the whole procedure; read it before you need it.
An instance that has no escrow — one created with --no-escrow, or with --dev — can be
given one later without being rebuilt: dcctl upgrade writes the missing artifact when you
pass it a passphrase, and checks an existing one every time it runs. See
the escrow reconcile.
--compact
A preset for small clusters. It composes levers that already exist rather than adding a tuning axis of its own:
- lower JetStream and KV per-stream ceilings, and the smaller volumes those permit (2Gi JetStream, 2Gi relational Postgres, 4Gi TimescaleDB);
- lower scheduling requests (25m / 64Mi), so pods fit a small node — limits are untouched, since lowering the memory limit converts pressure into OOMKills and lowering the CPU limit throttles, neither of which shrinks anything;
- no monitoring stack, the single largest consumer;
- no cert-manager, since with TLS off nothing needs a certificate issued (keep TLS and cert-manager stays — see below), and consequently no database backup plugin.
It does not change which services run — that stays on --profile, where it is named
and visible. A profile larger than default — today only full — is rejected: the
published compact numbers are measured on default, so they would not describe an
instance running three more services. The smaller profiles (telemetry, ingest-only)
are accepted.
Both TLS and monitoring can be kept: an explicit --no-tls=false or --no-monitoring=false
is honoured, and every other compact lever still applies. Keeping TLS also keeps
cert-manager, which is what issues the certificate. --grafana-sso needs the monitoring
stack Grafana lives in, so it is rejected unless you keep it with --no-monitoring=false.
--compact --no-tls drops the backup pluginThe Barman Cloud plugin issues its own certificates through cert-manager, so dropping
cert-manager drops the plugin with it. Turning TLS back on (--no-tls=false) restores
both. Note it takes both flags: --no-tls on its own — as in the local-URL example
below — keeps cert-manager and therefore keeps the plugin.
The CloudNativePG operator itself is installed on every bring-up, compact included — one Deployment requesting 100m/128Mi, plus its CRDs. That is a footprint cost compact does not avoid, and it is deliberate: backup is not a high-availability feature, so the storage tier has one shape everywhere.
Both databases now run on the operator — the relational store and the event store alike.
The JetStream volume is derived: the per-stream ceilings are reserved up front, so it is
sized to hold their sum. The two database volumes are not. Nothing prunes the command or
alarm tables, and retentionDays defaults to 0 — keep data forever — so on a compact
instance meant to run indefinitely, set a retention window rather than relying on the
volume size.
Lowering a ceiling below what a stream or KV bucket already holds succeeds silently,
truncates nothing, and refuses writes until the data ages out. --compact is safe on a
first bring-up; it is not the same operation applied to a running instance.
dcctl bootstrap local my-instance --build --host localhost --no-tls exposes the
console at http://localhost/ — no hosts-file entry and no certificate warning.
--ha
Runs the message broker as a 3-node RAFT cluster, one server per node, with every JetStream stream and KV bucket replicated across it. The instance then survives the loss of any one node without losing messages, device sessions, or live state.
dcctl bootstrap local my-instance --ha
Both halves are set from that one flag, and that is the point of it. The broker's size is infrastructure (OpenTofu); the per-stream replica factor is instance configuration (Helm). They live in different tools, neither of which can see the other, and raising only the first is the failure mode this flag exists to prevent: a three-node cluster whose every stream is still single-replica costs three times the compute, reports three healthy peers, and survives nothing.
Three servers commit on a majority, so two remain a quorum and one does not. Losing a second node — including losing one to a rolling node upgrade while another is already down — stops writes until a node returns. Plan maintenance one node at a time. Surviving two concurrent losses needs a 5-server cluster, which is not a supported topology today.
Three schedulable nodes, not three nodes. The servers carry a hard anti-affinity
constraint, so if the cluster cannot place one per node the surplus stays Pending
rather than doubling up — co-located replicas would cost what replication costs and
protect against nothing. dcctl counts schedulable nodes and refuses before provisioning
anything. On a local kind cluster this means three workers: kind only removes the
control-plane's taint on a single-node cluster, so a control plane plus two workers is a
three-node cluster with two usable nodes.
What it also does. It runs the relational database as three instances with
synchronous replication, behind the same dc-postgresql hostname clients already use —
that hostname is maintained by the operator and follows the primary across a failover, so
no service configuration changes.
Synchronous replication is what forces three instances rather than two. One standby must confirm every commit, so with only two instances the loss of either one stalls every write: worse availability than a single node, in exchange for better durability. A third instance means a standby can be lost without the cluster losing its confirming replica.
The event store is replicated to three instances too, but with a deliberate difference: it does not hold a write waiting for a standby. If no standby is available it falls back to asynchronous replication and catches up when one returns. That trade is right for this store and wrong for the other one. Events are already held durably upstream in the messaging layer until they are persisted, so a failover's worth of writes can be replayed; the audit journal in the relational store has no such upstream, which is why it stalls instead. The cost is that the event store's recovery point is bounded by replication lag rather than being zero.
What it does not do. The number of service replicas is unchanged, and nothing here survives a node loss on its own — replication is what makes recovery possible, not what performs it.
This applies to the relational store, which is the one that stalls.
When no standby is available, a write does not fail — it waits, and the row is already
committed locally. A client that gives up and retries will write twice unless the operation
is idempotent. Note also that statement_timeout does not bound this wait, because the
wait happens after the commit rather than during the statement.
Verifying it
An HA claim is only worth what the broker actually holds, so check it there rather than in the rendered configuration:
dcctl ha verify --instance my-instance
This reads the live broker and asserts that every stream, KV bucket and durable consumer carries the declared replica factor with all peers current, and that the three servers are on three distinct nodes. It exits non-zero if anything falls short, and prints what it examined so a pass over an empty set is not mistaken for a pass.
After bootstrap
The command prints the namespace, the superuser credential, and how to reach the instance through the cluster ingress. The superuser is seeded with a default password — change it immediately.
The instance includes the web console: the ingress serves it at the host root
(https://<host>/) and routes https://<host>/api/<area>/graphql to each
functional-area service. Open the console in a browser and sign in with the
superuser's email and password. A fresh instance is tenant-less, so you land
in the admin console (/admin) to create your first tenant and assign
memberships; switch into a tenant to reach the tenant console. (For a
headless/ingest-only instance, deploy with the console disabled — see the chart's
frontend.enabled value.)
To inspect the running instance:
kubectl --context <kube-context> get pods -n my-instance
To explore the console against a moving fleet rather than an empty one, run a
simulation. sim create mints a scoped identity and tenant on the instance
and writes the handshake file the dc-simulator process reads to come up:
dcctl sim create demo --instance my-instance --server localhost
The simulator then drives telemetry and alarms in over the same device wire real hardware uses — see Trying it with simulated data.