Choosing a size
An instance's size is one choice with a name. dcctl turns the name into the same set of
settings every time, so a number measured on one cluster can be reproduced on another instead of
being rebuilt from a list of overlay entries and TF_VAR_ exports.
A size is a preset, not a dial. It composes settings that already exist, it never changes
which services run (that stays on the profile), and it never turns a behaviour
on: event envelopes and the fail-closed MQTT acknowledgement stay as you set them. It is separate
from --ha, which decides how many copies run, while a size decides how big each copy is.
| Size | For | Status |
|---|---|---|
base | A laptop or a small cluster. The default: an instance that names no size is base, and renders exactly what it rendered before sizes existed. | The standard sizing, described in Service sizing. |
compact | A laptop with little to spare. The cluster's setting, chosen at install; --compact is the same thing and stays. | Described under --compact. |
perf-12k | One cluster without --ha, platform pools of about 40 vCPU, sustained 12,000 events a second with event envelopes on. | Measured: 12,000 events a second held clean on GKE (acknowledgement p99 under 4 ms), with every event accounted for. A few values are corrected from that run and not yet run as written; see below. |
perf-16k | The same shape for sustained 16,000 events a second: a larger broker, write-ahead log allowance and event-processing pod. 20,000 events a second is a throughput-only ceiling, not the target. | Derived from the same GKE run, which held 16,000 events a second exactly; the sizing here targets the 50 ms acknowledgement latency that run narrowly missed (63 ms on the perf-12k sizing). Not yet run as written; see below. |
A size is named for the rate it holds inside the latency target. The name is the claim, so a size is
never published as a target. That is why there is no perf-20k: the measured shape stored 20,000
events a second exactly, with none lost or duplicated, but acknowledgements took 245 ms at the
99th percentile, so 20,000 is documented as a throughput ceiling of perf-16k and not as a size.
Choosing one
dcctl install <provider> --kube-context <context>
dcctl bootstrap <provider> <instance> --size perf-12k
--size compact on dcctl install is the same as --compact. The perf sizes are chosen per
instance, on dcctl bootstrap. A perf size is refused on a compact cluster.
The size is recorded on the instance and dcctl upgrade applies it again every time, so an
upgrade never changes the size of an instance. A release that retunes a size changes the
instances recorded with it on their next upgrade, and its upgrade notes say so.
What perf-12k sets
Each row is one pod. --ha decides the replica count of event-management (two) and the number of broker servers and database instances (three each); the replica counts below for the other services stay as written, and the size of every copy stays as written.
| Service | Replicas | CPU limit | Memory limit | CPU request |
|---|---|---|---|---|
device-management | 2 | 3 | - | 400m |
device-state | 2 | 3 | 512Mi | 250m |
event-management | - | 4 | 1Gi | 400m |
event-processing | - | 2 | 1Gi | 500m |
event-sources | 3 | 3 | 256Mi | 600m |
A dash means the size states nothing and the chart's own value applies. device-management keeps
its memory floor rather than a limit of its own, because a limit written by a size would skip the
floor. event-management keeps one replica unless --ha decides otherwise.
The broker, the volumes and the event store's write-ahead log allowance:
| Setting | Value |
|---|---|
streamDesignRate | 12000 |
nats_cpu_request | 4 |
nats_memory_request | 4Gi |
nats_memory_limit | 8Gi |
nats_jetstream_storage | 28Gi |
timescale_storage | 160Gi |
timescale_max_wal_size_mib | 20480 |
postgres_max_wal_size_mib | 4096 |
The JetStream volume is sized so that the reservation of the design rate fits in it; a volume that an instance already has is never resized by an upgrade, so a size applied to an existing instance keeps the volumes it was built with.
The last row is the cluster's relational store, not the instance's. A bootstrap does not pass it: state it when you run dcctl install, together with a volume that leaves it room, for example TF_VAR_postgres_max_wal_size_mib=4096 and TF_VAR_postgres_storage=16Gi. It reaches the database through the cluster definition (the database operator does not allow ALTER SYSTEM), changes nothing about running pods, and the plan refuses a value above a quarter of postgres_storage, because the log shares the data volume. The default of 0 leaves PostgreSQL's own 1GB, which at this rate is checkpointed by log volume every minute or two. dcctl does not record postgres_max_wal_size_mib: a later dcctl install re-run without the variable sets it back to PostgreSQL's 1GB. The change is reload-only and puts no data at risk, but state the variable on every install run (for example in terraform.tfvars).
Measured and derived
The shape (the replicas and CPU limits above, and the 12,000 events a second) was measured on GKE with 40 vCPU of platform pools; the rest of the 48 vCPU of that cluster was the load generator. A later run on the same shape held 12,000 events a second with every event accounted for, an acknowledgement p99 of 3.7 ms, no restarts and no checkpoint requested by log volume. The same run read three values off it that have not yet been run as written: device-management's 400m request, event-sources' 256Mi memory limit (81 MiB resident at most), and the relational store's 4096 MiB allowance. The upgrade notes say if a later round moves them.
How long the event store volume lasts
A stored event costs about 756 bytes in the event store, indexes included and before compression (345 bytes in the events table and 454 in the measurements table). At a saturated 12,000 events a second that is 32.7 GB an hour, so the 160Gi volume holds about three and a half hours of continuous ingest after the log allowance and working room. perf-16k sets 260Gi, which holds about four hours at 16,000 events a second (43.6 GB an hour). These are burst and benchmark volumes, not retention. Chunks older than lifecycle.compressAfterDays (7 by default) are compressed, so for sustained use set the data lifecycle and size the volume for it: see Sizing the event store. The DatabaseVolumeFillsWithinDay alert fires when the volume is on course to fill.
What perf-16k sets
perf-16k is the perf-12k shape with the knobs that the 12,000 to 20,000 ramp showed limiting it raised. The replica counts and CPU limits of the services are unchanged; the requests rise with the measured use per pod.
| Service | Replicas | CPU limit | Memory limit | CPU request |
|---|---|---|---|---|
device-management | 2 | 3 | - | 500m |
device-state | 2 | 3 | 512Mi | 300m |
event-management | - | 4 | 1Gi | 450m |
event-processing | - | 2 | 1536Mi | 700m |
event-sources | 3 | 3 | 256Mi | 750m |
event-processing gets 2 CPU and 1536Mi: one CPU was throttled from 16,000 events a second, and 1.5Gi with 2 CPU was the only configuration whose detection lag stayed under the watermark at 20,000.
The broker, the volumes and the write-ahead log allowances:
| Setting | Value |
|---|---|
streamDesignRate | 16000 |
nats_cpu_request | 6 |
nats_memory_request | 6Gi |
nats_memory_limit | 10Gi |
nats_jetstream_storage | 36Gi |
timescale_storage | 260Gi |
timescale_max_wal_size_mib | 40960 |
postgres_max_wal_size_mib | 8192 |
The stream design rate is 16000 and the JetStream volume 36Gi: the reservation for a 15-minute outage at that rate (about 17.5 GiB, and 30 GiB with the fail-closed acknowledgement on) has to fit in it. The broker requests 6 CPU because it used 4.25 cores at 16,000 events a second and 5.2 at 20,000, which is more than the 4 that perf-12k requests. Its memory limit is 10Gi (working set 3.6 GiB at 16,000 and 4.4 GiB at 20,000). The event store allowance is sized so that checkpoints stay on the clock at 20,000 events a second. As above, the last row is the cluster's relational store: state it at install, with TF_VAR_postgres_storage=32Gi beside it.
Measured and derived
The measured run used the perf-12k sizing and held 16,000 events a second exactly, with an acknowledgement p99 of 63 ms (50 ms is the target). Every value in this table is read off that run and has not yet been run as written; the next GKE round confirms it, and the upgrade notes say if it moves.
20,000 events a second
On the perf-12k shape, 20,000 events a second were stored exactly (13.2 million events, none missing, none duplicated) with no connection lost, at an acknowledgement p99 of 245 ms. It is a throughput ceiling: the acknowledgement latency is over its target and the single broker server is the limit (5.2 cores in use, peak 5.5). The perf-16k sizing carries it, but it is not a promise about latency.
What the perf sizes do not set
- Event envelopes. The figures were measured with them on, and they cut CPU by about a fifth.
They are a separate switch, so
dcctl bootstraptells you when aperfsize runs with them off. --ha. The rate was measured on one broker server. The combination is allowed, anddcctl bootstraptells you it has not been measured.- The shared database and the nodes. The relational store's volume, the database CPU and
memory requests, the monitoring volume and the node pools belong to the cluster and stay as
dcctl installand your cluster left them. The relational store'smax_wal_sizeis the same kind of setting: the size publishes the value it was measured with, and you state it at install (see above). The measured cluster used three node pools oft2d-standard-8for services (two nodes),n2d-standard-8for databases (two nodes), and at2d-standard-8for the broker alone, with the load generator elsewhere.
Which value wins
From lowest to highest:
- the chart's defaults and the OpenTofu variables' defaults;
- the size;
- what
dcctlworks out itself: the HA replica counts, the envelope switch, the services that run, the volumes an instance already has; - your
--valuesoverlay; - an OpenTofu variable you state in
TF_VAR_*,terraform.tfvarsor an*.auto.tfvarsfile.
Anything you state wins. dcctl does not pass a size's OpenTofu variable at all when you have
stated it yourself, because OpenTofu would otherwise take the size's value over yours. The line
dcctl bootstrap prints names the size it applied.
Changing the size of a running instance
An instance keeps the size it was bootstrapped with. Moving an existing instance to another size
is not part of this release; until then, state the settings you want in an overlay, which wins
over any size. Whatever the size, a JetStream volume cannot be grown by dcctl: see
how long the event streams can wait out an outage for the procedure.