Scale

Capacity Planning

Measure req/s and p99 with a worker-thread load generator, then plan instances, memory and connections with planCapacity() from @axiljs/scale.

10 min readUpdated DocumentationEdit this page
On this page11

axil capacity is built on a small programmatic API that you can use directly, for example in CI or in a custom deployment script.

TypeScript
TS
1import { calibrate, detectSystem, loadScalingConfig, planCapacity, resolveScaling } from '@axiljs/scale'
2
3async function plan() {
4  const system  = detectSystem()
5  const scaling = resolveScaling(loadScalingConfig() ?? {}, system)
6  const cal     = await calibrate({ coreEntry: require.resolve('@axiljs/core') })
7
8  const report = planCapacity({
9    scaling,
10    system,
11    measurement: { rpsPerWorker: cal.rpsPerWorker, p99Ms: cal.latency.p99, source: 'calibrated', label: cal.server }
12  })
13
14  report.verdict     // 'ok' | 'warn' | 'fail'
15  report.required    // { rps, instances } | null
16  report.findings    // [{ level, area, title, detail?, fix? }]
17  return report
18}

The flow is always: describe the host (detectSystem), measure one worker (calibrate or loadTest), plan (planCapacity).

detectSystem()

Returns what this machine can give the app:

FieldTypeDescription
coresnumberCores usable by this process (os.availableParallelism()). Respects CPU affinity, not cgroup quotas
cpuModelstringCPU model name
memoryBytesnumberTotal memory
freeMemoryBytesnumberFree memory
platformNodeJS.Platform'linux', 'darwin', 'win32', ...
nodeVersionstringprocess.version
fdLimitnumber | nullSoft open-file limit from /proc/self/limits (Infinity when unlimited). null outside Linux
cgroupCpusnumber | nullCPU limit from a cgroup v2 quota (/sys/fs/cgroup/cpu.max). null when unlimited, unknown or not Linux
cgroupMemoryBytesnumber | nullcgroup v2 memory limit (/sys/fs/cgroup/memory.max). null when unlimited, unknown or not Linux

calibrate(options?)

Measures the framework ceiling: what one worker process serves on this machine for a hello-world route.

It starts a single-process server in a child process, load-tests it from other cores with loadTest() (below), then stops it.

TypeScript
TS
1const cal = await calibrate({ coreEntry: require.resolve('@axiljs/core'), durationMs: 5000 })
2
3cal.server          // e.g. '@axiljs/core 0.3.6', or 'node:http'
4cal.rpsPerWorker    // req/s of ONE worker
5cal.latency         // { p50, p90, p99, max, mean } in ms
6cal.contended       // true on machines with fewer than 3 cores
7cal.result          // the full LoadTestResult
OptionDescription
coreEntryResolved path to @axiljs/core's entry. The server is an Application with logging off in production mode, answering GET / with res.success({ ok: true }). Omit it to measure a raw node:http server returning the same JSON envelope
any loadTest() option except urldurationMs, warmupMs, connections, threads, ...

The calibration server must print that it is ready within 15 seconds, or calibrate() rejects.

Note

Calibration is a ceiling. Real routes with validation, auth and database calls are slower, often by a large factor. Plan with numbers from your real route (loadTest() against it) whenever you can.

loadTest(options)

A dependency-free HTTP load generator. It opens keep-alive connections across worker_threads (one event loop per thread), records every response latency in a histogram, and reports throughput and percentiles.

TypeScript
TS
1import { loadTest } from '@axiljs/scale'
2
3const r = await loadTest({
4  url: 'http://127.0.0.1:3000/api/users/u_1',
5  durationMs: 10_000,
6  connections: 256,
7  headers: { authorization: `Bearer ${token}` }
8})
9
10console.log(`${r.rps} req/s, p99 ${r.latency.p99} ms, ${r.errors} errors, ${r.non2xx} >= 400`)
OptionTypeDefaultDescription
urlstringrequiredTarget URL, http: or https:. Validated before starting
durationMsnumber5000Measured duration
warmupMsnumber1000Warm-up before measuring; its responses are discarded
connectionsnumber128Total keep-alive connections across all threads (at least one per thread)
threadsnumbermin(4, cores − 1), at least 1Generator threads
methodstring'GET'HTTP method
headersRecord<string, string>{}Request headers
bodystringnoneRequest body

Each connection sends a request, waits for the full response, then sends the next one (closed loop). A network error (refused, reset) is counted and the connection retries after 10 ms instead of spinning.

LoadTestResult

FieldDescription
url, durationMs, connections, threadsThe effective settings
requestsResponses completed during the measured window
rpsrequests ÷ (durationMs ÷ 1000), rounded
non2xxResponses with status >= 400 (despite the name, 3xx responses are not counted)
errorsNetwork errors (refused, reset, timeout)
bytesResponse bytes received
latency{ p50, p90, p99, max, mean } in milliseconds

Latencies go into a histogram of 50 µs buckets covering 5 seconds; slower responses land in the last bucket, so max and the percentiles top out at about 5000 ms. summarize(hist) turns such a histogram (a Uint32Array) into the latency summary and is exported for custom tooling.

Warning

The generator needs CPU too. Load-testing a server on the same machine understates what that server could do on its own hardware. For decisions that matter, run the generator from another machine against a real deployment. When errors or non2xx are not zero, the throughput number is not trustworthy.

planCapacity(input)

A pure function: given the resolved config, the host and a measurement, it decides whether the configured vertical and horizontal scaling can carry the target and says what to change.

TypeScript
TS
1const report = planCapacity({
2  scaling,                         // ResolvedScaling
3  system,                          // SystemInfo
4  measurement: {
5    rpsPerWorker: Math.round(r.rps / 4),   // the measured server ran 4 workers
6    p99Ms: r.latency.p99,
7    source: 'measured',
8    label: r.url
9  },
10  clusterEfficiency: 0.85,         // optional
11  memoryPerConnectionKb: 48        // optional
12})
InputTypeDefaultDescription
scalingResolvedScalingrequiredFrom resolveScaling()
systemSystemInforequiredFrom detectSystem()
measurement.rpsPerWorkernumberrequiredRequests/second ONE worker sustained
measurement.p99Msnumbernonep99 latency observed while measuring
measurement.source'calibrated' | 'measured' | 'assumed'requiredWhere the number came from
measurement.labelstringnoneShown in findings, e.g. the URL or server name
clusterEfficiencynumber0.85Share of linear speed-up kept per extra worker
memoryPerConnectionKbnumber48Memory per open keep-alive connection. WebSockets cost more

If the server you load-tested ran several workers, divide its rps by the worker count to get rpsPerWorker, as axil capacity --server-workers N does.

The model

Output
TXT
instance rps   = rpsPerWorker × min(workers, cpus) × clusterEfficiency   (efficiency only when > 1 worker)
required rps   = target.rps × (1 + headroom)
instances      = ceil(required rps ÷ instance rps)
instance mem   = workers × memoryPerWorkerMb + connections × memoryPerConnectionKb

For the instance, workers is instanceSize.cpus when vertical.workers is "auto", otherwise the configured number. Usable instance memory is 85% of instanceSize.memoryGb. Connections per instance are target.concurrentConnections ÷ instances.max when a target is set, otherwise workers × maxConnectionsPerWorker.

CapacityReport

FieldDescription
verdict'fail' if any finding fails, else 'warn' if any warns, else 'ok'
model{ rpsPerWorker, clusterEfficiency, memoryPerConnectionKb, headroom, source } used for the maths
thisMachine{ cores, workers, rps }: what the current host serves with the configured workers
instance{ cpus, memoryGb, workers, rps, memoryMbNeeded, connections } for one instance of instanceSize
deployment{ minInstances, maxInstances, rpsAtMin, rpsAtMax }
required{ rps, instances } for target plus headroom, or null without target.rps
recommended{ workers, instances: { min, max } }, or null without target.rps
findingsEvery check, as Finding objects

recommended.workers is the instance's CPU count. recommended.instances.max is the required count; min is a third of it, but at least 2 when 2 or more are needed.

Findings

Each Finding has a level ('ok' | 'info' | 'warn' | 'fail'), an area ('throughput' | 'vertical' | 'horizontal' | 'connections' | 'memory' | 'latency' | 'os' | 'measurement'), a title and optionally detail and fix.

CheckFails or warns when
Throughputfail: the required instance count exceeds instances.max. The fix names the count needed, or the CPUs per instance at the current max
Measurementfail: rpsPerWorker is 0
Connectionsfail: concurrentConnections spread over instances.max exceeds maxConnectionsPerWorker per worker
Memoryfail: one instance needs more than 85% of instanceSize.memoryGb
Latencywarn: measured p99 exceeds target.p99Ms (measured at saturation; headroom usually brings it down)
Oversubscribed instancewarn: a fixed workers larger than instanceSize.cpus
Host checkssee below

Informational findings explain the rest: the autoscaler must grow from min to the required count for peak, a single server cannot reach the target, instanceSize was not set, or the numbers come from a calibration rather than your route.

hostChecks(scaling, system)

The subset of checks that need no measurement. serve() runs them at startup and planCapacity() includes them.

LevelConditionFix suggested
warnworkers: "auto" but the container's CPU quota is below the visible coresSet vertical.workers (or AXIL_WORKERS) to the quota
warnMore workers than CPUsworkers: "auto" or fewer workers
warnworkers × memoryPerWorkerMb above 85% of host (or cgroup) memoryFewer workers or a lower memoryPerWorkerMb
failLinux open-file limit below maxConnectionsPerWorker + 256ulimit -n, systemd LimitNOFILE, or Docker --ulimit nofile
infoWindows hostRun capacity checks on the OS you deploy to

Example: fail CI when a route cannot carry the target

capacity-check.ts
TS
1import { detectSystem, fmtNum, loadScalingConfig, loadTest, planCapacity, resolveScaling } from '@axiljs/scale'
2
3const SERVER_WORKERS = 4
4
5async function main() {
6  const system = detectSystem()
7  const scaling = resolveScaling(loadScalingConfig() ?? {}, system)
8
9  const r = await loadTest({ url: 'http://127.0.0.1:3000/api/users/u_1', durationMs: 10_000 })
10  if (r.errors + r.non2xx > 0) throw new Error(`${r.errors} errors, ${r.non2xx} failed responses`)
11
12  const report = planCapacity({
13    scaling,
14    system,
15    measurement: { rpsPerWorker: Math.round(r.rps / SERVER_WORKERS), p99Ms: r.latency.p99, source: 'measured', label: r.url }
16  })
17
18  for (const f of report.findings) {
19    console.log(`[${f.level}] ${f.title}${f.fix ? `  -> ${f.fix}` : ''}`)
20  }
21  if (report.required) {
22    console.log(`needs ${report.required.instances} instances for ${fmtNum(report.required.rps)} req/s`)
23  }
24  process.exitCode = report.verdict === 'fail' ? 1 : 0
25}
26
27main().catch(err => { console.error(err); process.exitCode = 1 })

fmtNum() formats rates the way the report does (1500 as 1.5k, 2000000 as 2M).

The CLI does the same in one command: axil capacity --url http://127.0.0.1:3000/api/users/u_1 --server-workers 4 --json exits with code 1 when the verdict is fail. See axil capacity.

Was this page helpful?

Help improve the documentation

AxilJS is open source and documentation improvements are welcome.

AxilJS DocumentationMIT License · Built by SyntaxilitY