Scale

Configuration

Every axil.scaling.json field with type, default and validation rule: target, vertical workers and limits, horizontal instances, headroom, AXIL_WORKERS.

7 min readUpdated DocumentationEdit this page
On this page12

Scaling is described in one file, axil.scaling.json, at the project root. It is read by serve(), axil capacity and axil deploy, so the process that runs, the capacity check and the deployment manifest all agree.

JSON
JSON
1{
2  "target":     { "rps": 1500000, "concurrentConnections": 2000000, "p99Ms": 50 },
3  "vertical":   { "workers": "auto", "maxConnectionsPerWorker": 20000, "memoryPerWorkerMb": 512,
4                  "shutdownTimeoutMs": 10000, "keepAliveTimeoutMs": 65000 },
5  "horizontal": { "instances": { "min": 6, "max": 18 }, "instanceSize": { "cpus": 8, "memoryGb": 16 } },
6  "headroom": 0.3
7}

Every field is optional. A missing file means "all defaults".

Terminal
❯axil capacity --init # write a starter axil.scaling.json

target

What the whole deployment must sustain at peak. Used by the capacity planner only; serve() ignores it.

FieldTypeDefaultDescription
rpsnumbernot setPeak requests per second across all instances
concurrentConnectionsnumbernot setPeak open connections (roughly concurrent users with keep-alive or WebSockets)
p99Msnumbernot setLatency budget for the 99th percentile, ms

Each value must be a positive number when present. Without target.rps, axil capacity reports what the deployment can carry but gives no pass/fail verdict on throughput.

vertical

How each instance uses its cores. Applied by serve() to every worker.

FieldTypeDefaultDescription
workerspositive integer or "auto""auto"Worker processes per instance. "auto" is one per CPU core
maxConnectionsPerWorkerpositive integer10000Open connections one worker accepts (server.maxConnections)
memoryPerWorkerMbpositive number512V8 heap limit per worker (--max-old-space-size)
shutdownTimeoutMspositive integer10000Grace period for in-flight requests on shutdown
keepAliveTimeoutMspositive integerNode's default (5000)Idle keep-alive timeout (server.keepAliveTimeout)

workers and AXIL_WORKERS

The AXIL_WORKERS environment variable overrides vertical.workers, so one file works across machine sizes:

Terminal
AXIL_WORKERS=1 npm run dev # one process while developing
AXIL_WORKERS=auto npm start # one worker per core
AXIL_WORKERS=4 npm start # exactly four

An empty AXIL_WORKERS is ignored. Any other value that is not auto or a positive integer fails validation.

"auto" uses os.availableParallelism(): the cores this process may run on. It does not see container CPU quotas (cgroup limits). In a container limited to 2 CPUs on a 32-core node, "auto" starts 32 workers that compete for 2 CPUs. serve() and axil capacity warn about this at startup. Set AXIL_WORKERS to the container's CPU count; axil deploy does this for you by pinning AXIL_WORKERS to the pod's CPUs.

keepAliveTimeoutMs behind a load balancer

Load balancers keep idle connections to your instances open and reuse them. If Node.js closes an idle socket at the same moment the balancer sends a new request on it, the balancer answers the client with a 502. These show up as rare, random 502s under load.

The fix is to keep sockets open longer than the balancer does:

Load balancerDefault idle timeoutSuggested keepAliveTimeoutMs
AWS Application Load Balancer60 s65000

Node's default keep-alive timeout is 5 s, far below that. When keepAliveTimeoutMs is set, serve() also raises server.headersTimeout to at least keepAliveTimeoutMs + 1000, because Node may drop reused sockets when the headers timeout is not above the keep-alive timeout.

horizontal

How many instances run, and how big each is. Used by the capacity planner and axil deploy; serve() only runs one instance.

FieldTypeDefaultDescription
instancespositive integer, or { "min": n, "max": n }1A fixed count, or autoscaling bounds
instanceSize.cpuspositive numberthis machine's coresCPUs per instance (VM or pod)
instanceSize.memoryGbpositive numberthis machine's memoryMemory per instance in GB
  • With a fixed number, min and max are both that number.
  • With an object, min defaults to 1, max defaults to min, both must be positive integers, and max must be >= min.
  • When instanceSize is omitted, the planner assumes instances like the machine it runs on and says so in an info finding. When instanceSize is present but one of its fields is missing, that field defaults to 1.

axil deploy uses instances.min as the replica count, instances.max as the autoscaler ceiling and instanceSize for pod resources.

headroom

FieldTypeDefaultRule
headroomnumber0.30 up to (not including) 5

Spare capacity kept above target.rps: the planner sizes for target.rps × (1 + headroom). The default 30% covers traffic spikes, rolling deploys and losing an instance. At saturation latency climbs steeply, so running instances cooler is what keeps p99 inside budget.

Validation

Values are validated when the config is resolved (at serve() startup and in axil capacity). A bad value throws with the field name:

Output
TXT
[@axiljs/scale] vertical.workers must be a positive integer, got 0
[@axiljs/scale] horizontal.instances.max must be >= min
[@axiljs/scale] headroom must be between 0 and 5
[@axiljs/scale] axil.scaling.json is not valid JSON: Unexpected token ...

Warning

0 is not "unset". "target": { "rps": 0 } fails validation; leave the field out instead. JSON also has no comments: put notes in your README, not in axil.scaling.json.

Configuration in code

All of this is also available programmatically:

TypeScript
TS
1import { defineScaling, detectSystem, loadScalingConfig, resolveScaling, serve } from '@axiljs/scale'
2
3// Typed config in code (identity helper)
4const scaling = defineScaling({
5  vertical: { workers: 'auto', keepAliveTimeoutMs: 65000 },
6  horizontal: { instances: { min: 2, max: 8 }, instanceSize: { cpus: 4, memoryGb: 8 } }
7})
8
9serve(() => createApp(), { port: 3000, scaling })     // instead of axil.scaling.json
10
11// Read and resolve the file yourself
12const raw = loadScalingConfig()                        // undefined when the file is absent
13const resolved = resolveScaling(raw ?? {}, detectSystem())
14resolved.vertical.workers                              // a number; 'auto' already resolved
ExportDescription
defineScaling(config)Returns config unchanged, typed as ScalingConfig
loadScalingConfig(dir?)Reads axil.scaling.json from dir (default process.cwd()). Returns undefined when absent, throws on invalid JSON
resolveScaling(config?, machine?)Fills defaults, applies AXIL_WORKERS, validates, returns ResolvedScaling. machine is { cores, memoryBytes }; pass detectSystem()
SCALING_FILE'axil.scaling.json'
SCALING_TEMPLATEThe starter config written by axil capacity --init

Note

Pass detectSystem() as the second argument to resolveScaling(). Without it, the default instance memory falls back to 1 GB, because the built-in default machine reports no memory.

ResolvedScaling

TypeScript
TS
1{
2  target: { rps: number; concurrentConnections: number; p99Ms: number }   // 0 = not set
3  vertical: {
4    workers: number
5    workersAuto: boolean
6    maxConnectionsPerWorker: number
7    memoryPerWorkerMb: number
8    shutdownTimeoutMs: number
9    keepAliveTimeoutMs?: number
10  }
11  horizontal: {
12    min: number
13    max: number
14    instanceSize: { cpus: number; memoryGb: number }
15    instanceSizeDefaulted: boolean
16  }
17  headroom: number
18}

Starter template

SCALING_TEMPLATE, as written by axil capacity --init:

JSON
JSON
1{
2  "target":     { "rps": 10000, "concurrentConnections": 20000, "p99Ms": 100 },
3  "vertical":   { "workers": "auto", "maxConnectionsPerWorker": 10000, "memoryPerWorkerMb": 512 },
4  "horizontal": { "instances": { "min": 2, "max": 6 }, "instanceSize": { "cpus": 4, "memoryGb": 8 } },
5  "headroom": 0.3
6}

Docker images

serve() reads axil.scaling.json from the working directory. If your image copies only dist/, also copy the file, otherwise containers run with defaults (no keepAliveTimeoutMs, default limits):

Dockerfile
DOCKER
COPY dist ./dist
COPY axil.scaling.json ./

Was this page helpful?

Help improve the documentation

AxilJS is open source and documentation improvements are welcome.

AxilJS DocumentationMIT License · Built by SyntaxilitY