Scale

Cluster

Use serve() from @axiljs/scale to fork one worker per core, restart crashed workers with backoff and drain gracefully on SIGTERM with node:cluster.

8 min readUpdated DocumentationEdit this page
On this page12

serve() runs an app on every core of a machine with node:cluster, keeps the workers alive, and shuts them down without dropping requests.

TypeScript
TS
1import { Application } from '@axiljs/core'
2import { serve } from '@axiljs/scale'
3
4serve(() => {
5  const app = new Application()
6  app.get('/', (_req, res) => res.success({ ok: true, pid: process.pid }))
7  return app
8}, { port: 3000 })

The first argument is a factory: a function that builds and returns the app (or a Promise of it). serve() calls it in each worker, never in the supervising primary process.

Roles

serve() resolves the scaling config (the scaling option, else axil.scaling.json in the working directory, else defaults, with AXIL_WORKERS on top), then behaves according to the worker count and the current process:

RoleWhenWhat happens
primaryworkers > 1, top-level processPrints host-check warnings, forks one worker per slot, supervises them. Never calls the factory
workerA process forked by the primaryCalls the factory, listens, applies limits, drains on request
singleworkers is 1Prints host-check warnings, calls the factory and listens in-process, with the same graceful shutdown

All workers listen on the same port; node:cluster distributes incoming connections between them.

serve() returns a ServeHandle:

FieldTypeDescription
role'primary' | 'worker' | 'single'Which of the above this process is
workersnumberConfigured worker count (1 for single)
scalingResolvedScalingThe resolved config
findingsFinding[]Startup host-check findings (empty in workers)
shutdown()() => Promise<void>Drain and stop. On the primary, drains every worker

On the primary, the returned promise resolves once every worker is listening.

Options

OptionTypeDefaultDescription
portnumberthe app's defaultPort passed to app.listen()
hoststringthe app's defaultHost passed to app.listen()
scalingScalingConfigaxil.scaling.json, else defaultsScaling config in code
maxRestartsnumber5Crashes allowed per minute before the primary gives up
checkOnStartbooleantruePrint host-check warnings at startup
log(level, message) => voidconsole with an [axil:scale] prefixLogger for supervisor messages; level is 'info', 'warn' or 'error'
onPrimary() => void | (() => void)Runs once in the primary before workers are forked. See below

Worker count

vertical.workers decides how many workers the primary forks: "auto" (default) means one per core as reported by os.availableParallelism(). Override it per environment with AXIL_WORKERS:

Terminal
AXIL_WORKERS=1 node dist/main.js # single process
AXIL_WORKERS=auto node dist/main.js # one per core

Each worker gets a stable slot number from 1 to workers, exposed as the AXIL_WORKER_ID environment variable. A restarted worker reuses its slot. Only worker 1 prints @axiljs/core's startup banner; the others are silenced while they start listening.

Per-worker limits

Every worker applies the vertical settings to its HTTP server:

SettingApplied as
memoryPerWorkerMb--max-old-space-size=<memoryPerWorkerMb> on each forked worker. Skipped if you already pass --max-old-space-size to the primary
maxConnectionsPerWorkerserver.maxConnections
keepAliveTimeoutMs (when set)server.keepAliveTimeout, and server.headersTimeout raised to keepAliveTimeoutMs + 1000 if it is not already higher

Note

The heap flag is added when the primary forks workers. In single mode, nothing is forked, so pass --max-old-space-size to node yourself if you need it.

Crash restart

When a worker dies unexpectedly, the primary forks a replacement in the same slot after a backoff:

Output
TXT
delay = min(10 000 ms, 250 ms × 2^(crashes in the last minute − 1))
        250 ms, 500 ms, 1 s, 2 s, 4 s, ...

If more than maxRestarts crashes (default 5) happen within one minute, the primary treats it as a crash loop: it logs an error, drains the remaining workers and exits with code 1. Your process manager (systemd, Kubernetes) then decides what to do, instead of the primary restarting a broken build forever.

A worker that exits during startup (a throwing factory, a port already in use) also stops everything: serve() drains the other workers and rejects with a worker exited during startup (code N); see its error above.

Graceful shutdown

On SIGTERM or SIGINT, the primary drains every worker:

  1. Sends each worker a shutdown message.
  2. Each worker stops accepting new connections and closes idle keep-alive sockets.
  3. In-flight requests finish. While draining, the worker closes keep-alive sockets as soon as they become idle (checked every 50 ms), so finished connections do not linger until their keep-alive timeout.
  4. After vertical.shutdownTimeoutMs (default 10 s), any connection still open is force-closed.
  5. Each worker exits with code 0 once its server has closed.
  6. Workers still alive shutdownTimeoutMs + 2000 ms after the signal are killed with SIGKILL.
  7. The primary exits.

In single mode, steps 2 to 5 run in-process on SIGTERM/SIGINT.

Workers also drain if the primary disappears (IPC disconnect) or if they receive SIGTERM directly. They ignore SIGINT, so pressing Ctrl+C in a terminal (which signals the whole process group) is coordinated by the primary.

If the app has a close() method (as @axiljs/core's Application does), the drain calls it; otherwise it closes the HTTP server returned by getServer().

Tip

In Kubernetes, set terminationGracePeriodSeconds above shutdownTimeoutMs so the pod is not killed mid-drain. axil deploy sets a grace period that covers the drain. Also make sure the container's entrypoint is node itself (or forwards signals) so SIGTERM reaches the primary.

Shutting down from code

TypeScript
TS
1const handle = await serve(() => createApp(), { port: 3000 })
2
3// later, e.g. from your own admin hook or test teardown
4await handle.shutdown()

shutdown() is idempotent: calling it twice returns the same promise. It drains and stops the server (or every worker, on the primary) but does not call process.exit(); the signal handlers do that.

Primary-side services: onPrimary

Note

New in 0.3.4: the onPrimary option.

Each worker is a separate process, so in-memory state is per worker. onPrimary runs once in the primary, before the first worker is forked, so services that workers talk to over IPC are already listening when the workers start. Use it to install the primary side of cluster rate limiting and the cluster event relay:

TypeScript
TS
1import { serve } from '@axiljs/scale'
2import { installClusterRateLimitPrimary } from '@axiljs/security'
3import { installClusterEventRelay } from '@axiljs/events'
4
5serve(createApp, {
6  port: 3000,
7  onPrimary: () => {
8    const stopRateLimits = installClusterRateLimitPrimary()
9    const stopRelay = installClusterEventRelay()
10    return () => { stopRateLimits(); stopRelay() }
11  }
12})

Inside the app, use rateLimit({ store: new ClusterStore() }) and new EventBus({ transport: new ClusterTransport() }). Both behave like their in-process versions when there is only one worker.

  • onPrimary runs only in cluster mode (primary role). It is not called in workers or in single mode.
  • If it returns a function, that function is called during shutdown, after every worker has stopped.
  • These services share state between the workers of one machine. For several servers, see Running multiple instances.

Startup host checks

Before forking, the primary (or single process) runs hostChecks() and prints every warning and failure, with its fix:

  • workers: "auto" inside a container whose CPU quota is smaller than the visible cores
  • more workers than CPUs (oversubscription)
  • workers × memoryPerWorkerMb above 85% of the machine's (or container's) memory
  • an open-file limit (ulimit -n) below maxConnectionsPerWorker + 256, on Linux

They are also returned in handle.findings. Turn the printing off with checkOnStart: false.

The Listenable interface

serve() does not require @axiljs/core. The factory may return anything with this shape:

TypeScript
TS
1interface Listenable {
2  listen(port?: number, host?: string): Promise<unknown>
3  close?(): Promise<void>
4  getServer?(): import('node:http').Server | null
5}
  • listen() is required.
  • getServer() lets serve() apply maxConnections and keep-alive settings and force-close connections at the drain deadline. Without it, those limits are not applied.
  • close() is preferred for draining; without it, serve() closes the server from getServer().

Project layout

Keep the factory free of side effects at import time, so tests can build the app without forking:

app.ts
TS
1import { Application } from '@axiljs/core'
2
3export function createApp(): Application {
4  const app = new Application()
5  // routes, middleware ...
6  return app
7}
main.ts
TS
1import { serve } from '@axiljs/scale'
2import { createApp } from './app.js'
3
4await serve(() => createApp(), { port: Number(process.env.PORT ?? 3000) })
TypeScript
TS
1// src/app.test.ts: single process, the cluster is never forked
2import { createTestClient } from '@axiljs/testing'
3import { createApp } from './app.js'
4
5const client = createTestClient(createApp())

Everything created inside the factory exists once per worker. Anything kept in memory there (caches, rate-limit counters, schedulers) is multiplied by the worker count. See readiness, and Running multiple instances for sharing that state.

Was this page helpful?

Help improve the documentation

AxilJS is open source and documentation improvements are welcome.

AxilJS DocumentationMIT License · Built by SyntaxilitY