Resilience and Concurrency
How @axiljs/runtime executes tCircuitBreaker, tFallback, tBulkhead, tConcurrency, tMutex, tLock, tDedupe, tHedge, tCache, tRateLimit, tThrottle and tDebounce: options, defaults, key specs, errors and order.
On this page24
- State is per process
- Errors
- Failure handling
- `tCircuitBreaker({ threshold, resetTimeout, halfOpenAttempts })`
- `tFallback({ value })` / `tFallback({ method })`
- `tHedge({ delay, maxAttempts })`
- Concurrency
- `tBulkhead({ name, concurrency, queue })`
- `tConcurrency({ limit, strategy, key })`
- `tMutex({ key })`
- `tLock({ key, ttl, wait })`
- `tDedupe({ key, window })`
- Caching
- `tCache({ key, ttl, tags, condition })`
- `tCachePut`, `tCacheEvict`, `tCacheInvalidate`
- Rate limiting
- `tRateLimit({ limit, window, key })`
- `tThrottle(calls, periodMs)`
- `tDebounce(ms)`
- Key specs
- Interceptor order
- Events
- Testing
- Related
@axiljs/runtime executes the reliability, concurrency, caching and rate-limit decorators from @axiljs/decorator. Put them on a method, wire() the class, and the behaviour applies to every call.
Note
New in 0.3.4: the interceptors on this page. Together with tAudit, tRetry, tTimeout and the intent decorators, the runtime enforces 26 decorators, up from 3. New in 0.3.5: setRateLimitStore().
1import { tCache, tCircuitBreaker, tFallback, tRetry, tTimeout } from '@axiljs/decorator'
2import { wire } from '@axiljs/runtime'
3
4class Pricing {
5 @tCache({ key: 'price:{0}', ttl: 60_000, tags: ['prices'] })
6 @tFallback({ method: 'cachedPrice' })
7 @tRetry({ attempts: 3, delay: 100, retryOn: ['ECONNRESET'] })
8 @tCircuitBreaker({ threshold: 5, resetTimeout: 30_000 })
9 @tTimeout(800)
10 async price(sku: string) {
11 return this.upstream.getPrice(sku)
12 }
13
14 async cachedPrice(sku: string, error: unknown) {
15 return this.lastKnown.get(sku)
16 }
17}
18
19wire(Pricing)The order in which you write the decorators does not matter; the interceptor order does. For price() above: a cache hit returns immediately; on a miss, up to three attempts each pass the circuit breaker and get their own 800 ms timeout; if all fail, cachedPrice(sku, error) supplies the answer.
State is per process
Circuit states, bulkheads, concurrency limits, mutexes, deduplication and debouncing live in the memory of one process. Rate-limit counters, cache entries, locks and idempotency results also default to in-memory, but have pluggable stores. With several workers or instances, install shared stores with useStores().
Errors
All errors below extend axilError from @axiljs/common, so @axiljs/core's default error handler answers with their status and code.
| Error | Status | Code | Thrown by |
|---|---|---|---|
RateLimitExceededError | 429 | RATE_LIMITED | tRateLimit, tThrottle |
CircuitOpenError | 503 | CIRCUIT_OPEN | tCircuitBreaker |
CapacityExceededError | 503 | CAPACITY_EXCEEDED | tBulkhead, tConcurrency |
LockTimeoutError | 409 | LOCK_TIMEOUT | tLock |
DeadlineExceededError | 410 | DEADLINE_EXCEEDED | tDeadline (intent) |
IntentViolationError | 500 | INTENT_VIOLATION | Intent checks in enforce mode |
Every error has a target property (Class.method). RateLimitExceededError also has retryAfterMs, CapacityExceededError has resource, and LockTimeoutError has key and waitMs.
Note
TimeoutError from tTimeout is a plain Error with code: 'ETIMEDOUT', not an axilError, so it reaches the client as a 500 unless your error handler maps it (the high-scale-api template maps it to 504). See Interceptors.
Failure handling
tCircuitBreaker({ threshold, resetTimeout, halfOpenAttempts })
| Option | Type | Default | Description |
|---|---|---|---|
threshold | number | required | Consecutive failures that open the circuit |
resetTimeout | number | 30000 | Time the circuit stays open before probe calls are allowed, ms |
halfOpenAttempts | number | 1 | Probe calls allowed at the same time while half-open |
- Closed: calls run; a success resets the failure count. After
thresholdconsecutive failures the circuit opens. - Open: calls fail immediately with
CircuitOpenErroruntilresetTimeouthas passed. - Half-open: up to
halfOpenAttemptsprobe calls run; further calls getCircuitOpenError. A successful probe closes the circuit, a failed one opens it again.
There is one circuit per method (Class.method), shared by all instances in the process. Each transition emits runtime.circuit with { target, from, to }, and getCircuitStates() returns { 'Pricing.price': 'CLOSED' | 'OPEN' | 'HALF_OPEN' } for every method that has been called.
Because the breaker (order 250) runs inside tRetry (200), every retry attempt counts as a failure: with attempts: 3 and threshold: 5, two failed calls are enough to open it.
tFallback({ value }) / tFallback({ method })
Replaces a failure with a value, or with the result of another method of the same instance. The method receives the original arguments followed by the error:
1class Recommendations {
2 @tFallback({ value: [] })
3 async forUser(id: string) { /* ... */ }
4
5 @tFallback({ method: 'popular' })
6 async personalised(id: string) { /* ... */ }
7
8 async popular(id: string, error: unknown) { return this.top10 }
9}If method does not name a function, the original error is re-thrown. Each fallback emits runtime.fallback with { target, error } (the error message). tFallback (150) runs outside tRetry and tCircuitBreaker, so it handles the final failure, including CircuitOpenError.
tHedge({ delay, maxAttempts })
Starts another attempt when the first is slower than delay ms; the first success wins.
| Option | Type | Default | Description |
|---|---|---|---|
delay | number | required | Wait before starting the next attempt, ms |
maxAttempts | number | 2 | Total attempts, including the first |
If every attempt started so far has failed, the next one starts immediately. The call rejects with the last error when all maxAttempts have failed. Each extra attempt emits runtime.hedge with { target, attempt }. Slower attempts are not cancelled.
Warning
Hedging executes the method more than once. Use it only on reads and other operations that are safe to repeat.
Concurrency
tBulkhead({ name, concurrency, queue })
A named capacity shared by every method that uses the same name, so one slow dependency cannot use up the whole process.
| Option | Type | Default | Description |
|---|---|---|---|
name | string | required | Bulkhead name; methods with the same name share it |
concurrency | number | required | Calls running at the same time |
queue | number | 0 | Calls allowed to wait for a free slot |
When all slots and the queue are full, the call fails with CapacityExceededError (503). The bulkhead is created on first use with that call's concurrency and queue.
1class Reports {
2 @tBulkhead({ name: 'warehouse', concurrency: 4, queue: 20 })
3 async daily() { /* ... */ }
4
5 @tBulkhead({ name: 'warehouse', concurrency: 4, queue: 20 })
6 async monthly() { /* ... */ } // shares the same 4 slots with daily()
7}tConcurrency({ limit, strategy, key })
A limit per method, or per key.
| Option | Type | Default | Description |
|---|---|---|---|
limit | number | required | Calls running at the same time |
strategy | 'queue' | 'reject' | 'drop-oldest' | 'queue' | What happens to calls over the limit |
key | key spec | none (one limit for the method) | Separate limit per key, e.g. 'tenant' |
queue: excess calls wait, without a bound.reject: excess calls fail at once withCapacityExceededError.drop-oldest: up tolimitcalls wait; when the queue is full, the oldest waiting call fails withCapacityExceededErrorand the new one takes its place.
tMutex({ key })
One call at a time per method, or per key (key is a string key spec). Other calls wait in order.
1class Accounts {
2 @tMutex({ key: 'account:{0}' })
3 async transfer(accountId: string, amount: number) { /* ... */ }
4}tMutex is per process. For a lock that spans processes, use tLock with a shared provider.
tLock({ key, ttl, wait })
A keyed lock with a wait timeout and automatic release.
| Option | Type | Default | Description |
|---|---|---|---|
key | key spec | required | Lock key; when it cannot be resolved, the method name is used |
ttl | number | 30000 | Maximum time the lock is held, ms; released automatically after it |
wait | number | 10000 | Maximum time to wait for the lock, ms |
When the lock cannot be acquired within wait, the call fails with LockTimeoutError (409). Lock keys are not scoped to the method: two methods that resolve to the same key share the lock. The default provider locks within the process; useStores({ lock }) or setLockProvider() installs a cross-process provider such as the SQL lease or PostgreSQL advisory lock providers from @axiljs/stores.
Warning
The lock is released after ttl even if the method is still running, so a hung holder cannot block forever. Choose a ttl longer than the slowest normal execution.
tDedupe({ key, window })
Calls with the same key while one is running share that one execution.
| Option | Type | Default | Description |
|---|---|---|---|
key | key spec | required | Calls with the same key are coalesced |
window | number | 0 | Also share a successful result for this many ms after it completes |
Keys are scoped to the method. Failures are never shared after they settle. When the key cannot be resolved, the call runs normally. Unlike tIdempotent, nothing is stored, so tDedupe is for collapsing bursts of identical reads.
Caching
tCache({ key, ttl, tags, condition })
| Option | Type | Default | Description |
|---|---|---|---|
key | key spec | required | Cache key |
ttl | number | required | Time to live, ms |
tags | string[] | [] | Tags for group invalidation |
condition | (result) => boolean | cache everything | Store the result only when it returns true |
A hit returns the stored value without calling the method. Errors are never cached.
Cache keys. A plain string without {, : or an args. prefix is treated as a name, and the key becomes name: followed by the JSON of the arguments, so @tCache({ key: 'revenue', ttl: 60_000 }) caches one entry per argument list. Templates ('price:{0}'), args. paths, header: specs and functions resolve as described under key specs. Cache keys are not scoped to the method, which is what lets tCachePut and tCacheEvict on other methods address the same entries.
tCachePut, tCacheEvict, tCacheInvalidate
| Decorator | After a successful call |
|---|---|
tCachePut({ key, ttl, tags }) | Always runs the method and stores its result under key |
tCacheEvict({ key, tags, allEntries }) | Deletes key and every entry tagged with tags; with allEntries: true, clears the whole cache |
tCacheInvalidate(...tags) | Deletes every entry tagged with any of tags |
Nothing is evicted when the method throws.
1class Products {
2 @tCache({ key: 'product:{0}', ttl: 300_000, tags: ['products'] })
3 async get(id: string) { return this.repo.findById(id) }
4
5 @tCachePut({ key: 'product:{0}', ttl: 300_000, tags: ['products'] })
6 async update(id: string, patch: Partial<Product>) { return this.repo.update(id, patch) }
7
8 @tCacheEvict({ key: 'product:{0}' })
9 async remove(id: string) { await this.repo.delete(id) }
10
11 @tCacheInvalidate('products')
12 async importCatalog(file: string) { /* ... */ }
13}The default store is MemoryCacheStore (up to 10,000 entries per process; the oldest entry is dropped when full). getCacheStore() returns the current store. For a cache shared by every instance, use useStores({ cache }) or setCacheStore().
tCache reports a cache.read side effect, which a tReadOnly method may perform but a tPure method may not. See Intent verification.
Rate limiting
tRateLimit({ limit, window, key })
A fixed-window limit per key.
| Option | Type | Default | Description |
|---|---|---|---|
limit | number | required | Calls allowed per window |
window | string | required | Window length: '500ms', '10s', '1m', '1h', '1d' |
key | string | one shared counter | 'ip', 'user', 'tenant', or another key spec |
1class Exports {
2 @tRateLimit({ limit: 5, window: '1m', key: 'user' })
3 async generate(format: string) { /* ... */ }
4}Over the limit, the call fails with RateLimitExceededError (429), whose message says when to retry (Exports.generate: rate limit exceeded, retry in 42s) and whose retryAfterMs gives the exact time. Each rejection emits runtime.rate_limited with { target, key, retryAfterMs }. Without key, all callers share one counter; when the key cannot be resolved (no user, for example), the caller counts as anonymous. Counters are per method.
tThrottle(calls, periodMs)
At most calls executions per periodMs for the method, across all callers. Calls over the limit are rejected with RateLimitExceededError, not delayed.
1class Sync {
2 @tThrottle(10, 1000) // at most 10 per second
3 async push(batch: unknown[]) { /* ... */ }
4}Both decorators count in memory by default. With several workers or instances each one counts separately, so the effective limit is multiplied. Install a shared counter store with useStores({ rateLimit }) or setRateLimitStore(); the ClusterStore from @axiljs/security and the SQL store from @axiljs/stores both fit.
tDebounce(ms)
Collapses a burst of calls into one. Each call restarts the timer; when no call has arrived for ms, the method runs once with the arguments of the last call, and every caller in the burst receives that result (or that error). Bursts are tracked per instance and method.
1class SearchIndex {
2 @tDebounce(250)
3 async rebuild(reason: string) { /* ... */ }
4}Key specs
Keyed decorators (tIdempotent, tCache, tCachePut, tCacheEvict, tDedupe, tLock, tMutex, tConcurrency, tRateLimit) take a key spec. It is either a function or a string:
| Spec | Resolves to |
|---|---|
(...args) => string | The function's return value, called with the method arguments (and the instance as this) |
'request.id' | The request ID from the request context |
'header:<name>' | A request header, e.g. 'header:idempotency-key' |
'user' | The authenticated user's ID |
'tenant' | The current tenant ID |
'ip' | The client IP |
'args.0.id' | A path into the arguments |
'order:{0.id}' | A template; each {…} is a path into the arguments |
'global' | One shared key for every call |
| any other string | Used literally |
'user.id', 'tenant.id', 'tenantId' and 'requestId' are accepted as aliases. A spec cannot be resolved when the value is missing (no such header, no user, a template path that is undefined); what happens then depends on the decorator: tIdempotent records a violation, tRateLimit counts the caller as anonymous, tLock uses the method name, tConcurrency and tMutex fall back to one limit for the method, and tCache and tDedupe simply run the method. Remember the tCache naming rule for plain strings.
resolveKey(spec, invocation) and parseDuration(value) are exported for custom interceptors.
Interceptor order
Every runtime interceptor has a fixed order. Lower runs outside higher, regardless of the order the decorators are written in.
| Order | Decorator | Page |
|---|---|---|
| 50 | tDeprecated | Intent |
| 60 | tSlo | Intent |
| 70 | tPure, tReadOnly | Intent |
| 75 | tSideEffect | Intent |
| 80 | tTenantIsolated | Intent |
| 90 | tIdempotent | Intent |
| 100 | tAudit | Interceptors |
| 110 | tRateLimit | this page |
| 115 | tThrottle | this page |
| 120 | tDeadline | Intent |
| 125 | tDebounce | this page |
| 130 | tCache | this page |
| 135 | tCachePut, tCacheEvict, tCacheInvalidate | this page |
| 140 | tDedupe | this page |
| 150 | tFallback | this page |
| 200 | tRetry | Interceptors |
| 250 | tCircuitBreaker | this page |
| 260 | tBulkhead | this page |
| 265 | tConcurrency | this page |
| 270 | tMutex | this page |
| 275 | tLock | this page |
| 290 | tHedge | this page |
| 300 | tTimeout | Interceptors |
Consequences worth knowing:
- An idempotent replay (90) returns before rate limits, the cache, retries or locks run.
- A cache hit (130) returns before retries, the breaker and locks, but has already been counted by
tRateLimit(110). tFallbackcatches the final error after all retries.- Each retry attempt passes the circuit breaker, bulkhead, concurrency limit and lock again, and gets its own timeout.
- Each hedged attempt (290) gets its own
tTimeout(300).
enforcedDecorators() returns the registered interceptors in this order, outermost first, including any you register yourself:
1import { enforcedDecorators } from '@axiljs/runtime'
2
3enforcedDecorators()
4// [{ decorator: 'tDeprecated', order: 50 }, { decorator: 'tSlo', order: 60 }, ..., { decorator: 'tTimeout', order: 300 }]axil verify uses this list to tell enforced decorators from metadata-only ones.
Events
| Event | Payload |
|---|---|
runtime.circuit | { target, from, to } |
runtime.fallback | { target, error } |
runtime.hedge | { target, attempt } |
runtime.rate_limited | { target, key, retryAfterMs } |
Events go to the global event bus from @axiljs/events and never break the business call.
Testing
resetResilienceState() clears all in-process resilience state (rate-limit windows, deduplication, circuits, bulkheads, limits, mutexes) and restores the default lock provider, cache store and rate-limit counters. Call it between tests that share decorated classes.
The interceptors are exported individually (circuitBreakerInterceptor, bulkheadInterceptor, cacheInterceptor, …), as are Semaphore and MemoryCacheStore, so you can replace one with registerInterceptor() or reuse its building blocks.
Related
- Interceptors:
wire(),tRetry,tTimeout,tAuditand custom interceptors - Intent verification
- Shared stores
- Reliability decorators and concurrency decorators
Was this page helpful?