Resilience

Module for building a fault-tolerant application using mechanisms such as CircuitBreaker, Retry, Timeout, RateLimiter and Fallback.

Every mechanism except Fallback is described by a specification interface: a typed contract that points at a configuration path. The annotation on the protected method references that interface, so the binding between a method and its resilience settings is checked by the compiler rather than by a string.

ResilientModule combines CircuitBreakerModule, RetryModule, TimeoutModule, FallbackModule and RateLimiterModule.

For a step-by-step walkthrough before the reference details, see Resilience.

Dependency¶

implementation "io.koraframework:resilient-kora"

@KoraApp
public interface Application extends ResilientModule { }

implementation("io.koraframework:resilient-kora")

@KoraApp
interface Application : ResilientModule

The annotation processor (annotation-processors) or the KSP processor (symbol-processors) is required: it both generates the specification implementations and applies the aspects.

Distributed state¶

By default every limiter and retry budget keeps its state in the memory of one application instance. The distributed rate limiter and the distributed retry budget share that state between instances through Redis. They need an extra module backed by the Lettuce client:

implementation "io.koraframework:resilient-kora-distributed-redis-lettuce"

@KoraApp
public interface Application extends ResilientModule, LettuceDistributedResilientModule { }

implementation("io.koraframework:resilient-kora-distributed-redis-lettuce")

@KoraApp
interface Application : ResilientModule, LettuceDistributedResilientModule

The artifact brings resilient-kora-distributed (the backend-agnostic DistributedRateLimiterClient and DistributedRetryBudgetClient contracts, the algorithms and @RateLimiterDistributedSpec) and redis-lettuce. LettuceDistributedResilientModule extends LettuceModule, so the connection is configured in the lettuce section exactly as for the Redis cache. Standalone and cluster clients are both supported, and all operations are plain Redis commands without server-side Lua scripts.

To give the resilience components a Redis client of their own instead of the shared one, register an AbstractRedisClient tagged with @Tag(LettuceDistributedResilientModule.class).

Specifications¶

A specification is an interface that extends a resilience contract and carries the annotation with the configuration path:

The method annotations live in the annotation sub-package of each contract package, for example io.koraframework.resilient.circuitbreaker.annotation.CircuitBreakable.

@RateLimited also accepts an interface annotated with @RateLimiterDistributedSpec (io.koraframework.resilient.distributed.ratelimiter.annotation), which shares its limit across instances — see Distributed rate limiter.

@CircuitBreakerSpec("resilient.circuitbreaker.pet") //(1)!
public interface PetCircuitBreaker extends CircuitBreaker { }

@RetrySpec("resilient.retry.pet")
public interface PetRetry extends Retry { }

@TimeoutSpec("resilient.timeout.pet")
public interface PetTimeouter extends Timeouter { }

@Component
public class PetService {

    @CircuitBreakable(PetCircuitBreaker.class) //(2)!
    @Retryable(PetRetry.class)
    @Timeout(PetTimeouter.class)
    public Optional<Pet> findById(long id) {
        return petRepository.findById(id);
    }
}

@CircuitBreakerSpec("resilient.circuitbreaker.pet") //(1)!
interface PetCircuitBreaker : CircuitBreaker

@RetrySpec("resilient.retry.pet")
interface PetRetry : Retry

@TimeoutSpec("resilient.timeout.pet")
interface PetTimeouter : Timeouter

@Component
open class PetService {

    @CircuitBreakable(PetCircuitBreaker::class) //(2)!
    @Retryable(PetRetry::class)
    @Timeout(PetTimeouter::class)
    open fun findById(id: Long): Pet? = petRepository.findById(id)
}

What the processor does with a specification:

• generates its implementation and a module that publishes it, and the module is picked up by @KoraApp automatically — nothing has to be wired by hand;
• publishes the specification interface itself as an application graph component, so it can be injected for imperative use;
• reads the configuration from exactly the path given in the annotation.

The generated implementations inherit a toString() that reports their current state — the breaker state and its counters, the permits left in a local rate limiter, the retry settings and the retry budget tokens — so logging the injected specification or inspecting it in a debugger shows what it is doing right now.

Common compile-time errors:

• @CircuitBreakerSpec can only be applied to an interface — the annotation was placed on a class or a record.
• @CircuitBreakerSpec annotated interface 'X' must extend io.koraframework.resilient.circuitbreaker.CircuitBreaker — the interface does not extend the contract.
• config path can't be blank — the annotation value is an empty string.
• @CircuitBreakable on 'X#y()' references an invalid resilient component type — the class passed to the method annotation does not implement the expected contract.

CircuitBreaker¶

CircuitBreaker is a proxy that controls the request flow to a particular method and can temporarily prohibit execution of this method if it throws many exceptions matching the configured filter.

The purpose of applying CircuitBreaker is to give the system time to correct the error that caused the failure before allowing the application to attempt the operation again. The CircuitBreaker pattern provides stability while the system recovers from the failure and reduces the impact on performance. CircuitBreaker can be in one of several states: CLOSED, OPEN, HALF_OPEN.

• CLOSED: an application request is passed to the protected operation. The proxy counts recent failures within the configured number of operations (countBased.windowSize) passing through it, and increments this count when the operation does not complete successfully. If the number of requests exceeds the minimum amount required for calculation (minimumRequiredCalls) and the number of recent failures exceeds the configured threshold (failureRateThreshold), the proxy moves to OPEN.
• OPEN: While in this status, the request from the application immediately terminates with an error and an exception is returned to the application. At this point, the proxy starts a wait timer (waitDurationInOpenState), and when it expires, the proxy moves to HALF_OPEN.
• HALF_OPEN: a limited number of requests (permittedCallsInHalfOpenState) from the application are allowed to pass through and invoke the operation. If these requests are successful, it is assumed that the error that previously caused the failure has been resolved, and CircuitBreaker enters the CLOSED state (the failure counter is reset). If any request terminates with a failure, CircuitBreaker assumes that the fault is still present, so it returns to the OPEN state and restarts the wait time timer (waitDurationInOpenState) to give the system additional time to recover from the failure.

The HALF_OPEN state helps prevent requests to the service from growing rapidly: after recovery starts, the service may be able to handle only a limited number of requests for some time.

Initially it has the CLOSED state.

Declarative usage¶

If CircuitBreaker is in the OPEN state, the call fails with CallNotPermittedException.

@CircuitBreakerSpec("resilient.circuitbreaker.custom")
public interface CustomCircuitBreaker extends CircuitBreaker { }

@Component
public class SomeService {

    @CircuitBreakable(CustomCircuitBreaker.class)
    public String getValue() {
        throw new IllegalStateException("Ops");
    }
}

@CircuitBreakerSpec("resilient.circuitbreaker.custom")
interface CustomCircuitBreaker : CircuitBreaker

@Component
open class SomeService {

    @CircuitBreakable(CustomCircuitBreaker::class)
    open fun value(): String = throw IllegalStateException("Ops")
}

Configuration¶

The section that @CircuitBreakerSpec points at is described by the CircuitBreakerConfig class:

resilient {
    circuitbreaker {
        custom {
            type = STRIPED_APPROX //(1)!
            failureRateThreshold = 50 //(2)!
            minimumRequiredCalls = 10 //(3)!
            waitDurationInOpenState = "25s" //(4)!
            permittedCallsInHalfOpenState = 15 //(5)!
            enabled = true //(6)!
            countBased {
                windowSize = 100 //(7)!
                stripedApprox {
                    stripes = 16 //(8)!
                }
            }
        }
    }
}

resilient:
  circuitbreaker:
    custom:
      type: STRIPED_APPROX #(1)!
      failureRateThreshold: 50 #(2)!
      minimumRequiredCalls: 10 #(3)!
      waitDurationInOpenState: "25s" #(4)!
      permittedCallsInHalfOpenState: 15 #(5)!
      enabled: true #(6)!
      countBased:
        windowSize: 100 #(7)!
        stripedApprox:
          stripes: 16 #(8)!

The telemetry key inside the same section overrides the module-wide settings described in Telemetry.

Module metrics are described in the Metrics Reference section.

Implementations¶

type selects how the CLOSED-state statistics are collected. The state machine itself is identical and strictly atomic in all four.

TIME_BASED ignores countBased and reads its own section instead:

resilient {
    circuitbreaker {
        custom {
            type = TIME_BASED
            failureRateThreshold = 50
            minimumRequiredCalls = 10
            waitDurationInOpenState = "25s"
            permittedCallsInHalfOpenState = 15
            timeBased {
                windowDuration = "10s" //(1)!
                sampleCount = 16 //(2)!
                counterStripes = 16 //(3)!
                counterType = ATOMIC //(4)!
            }
        }
    }
}

resilient:
  circuitbreaker:
    custom:
      type: TIME_BASED
      failureRateThreshold: 50
      minimumRequiredCalls: 10
      waitDurationInOpenState: "25s"
      permittedCallsInHalfOpenState: 15
      timeBased:
        windowDuration: "10s" #(1)!
        sampleCount: 16 #(2)!
        counterStripes: 16 #(3)!
        counterType: ATOMIC #(4)!

Exception filtering¶

CircuitBreaker records every error as a failure by default, except exceptions that implement the NonCircuitableException marker interface. Marking your own exception type is the simplest way to keep an expected business error away from the breaker, and the same type can also implement NonRetryableException to opt out of Retry:

public final class InvalidPetException extends RuntimeException
    implements NonCircuitableException, NonRetryableException { //(1)!

    public InvalidPetException(String message) {
        super(message);
    }
}

class InvalidPetException(message: String) : RuntimeException(message),
    NonCircuitableException, NonRetryableException //(1)!

When the decision cannot be expressed through the exception type, there are two ways to change the default.

The first one is to override isFailure right on the specification interface — no extra component and no configuration:

@CircuitBreakerSpec("resilient.circuitbreaker.custom")
public interface CustomCircuitBreaker extends CircuitBreaker {

    @Override
    default boolean isFailure(Throwable throwable) {
        return !(throwable instanceof HttpServerResponseException e) || e.code() >= 500;
    }
}

@CircuitBreakerSpec("resilient.circuitbreaker.custom")
interface CustomCircuitBreaker : CircuitBreaker {

    override fun isFailure(throwable: Throwable): Boolean =
        throwable !is HttpServerResponseException || throwable.code() >= 500
}

The second way is a CircuitBreakerPredicate component bound to the specification with @Tag. It takes precedence over isFailure and is the right choice when the filter itself needs dependencies:

@Tag(CustomCircuitBreaker.class) //(1)!
@Component
public final class MyFailurePredicate implements CircuitBreakerPredicate {

    @Override
    public boolean isCircuitBreakerFailure(Throwable throwable) { //(2)!
        return !(throwable instanceof HttpServerResponseException e) || e.code() >= 500;
    }
}

@Tag(CustomCircuitBreaker::class) //(1)!
@Component
class MyFailurePredicate : CircuitBreakerPredicate {

    override fun isCircuitBreakerFailure(throwable: Throwable): Boolean = //(2)!
        throwable !is HttpServerResponseException || throwable.code() >= 500
}

An exception that the filter rejects is neither counted as a failure nor as a success: it is simply ignored by the breaker and propagates to the caller unchanged.

Imperative usage¶

The specification interface is an ordinary application graph component, so a breaker can be used in imperative code by injecting it directly:

@Component
public final class SomeService {

    private final CustomCircuitBreaker circuitBreaker;

    public SomeService(CustomCircuitBreaker circuitBreaker) {
        this.circuitBreaker = circuitBreaker;
    }

    public String doWork() {
        return circuitBreaker.accept(this::doSomeWork);
    }

    private String doSomeWork() {
        // do some work
    }
}

@Component
class SomeService(private val circuitBreaker: CustomCircuitBreaker) {

    fun doWork(): String {
        return circuitBreaker.accept(ThrowableCallable { doSomeWork() })
    }

    private fun doSomeWork(): String {
        // do some work
    }
}

The accept methods take ThrowableCallable<T, E> or ThrowableRunnable<E> from io.koraframework.resilient.common, so the protected code may throw checked exceptions.

To return a fallback value instead of throwing CallNotPermittedException when the breaker is OPEN, use the accept overload that takes a second callable:

public String doWork() {
    return circuitBreaker.accept(this::doSomeWork, () -> "fallback");
}

fun doWork(): String {
    return circuitBreaker.accept(ThrowableCallable { doSomeWork() }, ThrowableCallable { "fallback" })
}

When the protected call cannot be wrapped in a single callable, acquire and release the permit manually. Call acquire() (throws CallNotPermittedException when the breaker is OPEN, or HALF_OPEN with no test calls left) to obtain a permit, then always report the outcome with releaseOnSuccess() or releaseOnError(Throwable) — otherwise the breaker leaks a permit and its accounting becomes incorrect:

public String doWork() {
    circuitBreaker.acquire(); // throws CallNotPermittedException when the call is not permitted
    try {
        var result = doSomeWork();
        circuitBreaker.releaseOnSuccess();
        return result;
    } catch (Throwable e) {
        circuitBreaker.releaseOnError(e);
        throw e;
    }
}

fun doWork(): String {
    circuitBreaker.acquire() // throws CallNotPermittedException when the call is not permitted
    try {
        val result = doSomeWork()
        circuitBreaker.releaseOnSuccess()
        return result
    } catch (e: Throwable) {
        circuitBreaker.releaseOnError(e)
        throw e
    }
}

tryAcquire() is the non-throwing alternative: it returns false when the call is not permitted, so you can branch without catching CallNotPermittedException. When acquire() does throw, the current breaker state (OPEN or HALF_OPEN) is available via CallNotPermittedException#state().

Retry¶

Retry provides the ability to configure repeated invocation of annotated methods. It allows you to specify when a method should be retried and configure retry parameters when the method throws an exception matching the configured filter.

Declarative usage¶

If all attempts are exhausted, the call fails with RetryExhaustedException.

@RetrySpec("resilient.retry.custom")
public interface CustomRetry extends Retry { }

@Component
public class SomeService {

    @Retryable(CustomRetry.class)
    public String getValue() {
        throw new IllegalStateException("Ops");
    }
}

@RetrySpec("resilient.retry.custom")
interface CustomRetry : Retry

@Component
open class SomeService {

    @Retryable(CustomRetry::class)
    open fun execute(arg: String): Unit = throw IllegalStateException("Ops")
}

Configuration¶

The section that @RetrySpec points at is described by the RetryConfig class:

resilient {
    retry {
        custom {
            delay = "100ms" //(1)!
            attempts = 2 //(2)!
            delayStep = "100ms" //(3)!
            enabled = true //(4)!
        }
    }
}

resilient:
  retry:
    custom:
      delay: "100ms" #(1)!
      attempts: 2 #(2)!
      delayStep: "100ms" #(3)!
      enabled: true #(4)!

The optional backoff, jitter and retryBudget sections are described in Backoff and jitter and Retry budget, and the telemetry key overrides the settings from Telemetry.

Backoff and jitter¶

The backoff section replaces the linear delayStep progression with an exponential one, and jitter spreads the delays of concurrent callers so they do not retry in lockstep:

resilient {
    retry {
        custom {
            delay = "100ms"
            attempts = 4
            backoff {
                type = EXPONENTIAL //(1)!
                multiplier = 2.0 //(2)!
                delayMax = "5s" //(3)!
            }
            jitter {
                type = FULL //(4)!
                ratio = 1.0 //(5)!
            }
        }
    }
}

resilient:
  retry:
    custom:
      delay: "100ms"
      attempts: 4
      backoff:
        type: EXPONENTIAL #(1)!
        multiplier: 2.0 #(2)!
        delayMax: "5s" #(3)!
      jitter:
        type: FULL #(4)!
        ratio: 1.0 #(5)!

With the settings above the computed delay for attempt n is delay * multiplier^(n-1), capped at delayMax: 100ms, 200ms, 400ms, 800ms. Jitter then picks the actual delay uniformly from [computed - computed * ratio, computed], so ratio = 1.0 means anything from 0 up to the computed delay.

Retry budget¶

A retry budget caps how much extra load retries may add. It is a token bucket: every retry takes one token, every successful call returns ratio tokens, and when the bucket is empty the retry is refused and the original exception is rethrown as is — without waiting and without RetryExhaustedException.

resilient {
    retry {
        custom {
            delay = "100ms"
            attempts = 3
            retryBudget {
                enabled = true //(1)!
                ratio = 0.1 //(2)!
                tokensMax = 100 //(3)!
                tokensInitial = 10 //(4)!
                minTokensPerSecond = 0.0 //(5)!
            }
        }
    }
}

resilient:
  retry:
    custom:
      delay: "100ms"
      attempts: 3
      retryBudget:
        enabled: true #(1)!
        ratio: 0.1 #(2)!
        tokensMax: 100 #(3)!
        tokensInitial: 10 #(4)!
        minTokensPerSecond: 0.0 #(5)!

Custom and distributed budget¶

The budget itself is a RetryBudget (tryAcquireRetryToken(), onSuccess(), availableTokens()), and every retry specification obtains its instance from a RetryBudgetFactory component through get(name, config), where name is the simple name of the specification interface and config is its RetryConfig. A factory that returns null leaves the retry without a budget. RetryModule registers DefaultRetryBudgetFactory, which builds the in-memory KoraRetryBudget from the retryBudget section, so by default every application instance has a bucket of its own.

A factory is replaced the usual way: an untagged RetryBudgetFactory component overrides the default one for every retry, and a factory tagged with a specification, @Tag(PetRetry.class), is used for that retry only and takes precedence over the untagged one.

DistributedRetryBudgetFactory from the distributed module keeps one token balance per specification in Redis, so the budget caps the retries of all instances together. It is not registered automatically:

@KoraApp
public interface Application extends ResilientModule, LettuceDistributedResilientModule {

    default RetryBudgetFactory distributedRetryBudgetFactory(DistributedRetryBudgetClient client) { //(1)!
        return new DistributedRetryBudgetFactory(client);
    }

    @Tag(PetRetry.class) //(2)!
    default RetryBudgetFactory petRetryBudgetFactory(DistributedRetryBudgetClient client) {
        return new DistributedRetryBudgetFactory(client, "pets:retrybudget", Duration.ofMinutes(30)); //(3)!
    }
}

@KoraApp
interface Application : ResilientModule, LettuceDistributedResilientModule {

    fun distributedRetryBudgetFactory(client: DistributedRetryBudgetClient): RetryBudgetFactory = //(1)!
        DistributedRetryBudgetFactory(client)

    @Tag(PetRetry::class) //(2)!
    fun petRetryBudgetFactory(client: DistributedRetryBudgetClient): RetryBudgetFactory =
        DistributedRetryBudgetFactory(client, "pets:retrybudget", Duration.ofMinutes(30)) //(3)!
}

The balance is stored under <keyPrefix>:<specification simple name>, and every change refreshes the key expiry, so an unused budget disappears from Redis and starts again from tokensInitial. Services that point at the same Redis and use specifications with the same simple name share one balance unless their key prefixes differ.

Exception filtering¶

Retry retries on every error by default, except exceptions that implement the NonRetryableException marker interface (see the example in the circuit breaker section). The two ways to narrow that down further mirror the circuit breaker: override isFailure on the specification, or register a RetryPredicate component tagged with the specification. The tagged component wins when both are present, and either of them replaces the default NonRetryableException check.

@RetrySpec("resilient.retry.custom")
public interface CustomRetry extends Retry {

    @Override
    default boolean isFailure(Throwable throwable) {
        return throwable instanceof IOException;
    }
}

@Tag(CustomRetry.class)
@Component
public final class MyRetryPredicate implements RetryPredicate {

    @Override
    public boolean isRetryFailure(Throwable throwable) {
        return throwable instanceof IOException;
    }
}

@RetrySpec("resilient.retry.custom")
interface CustomRetry : Retry {

    override fun isFailure(throwable: Throwable): Boolean = throwable is IOException
}

@Tag(CustomRetry::class)
@Component
class MyRetryPredicate : RetryPredicate {

    override fun isRetryFailure(throwable: Throwable): Boolean = throwable is IOException
}

An exception the filter rejects is rethrown immediately, without further attempts and without being wrapped in RetryExhaustedException.

Imperative usage¶

Inject the specification interface and call it directly:

@Component
public final class SomeService {

    private final CustomRetry retry;

    public SomeService(CustomRetry retry) {
        this.retry = retry;
    }

    public String doWork() {
        return retry.retry(this::doSomeWork);
    }

    private String doSomeWork() {
        // do some work
    }
}

@Component
class SomeService(private val retry: CustomRetry) {

    fun doWork(): String {
        return retry.retry(ThrowableCallable { doSomeWork() })
    }

    private fun doSomeWork(): String {
        // do some work
    }
}

To return a fallback value instead of throwing RetryExhaustedException once all attempts are used up, pass a second callable:

return retry.retry(this::doSomeWork, () -> "fallback");

return retry.retry(ThrowableCallable { doSomeWork() }, ThrowableCallable { "fallback" })

For asynchronous imperative code there is an overload that retries a Supplier<CompletionStage<T>> and returns a CompletionStage<T>, scheduling each attempt after the configured delay without blocking the calling thread.

Manual retry state¶

For full control over the retry loop, use retry.asState(), which returns a Retry.RetryState. It is AutoCloseable, so wrap it in try-with-resources (Java) or use (Kotlin) to record metrics on completion. On each caught exception call onException(Throwable), which returns a RetryStatus:

• ACCEPTED — another attempt is allowed; call doDelay() (blocks for the current backoff) and retry.
• REJECTED — the exception was rejected by the filter or the retry budget is empty, and it must not be retried; rethrow it.
• EXHAUSTED — all attempts are used up; throw RetryExhaustedException (or fall back to a default).

getAttempts() / getAttemptsMax() report progress and getDelayNanos() returns the next delay.

public String doWork() {
    try (var state = retry.asState()) {
        while (true) {
            try {
                return doSomeWork();
            } catch (Exception e) {
                switch (state.onException(e)) {
                    case ACCEPTED -> state.doDelay();   // wait, then loop and retry
                    case REJECTED -> throw e;           // not retryable
                    case EXHAUSTED -> throw new RetryExhaustedException("custom", state.getAttemptsMax(), e);
                }
            }
        }
    }
}

fun doWork(): String {
    retry.asState().use { state ->
        while (true) {
            try {
                return doSomeWork()
            } catch (e: Exception) {
                when (state.onException(e)) {
                    Retry.RetryState.RetryStatus.ACCEPTED -> state.doDelay()   // wait, then loop and retry
                    Retry.RetryState.RetryStatus.REJECTED -> throw e           // not retryable
                    Retry.RetryState.RetryStatus.EXHAUSTED -> throw RetryExhaustedException("custom", state.attemptsMax, e)
                }
            }
        }
    }
}

Timeout¶

Timeout sets the maximum execution time of the annotated method. For synchronous methods the call is offloaded to a virtual thread and interrupted once the limit is reached; for Kotlin suspend functions the coroutine is cancelled by withTimeout.

Declarative usage¶

If the method does not complete within duration, the call fails with TimeoutExhaustedException.

@TimeoutSpec("resilient.timeout.custom")
public interface CustomTimeouter extends Timeouter { }

@Component
public class SomeService {

    @Timeout(CustomTimeouter.class)
    public String getValue() {
        try {
            Thread.sleep(3000);
            return "OK";
        } catch (InterruptedException e) {
            throw new IllegalStateException(e);
        }
    }
}

@TimeoutSpec("resilient.timeout.custom")
interface CustomTimeouter : Timeouter

@Component
open class SomeService {

    @Timeout(CustomTimeouter::class)
    open fun value(): String = try {
        Thread.sleep(3000)
        "OK"
    } catch (e: InterruptedException) {
        throw IllegalStateException(e)
    }
}

Configuration¶

The section that @TimeoutSpec points at is described by the TimeoutConfig class:

resilient {
    timeout {
        custom {
            duration = "1s" //(1)!
            enabled = true //(2)!
        }
    }
}

resilient:
  timeout:
    custom:
      duration: "1s" #(1)!
      enabled: true #(2)!

The telemetry key inside the same section overrides the module-wide settings described in Telemetry.

Imperative usage¶

Inject the specification interface and call it directly:

@Component
public final class SomeService {

    private final CustomTimeouter timeouter;

    public SomeService(CustomTimeouter timeouter) {
        this.timeouter = timeouter;
    }

    public String doWork() {
        return timeouter.execute(this::doSomeWork);
    }

    private String doSomeWork() {
        // do some work
    }
}

@Component
class SomeService(private val timeouter: CustomTimeouter) {

    fun doWork(): String {
        return timeouter.execute(ThrowableCallable { doSomeWork() })
    }

    private fun doSomeWork(): String {
        // do some work
    }
}

Timeouter also exposes an execute overload for operations that return nothing, and timeout() returns the configured Duration:

Duration limit = timeouter.timeout();  // configured duration
timeouter.execute(this::cleanup);      // void operation, throws TimeoutExhaustedException on timeout

val limit: Duration = timeouter.timeout()           // configured duration
timeouter.execute(ThrowableRunnable { cleanup() })  // void operation, throws TimeoutExhaustedException on timeout

RateLimiter¶

RateLimiter caps how many times a method may be called within a period: limitForPeriod permits per limitRefreshPeriod. How the permits are handed out is chosen by the algorithm, a token bucket by default. Acquiring a permit never blocks — a call that finds no permit available fails immediately with RateLimitExceededException.

Declarative usage¶

@RateLimiterSpec("resilient.ratelimiter.custom")
public interface CustomRateLimiter extends RateLimiter { }

@Component
public class SomeService {

    @RateLimited(CustomRateLimiter.class)
    public String getValue() {
        return "OK";
    }
}

@RateLimiterSpec("resilient.ratelimiter.custom")
interface CustomRateLimiter : RateLimiter

@Component
open class SomeService {

    @RateLimited(CustomRateLimiter::class)
    open fun value(): String = "OK"
}

Configuration¶

The section that @RateLimiterSpec points at is described by the RateLimiterConfig class:

resilient {
    ratelimiter {
        custom {
            limitForPeriod = 100 //(1)!
            limitRefreshPeriod = "1s" //(2)!
            type = TOKEN_BUCKET //(3)!
            enabled = true //(4)!
        }
    }
}

resilient:
  ratelimiter:
    custom:
      limitForPeriod: 100 #(1)!
      limitRefreshPeriod: "1s" #(2)!
      type: TOKEN_BUCKET #(3)!
      enabled: true #(4)!

The telemetry key inside the same section overrides the module-wide settings described in Telemetry.

Algorithms¶

The type key selects one of two lock-free implementations, each of which keeps its whole state in a single atomic value:

FIXED_WINDOW packs the counter into 24 bits, so its limitForPeriod must stay below 16777216.

Imperative usage¶

Inject the specification interface and call it directly:

@Component
public final class SomeService {

    private final CustomRateLimiter rateLimiter;

    public SomeService(CustomRateLimiter rateLimiter) {
        this.rateLimiter = rateLimiter;
    }

    public String doWork() {
        return rateLimiter.execute(this::doSomeWork); //(1)!
    }

    public boolean doWorkIfPermitted() {
        if (!rateLimiter.tryAcquire()) { //(2)!
            return false;
        }
        doSomeWork();
        return true;
    }

    private String doSomeWork() {
        // do some work
    }
}

@Component
class SomeService(private val rateLimiter: CustomRateLimiter) {

    fun doWork(): String {
        return rateLimiter.execute(ThrowableCallable { doSomeWork() }) //(1)!
    }

    fun doWorkIfPermitted(): Boolean {
        if (!rateLimiter.tryAcquire()) { //(2)!
            return false
        }
        doSomeWork()
        return true
    }

    private fun doSomeWork(): String {
        // do some work
    }
}

acquire() takes a permit without running anything and throws RateLimitExceededException when there is none left.

Distributed rate limiter¶

A local limiter counts only the calls of its own instance. When the limit belongs to the whole service — a partner API quota, for example — declare the specification with @RateLimiterDistributedSpec instead of @RateLimiterSpec: the limit is then enforced through Redis for all instances together. The distributed module has to be connected.

@RateLimiterDistributedSpec("resilient.ratelimiter.partner") //(1)!
public interface PartnerRateLimiter extends RateLimiter { }

@Component
public class PartnerService {

    @RateLimited(PartnerRateLimiter.class) //(2)!
    public String getValue() {
        return "OK";
    }
}

@RateLimiterDistributedSpec("resilient.ratelimiter.partner") //(1)!
interface PartnerRateLimiter : RateLimiter

@Component
open class PartnerService {

    @RateLimited(PartnerRateLimiter::class) //(2)!
    open fun value(): String = "OK"
}

The section is described by DistributedRateLimiterConfig:

resilient {
    ratelimiter {
        partner {
            limitForPeriod = 100 //(1)!
            limitRefreshPeriod = "1s" //(2)!
            keyPrefix = "pets" //(3)!
            algorithm = TOKEN_BUCKET //(4)!
            enabled = true //(5)!
        }
    }
}

resilient:
  ratelimiter:
    partner:
      limitForPeriod: 100 #(1)!
      limitRefreshPeriod: "1s" #(2)!
      keyPrefix: "pets" #(3)!
      algorithm: TOKEN_BUCKET #(4)!
      enabled: true #(5)!

The telemetry key and the resilient.telemetry.rateLimiter settings apply exactly as for a local limiter.

How it works:

• The generated implementation extends KoraDistributedRateLimiter and requires a DistributedRateLimiterClient in the graph; LettuceDistributedResilientModule provides one.
• The limiter state lives under the key <keyPrefix>:<specification simple name>. FIXED_WINDOW uses a separate key per window, <keyPrefix>:<specification simple name>:<window number>, where windows are aligned to the wall clock rather than to the start of the application. Keys expire on their own once they are no longer needed.
• TOKEN_BUCKET takes the current time from the wall clock of the calling instance, so clock skew between instances makes the limit approximate. It is also slightly approximate right after an idle period, and exact in the steady state.
• Every permit costs synchronous Redis commands, so it adds a network round trip to the protected call. A Redis error is not swallowed: it propagates out of @RateLimited to the caller.

Fallback¶

Fallback specifies a method that is called when the annotated method fails. Unlike the other mechanisms it has no specification interface and no configuration section of its own: the fallback method is the whole contract.

The fallback method must match the return type of the annotated method and must be declared in the same class.

Declarative usage¶

An example of a backup method with no arguments:

@Component
public class SomeService {

    @Fallback(method = "getFallback()")
    public String getValue() {
        return "value";
    }

    protected String getFallback() {
        return "fallback";
    }
}

@Component
open class SomeService {

    @Fallback(method = "getFallback()")
    open fun value(): String = "value"

    fun getFallback(): String = "fallback"
}

Example for Fallback with arguments:

@Component
public class SomeService {

    @Fallback(method = "getFallback(arg3, arg1)")     // Passes the arguments of the annotated method in the specified order to the Fallback method
    public String getValue(String arg1, Integer arg2, Long arg3) {
        return "value";
    }

    protected String getFallback(Long argLong, String argString) {
        return "fallback";
    }
}

@Component
open class SomeService {

    // Passes the arguments of the annotated method in the specified order to the Fallback method
    @Fallback(method = "getFallback(arg3, arg1)")
    open fun getValue(arg1: String, arg2: Int, arg3: Long): String = "value"

    fun getFallback(argLong: Long, argString: String): String = "fallback"
}

The reference is validated at compile time. Common errors:

• @Fallback method reference '…' has invalid syntax — the value must look like name() or name(arg1, arg2).
• @Fallback method reference '…' uses unknown source arguments — an argument that the annotated method does not declare.
• @Fallback method '…' was not found — no method with that name in the same class.
• @Fallback method '…' does not match requested signature — the fallback method must accept exactly the referenced arguments plus an optional @Fallback.Reason parameter.

Exception filtering¶

By default the fallback is invoked for every Throwable. Adding a single @Fallback.Reason parameter to the fallback method both delivers the exception that triggered the fallback and narrows down what triggers it: an exception that is not an instance of the declared parameter type is rethrown and the fallback is not called.

@Component
public class SomeService {

    @Fallback(method = "getFallback()")
    public String getValue() {
        throw new IllegalStateException("Ops");
    }

    protected String getFallback(@Fallback.Reason RuntimeException reason) { //(1)!
        return "fallback: " + reason.getMessage();
    }
}

@Component
open class SomeService {

    @Fallback(method = "getFallback()")
    open fun value(): String = throw IllegalStateException("Ops")

    fun getFallback(@Fallback.Reason reason: RuntimeException): String = //(1)!
        "fallback: " + reason.message
}

In Java the parameter type must match what the annotated method can throw: RuntimeException when it declares no throws, Exception when it declares checked exceptions, and Throwable when it declares throws Throwable. A narrower type is reported as a compile-time error, so use a plain instanceof check inside the fallback for finer filtering.

Telemetry¶

Logging, metrics and tracing are configured per mechanism under resilient.telemetry, and every specification may override the module-wide values through a telemetry key inside its own configuration section:

resilient {
    telemetry {
        circuitBreaker { //(1)!
            logging.enabled = false //(2)!
            metrics {
                enabled = false //(3)!
                tags { "service" = "pets" } //(4)!
            }
            tracing {
                enabled = false //(5)!
                attributes { "component" = "resilient" } //(6)!
            }
        }
        retry {}
        timeout {}
        fallback {}
        rateLimiter {}
    }
    circuitbreaker {
        custom {
            telemetry.metrics.enabled = true //(7)!
        }
    }
}

resilient:
  telemetry:
    circuitBreaker: #(1)!
      logging:
        enabled: false #(2)!
      metrics:
        enabled: false #(3)!
        tags:
          service: "pets" #(4)!
      tracing:
        enabled: false #(5)!
        attributes:
          component: "resilient" #(6)!
    retry: {}
    timeout: {}
    fallback: {}
    rateLimiter: {}
  circuitbreaker:
    custom:
      telemetry:
        metrics:
          enabled: true #(7)!

The whole resilient.telemetry block is optional — omitting it leaves every mechanism with the defaults above. The metric names are listed in the Metrics Reference.

Combination¶

It is possible to combine all of the above annotations simultaneously over a single method.

The order in which the annotations are applied depends on the order in which the annotations are declared. You can change the order as you wish and combine it with other annotations that are also applied in the order of declaration.

@Component
public class SomeService {

    @Fallback(method = "getFallback(arg1)")           // 4
    @CircuitBreakable(CustomCircuitBreaker.class)     // 3
    @Retryable(CustomRetry.class)                     // 2
    @Timeout(CustomTimeouter.class)                   // 1
    public String getValueSync(String arg1) {
        return "result-" + arg1;
    }

    protected String getFallback(String arg1) {       // 4
        return "fallback-" + arg1;
    }
}

@Component
open class SomeService {

    @Fallback(method = "getFallback(arg1)")             // 4
    @CircuitBreakable(CustomCircuitBreaker::class)      // 3
    @Retryable(CustomRetry::class)                      // 2
    @Timeout(CustomTimeouter::class)                    // 1
    open fun getValueSync(arg1: String): String = "result-$arg1"

    protected fun getFallback(arg1: String): String = "fallback-$arg1"  // 4
}

In the example above:

1. @Timeout is applied and checks that the method does not run longer than the time specified in the configuration.
2. @Retryable is applied and attempts to repeat method execution the configured number of times if the method throws an exception in the chain, including an exception from @Timeout.
3. @CircuitBreakable is applied and works according to its configuration and state, depending on the successful method result or an exception in the chain, including exceptions from @Timeout and @Retryable.
4. @Fallback is applied and calls the getFallback method with the arg1 argument if the method throws an exception in the chain, including exceptions from @Timeout, @Retryable, and @CircuitBreakable.

Aspect invocation order follows the annotation order on the method: from top to bottom, so the topmost annotation is the outermost wrapper. @RateLimited fits into the same chain and is usually placed above @CircuitBreakable, so rejected calls never reach the breaker.

Example configuration for all aspects:

resilient {
    circuitbreaker {
        custom {
            type = FIXED_WINDOW
            countBased.windowSize = 1
            minimumRequiredCalls = 1
            failureRateThreshold = 100
            permittedCallsInHalfOpenState = 1
            waitDurationInOpenState = "1s"
        }
    }
    timeout {
        custom {
            duration = "300ms"
        }
    }
    retry {
        custom {
            delay = "100ms"
            attempts = 2
        }
    }
    ratelimiter {
        custom {
            limitForPeriod = 100
            limitRefreshPeriod = "1s"
        }
    }
}

resilient:
  circuitbreaker:
    custom:
      type: FIXED_WINDOW
      countBased:
        windowSize: 1
      minimumRequiredCalls: 1
      failureRateThreshold: 100
      permittedCallsInHalfOpenState: 1
      waitDurationInOpenState: "1s"
  timeout:
    custom:
      duration: "300ms"
  retry:
    custom:
      delay: "100ms"
      attempts: 2
  ratelimiter:
    custom:
      limitForPeriod: 100
      limitRefreshPeriod: "1s"

Policy ordering in practice¶

The following imperative composition makes the outer/inner boundaries explicit: