Resilience
Module for building a fault-tolerant application using mechanisms such as CircuitBreaker, Fallback, Retry, and Timeout. These mechanisms can be applied declaratively through aspect annotations or directly through manager components when protection is needed in imperative code.
ResilientModule combines CircuitBreakerModule, RetryModule, TimeoutModule, and FallbackModule.
For a step-by-step walkthrough before the reference details, see Resilience.
Dependency¶
Dependency build.gradle:
Module:
Dependency build.gradle.kts:
Module:
CircuitBreaker¶
CircuitBreaker is a proxy that controls the request flow to a particular method
and can temporarily prohibit execution of this method if it throws many exceptions matching the configured filter (CircuitBreakerPredicate).
The purpose of applying CircuitBreaker is to give the system time to correct the error that caused the failure before allowing the application to attempt the operation again.
The CircuitBreaker pattern provides stability while the system recovers from the failure and reduces the impact on performance.
CircuitBreaker can be in one of several states: CLOSED, OPEN, HALF_OPEN.
CLOSED: an application request is passed to the protected operation. The proxy counts recent failures within the configured number of operations (slidingWindowSize) passing through it, and increments this count when the operation does not complete successfully. If the number of requests exceeds the minimum amount required for calculation (minimumRequiredCalls) and the number of recent failures exceeds the configured threshold (failureRateThreshold), the proxy moves toOPEN.OPEN: While in this status, the request from the application immediately terminates with an error and an exception is returned to the application. At this point, the proxy starts a wait timer (waitDurationInOpenState), and when it expires, the proxy moves toHALF_OPEN.HALF_OPEN: a limited number of requests (permittedCallsInHalfOpenState) from the application are allowed to pass through and invoke the operation. If these requests are successful, it is assumed that the error that previously caused the failure has been resolved, andCircuitBreakerenters theCLOSEDstate (the failure counter is reset). If any request terminates with a failure,CircuitBreakerassumes that the fault is still present, so it returns to theOPENstate and restarts the wait time timer (waitDurationInOpenState) to give the system additional time to recover from the failure.
The HALF_OPEN state helps prevent requests to the service from growing rapidly: after recovery starts, the service may be able to handle only a limited number of requests for some time.
Initially it has the CLOSED state.
Declarative usage¶
If CircuitBreaker is in the OPEN state, the call fails with CallNotPermittedException.
Configuration¶
There is a default configuration that is applied to a CircuitBreaker when it is created and then the named settings of a particular CircuitBreaker are applied to override the default settings.
You can change the default settings for all CircuitBreakers at the same time by changing the default configuration.
Example of a complete configuration described in the CircuitBreakerConfig class (example values or default values are indicated):
resilient {
circuitbreaker {
default {
slidingWindowSize = 100 //(1)!
minimumRequiredCalls = 10 //(2)!
failureRateThreshold = 50 //(3)!
waitDurationInOpenState = "25s" //(4)!
permittedCallsInHalfOpenState = 15 //(5)!
enabled = true //(6)!
failurePredicateName = "MyPredicate" //(7)!
}
}
}
- Maximum number of requests used to calculate
failureRateThresholdand determine the state (required, default not specified). - Minimum number of requests required to start state calculation (
required, default not specified). - Percentage of failed requests required to transition to
OPEN; the value must be from1to100(required, default not specified). - Waiting time in
OPEN, after which the transition toHALF_OPENis performed (required, default not specified). - Number of requests in
HALF_OPENthat must complete successfully to transition toCLOSED(required, default not specified). - Enable or disable
CircuitBreaker(default:true). - Exception filter name from
CircuitBreakerPredicate#name()(all errors are recorded by default).
resilient:
circuitbreaker:
default:
slidingWindowSize: 100 #(1)!
minimumRequiredCalls: 10 #(2)!
failureRateThreshold: 50 #(3)!
waitDurationInOpenState: "25s" #(4)!
permittedCallsInHalfOpenState: 15 #(5)!
enabled: true #(6)!
failurePredicateName: "MyPredicate" #(7)!
- Maximum number of requests used to calculate
failureRateThresholdand determine the state (required, default not specified). - Minimum number of requests required to start state calculation (
required, default not specified). - Percentage of failed requests required to transition to
OPEN; the value must be from1to100(required, default not specified). - Waiting time in
OPEN, after which the transition toHALF_OPENis performed (required, default not specified). - Number of requests in
HALF_OPENthat must complete successfully to transition toCLOSED(required, default not specified). - Enable or disable
CircuitBreaker(default:true). - Exception filter name from
CircuitBreakerPredicate#name()(all errors are recorded by default).
An example of overriding named settings for a particular CircuitBreaker:
Constraints
The following are validated at application startup — violating any of them fails the graph build:
failureRateThreshold must be in range 1..100; slidingWindowSize ≥ 1; minimumRequiredCalls ≥ 1 and ≤ slidingWindowSize; permittedCallsInHalfOpenState ≥ 1.
Either the named or the default configuration must be present for every @CircuitBreaker, otherwise startup fails.
Note
Setting enabled = false turns the aspect into a transparent pass-through — the method is invoked directly with no circuit-breaking.
failurePredicateName defaults to KoraCircuitBreakerPredicate (records every error); a custom CircuitBreakerPredicate can be reused by several breakers by referencing its name().
Module metrics are described in the Metrics Reference section.
Exception filtering¶
In order to register which errors should be recorded as CircuitBreaker errors, you can override the default filter,
you need to implement CircuitBreakerPredicate and register your component in the context and specify in the CircuitBreaker configuration its name returned in the name() method.
CircuitBreaker records all errors by default.
Configuration:
- Exception filter name from
CircuitBreakerPredicate#name()(all errors are recorded by default).
Imperative usage¶
You can use a breaker in imperative code: inject CircuitBreakerManager
and get CircuitBreaker from it by the configuration name that would be specified in the annotation:
@Component
public final class SomeService {
private final CircuitBreakerManager manager;
public SomeService(CircuitBreakerManager manager) {
this.manager = manager;
}
public String doWork() {
var circuitBreaker = manager.get("custom");
return circuitBreaker.accept(this::doSomeWork);
}
private String doSomeWork() {
// do some work
}
}
To return a fallback value instead of throwing CallNotPermittedException when the breaker is OPEN, use the accept overload that takes a second Supplier:
When the protected call cannot be wrapped in a single Supplier, acquire and release the permit manually.
Call acquire() (throws CallNotPermittedException when the breaker is OPEN, or HALF_OPEN with no test calls left) to obtain a permit, then always report the outcome with releaseOnSuccess() or releaseOnError(Throwable) — otherwise the breaker leaks a permit and its accounting becomes incorrect:
public String doWork() {
var circuitBreaker = manager.get("custom");
circuitBreaker.acquire(); // throws CallNotPermittedException when the call is not permitted
try {
var result = doSomeWork();
circuitBreaker.releaseOnSuccess();
return result;
} catch (Throwable e) {
circuitBreaker.releaseOnError(e);
throw e;
}
}
fun doWork(): String {
val circuitBreaker = manager["custom"]
circuitBreaker.acquire() // throws CallNotPermittedException when the call is not permitted
try {
val result = doSomeWork()
circuitBreaker.releaseOnSuccess()
return result
} catch (e: Throwable) {
circuitBreaker.releaseOnError(e)
throw e
}
}
tryAcquire() is the non-throwing alternative: it returns false when the call is not permitted, so you can branch without catching CallNotPermittedException.
When acquire() does throw, the current breaker state (OPEN or HALF_OPEN) is available via CallNotPermittedException#state().
Retry¶
Retry provides the ability to configure repeated invocation of annotated methods.
It allows you to specify when a method should be retried and configure retry parameters when the method throws an exception matching the configured filter (RetryPredicate).
Declarative usage¶
If all attempts are exhausted, the call fails with RetryExhaustedException.
Configuration¶
There is a default configuration that is applied to Retry at creation
and then the named settings of a particular Retry are applied to override the default settings.
It is possible to change the default settings for all Retry at the same time by changing the default configuration.
Example of the complete configuration described in the RetryConfig class (default or example values are specified):
resilient {
retry {
default {
delay = "100ms" //(1)!
attempts = 2 //(2)!
delayStep = "100ms" //(3)!
enabled = true //(4)!
failurePredicateName = "MyPredicate" //(5)!
}
}
}
- Initial delay before a repeated call (
required, default not specified). - Number of retry attempts (
required, default not specified). - Delay increment for subsequent attempts (default:
0). - Enable or disable
Retry(default:true). - Exception filter name from
RetryPredicate#name()(all errors are recorded by default).
resilient:
retry:
default:
delay: "100ms" #(1)!
attempts: 2 #(2)!
delayStep: "100ms" #(3)!
enabled: true #(4)!
failurePredicateName: "MyPredicate" #(5)!
- Initial delay before a repeated call (
required, default not specified). - Number of retry attempts (
required, default not specified). - Delay increment for subsequent attempts (default:
0). - Enable or disable
Retry(default:true). - Exception filter name from
RetryPredicate#name()(all errors are recorded by default).
Constraints & delay progression
delay and attempts are required (resolved from the named or default config) and attempts must be ≥ 0; a missing delay/attempts or a negative attempts fails application startup.
attempts counts the retries after the initial call, so attempts = 2 allows up to 3 executions in total.
Each retry waits delayStep (default 0) longer than the previous one, so the delays are delay, delay + delayStep, delay + 2·delayStep, … .
Note
Setting enabled = false turns @Retry into a transparent pass-through (the method runs once).
failurePredicateName defaults to KoraRetryPredicate (retries on every error); a custom RetryPredicate can be reused by several retriers by referencing its name().
Exception filtering¶
In order to register which errors should be recorded as errors on the Retry side, you can override the default filter,
it is required to implement RetryPredicate and register its component in the context and specify in the Retry configuration its name returned in the name() method.
Retry records all errors by default.
Configuration:
- Exception filter name from
RetryPredicate#name()(all errors are recorded by default).
Imperative usage¶
You can use a retrier in imperative code: inject RetryManager
and get Retry from it by the configuration name that would be specified in the annotation:
To return a fallback value instead of throwing RetryExhaustedException once all attempts are used up, pass a second Supplier:
For asynchronous imperative code there is an overload that retries a Supplier<CompletionStage<T>> and returns a CompletionStage<T>, scheduling each attempt after the configured delay without blocking the calling thread.
Manual retry state¶
For full control over the retry loop, use retry.asState(), which returns a RetryState.
It is AutoCloseable, so wrap it in try-with-resources (Java) or use (Kotlin) to record metrics on completion.
On each caught exception call onException(Throwable), which returns a RetryStatus:
ACCEPTED— another attempt is allowed; calldoDelay()(blocks for the current backoff) and retry.REJECTED— the exception was rejected by theRetryPredicateand must not be retried; rethrow it.EXHAUSTED— all attempts are used up; throwRetryExhaustedException(or fall back to a default).
getAttempts() / getAttemptsMax() report progress and getDelayNanos() returns the next delay.
public String doWork() {
var retry = manager.get("custom");
try (var state = retry.asState()) {
while (true) {
try {
return doSomeWork();
} catch (Exception e) {
switch (state.onException(e)) {
case ACCEPTED -> state.doDelay(); // wait, then loop and retry
case REJECTED -> throw e; // not retryable
case EXHAUSTED -> throw new RetryExhaustedException("custom", state.getAttemptsMax(), e);
}
}
}
}
}
fun doWork(): String {
val retry = manager["custom"]
retry.asState().use { state ->
while (true) {
try {
return doSomeWork()
} catch (e: Exception) {
when (state.onException(e)) {
Retry.RetryState.RetryStatus.ACCEPTED -> state.doDelay() // wait, then loop and retry
Retry.RetryState.RetryStatus.REJECTED -> throw e // not retryable
Retry.RetryState.RetryStatus.EXHAUSTED -> throw RetryExhaustedException("custom", state.attemptsMax, e)
}
}
}
}
}
Timeout¶
Timeout sets the maximum execution time of the annotated method.
Declarative usage¶
If the method does not complete within duration, the call fails with TimeoutExhaustedException.
Configuration¶
There is a default configuration that is applied to the Timeout when it is created
and then the named settings of a particular Timeout are applied to override the default settings.
It is possible to change the default settings for all Timeouts at the same time by changing the default configuration.
Example of the complete configuration described in the TimeoutConfig class (default or example values are specified):
- Operation time limit after which
TimeoutExhaustedExceptionwill be thrown (required, default not specified). - Enable or disable
Timeout(default:true).
Note
duration is required (resolved from the named or default config) and startup fails without it.
Setting enabled = false turns @Timeout into a transparent pass-through — the method runs with no time limit.
Imperative usage¶
You can use a time limiter in imperative code: inject TimeoutManager
and get Timeout from it by the configuration name that would be specified in the annotation:
@Component
public final class SomeService {
private final TimeoutManager manager;
public SomeService(TimeoutManager manager) {
this.manager = manager;
}
public String doWork() {
var timeout = manager.get("custom");
return timeout.execute(this::doSomeWork);
}
private String doSomeWork() {
// do some work
}
}
Timeout also exposes execute(Runnable) for operations that return nothing, and timeout() returns the configured Duration:
Fallback¶
Fallback allows you to specify a method that will be called when an exception thrown by the annotated method matches the configured filters (FallbackPredicate).
The fallback method must match the return type of the annotated method.
Declarative usage¶
An example of a backup method with no arguments:
Example for Fallback with arguments:
@Component
public class SomeService {
@Fallback(value = "custom", method = "getFallback(arg3, arg1)") // Passes the arguments of the annotated method in the specified order to the Fallback method
public String getValue(String arg1, Integer arg2, Long arg3) {
return "value";
}
protected String getFallback(Long argLong, String argString) {
return "fallback";
}
}
@Component
open class SomeService {
// Passes the arguments of the annotated method in the specified order to the Fallback method
@Fallback(value = "custom", method = "getFallback(arg3, arg1)")
fun getValue(arg1: String, arg2: Int, arg3: Long): String = "value"
fun getFallback(argLong: Long, argString: String): String = "fallback"
}
Configuration¶
There is a default configuration that is applied to the Fallback at creation
and then the named settings of a particular Fallback are applied to override the defaults.
It is possible to change the default settings for all Fallbacks at the same time by changing the default configuration.
Example of the complete configuration described in the FallbackConfig class (default or example values are specified):
Note
Unlike the other aspects, @Fallback has no required properties — with no configuration it uses defaults.
Setting enabled = false disables the fallback so the original exception propagates.
failurePredicateName defaults to KoraFallbackPredicate (triggers the fallback for every error); a custom FallbackPredicate can be reused by several fallbacks by referencing its name().
Exception filtering¶
In order to register which errors should be recorded as Fallback errors, you can override the default filter,
you need to implement FallbackPredicate and register your component in the context and specify in the Fallback configuration its name returned in the name() method.
Fallback records all errors by default.
Imperative usage¶
You can use the fallback method in imperative code: inject FallbackManager
and get Fallback from it by the configuration name that would be specified in the annotation:
@Component
public final class SomeService {
private final FallbackManager manager;
public SomeService(FallbackManager manager) {
this.manager = manager;
}
public String doWork() {
var fallback = manager.get("custom");
return fallback.fallback(this::doSomeWork, () -> "BackupValue");
}
private String doSomeWork() {
// do some work
}
}
For operations that return nothing, use the Runnable overload; and canFallback(Throwable) reports whether a given exception would trigger the fallback according to the configured FallbackPredicate:
var fallback = manager.get("custom");
// canFallback tells whether the exception would trigger the fallback
if (fallback.canFallback(exception)) {
// exception matches the configured FallbackPredicate
}
// Runnable variant for operations that return nothing
fallback.fallback(
() -> { /* primary action */ },
() -> { /* fallback action */ });
val fallback = manager["custom"]
// canFallback tells whether the exception would trigger the fallback
if (fallback.canFallback(exception)) {
// exception matches the configured FallbackPredicate
}
// Runnable variant for operations that return nothing
fallback.fallback(Runnable { /* primary action */ }, Runnable { /* fallback action */ })
Combination¶
It is possible to combine all of the above annotations simultaneously over a single method.
The order in which the annotations are applied depends on the order in which the annotations are declared. You can change the order as you wish and combine it with other annotations that are also applied in the order of declaration.
@Component
public class SomeService {
@Fallback(value = "default", method = "getFallback(arg1)") // 4
@CircuitBreaker("default") // 3
@Retry("default") // 2
@Timeout("default") // 1
public String getValueSync(String arg1) {
return "result-" + arg1;
}
protected String getFallback(String arg1) { // 4
return "fallback-" + arg1;
}
}
@Component
open class SomeService {
@Fallback(value = "default", method = "getFallback(arg1)") // 4
@CircuitBreaker("default") // 3
@Retry("default") // 2
@Timeout("default") // 1
fun getValueSync(arg1: String): String = "result-$arg1"
protected fun getFallback(arg1: String): String = "fallback-$arg1" // 4
}
In the example above:
@Timeoutis applied and checks that the method does not run longer than the time specified in the configuration.@Retryis applied and attempts to repeat method execution the configured number of times if the method throws an exception in the chain, including an exception from@Timeout.@CircuitBreakeris applied and works according to its configuration and state, depending on the successful method result or an exception in the chain, including exceptions from@Timeoutand@Retry.@Fallbackis applied and calls thegetFallbackmethod with thearg1argument if the method throws an exception in the chain, including exceptions from@Timeout,@Retry, and@CircuitBreaker.
Aspect invocation order follows the annotation order on the method: from top to bottom.
Example configuration for all aspects:
Exceptions¶
All resilience exceptions extend ru.tinkoff.kora.resilient.ResilientException (a RuntimeException), which exposes name() — the configuration name of the aspect that raised it.
| Exception | Thrown by | Additional API |
|---|---|---|
ResilientException |
base type for all of the below | name() |
CallNotPermittedException |
@CircuitBreaker / CircuitBreaker#acquire() when the breaker is OPEN, or HALF_OPEN with no test calls left |
state() returns the CircuitBreaker.State (OPEN / HALF_OPEN) |
RetryExhaustedException |
@Retry / Retry#retry(...) when every attempt failed |
name(); message carries the number of attempts, the last failure is the getCause() |
TimeoutExhaustedException |
@Timeout / Timeout#execute(...) when the method exceeds duration |
name() |
Description — resilience aspects signal failure by throwing one of these unchecked exceptions out of the protected method.
Causes
CallNotPermittedException— the circuit breaker is short-circuiting calls because the failure rate reachedfailureRateThreshold; the call was rejected without invoking the method.RetryExhaustedException— the method kept throwing a retryable exception untilattemptswas reached; the underlying failure is available viagetCause().TimeoutExhaustedException— the method did not complete withinduration.
Recommendations
- Catch
ResilientExceptionto handle any resilience failure uniformly, or catch the concrete type when the handling differs. - When aspects are combined, a downstream aspect's exception propagates up the chain: e.g. a
TimeoutExhaustedExceptionfrom@Timeoutis observed by@Retry, then@CircuitBreaker, and finally@Fallback. Prefer a@Fallbackmethod or an imperative fallback over turning these into user-facing errors.
Handling example:
try {
return service.value()
} catch (e: CallNotPermittedException) {
log.warn("CircuitBreaker '{}' is {}", e.name(), e.state())
return cachedValue()
} catch (e: ResilientException) { // TimeoutExhaustedException, RetryExhaustedException, ...
log.warn("Resilient '{}' failed", e.name(), e)
return cachedValue()
}
Signatures¶
Available method signatures supported by these annotations out of the box: All four annotations support regular synchronous methods, asynchronous types, and reactive types, but the actual set depends on the language and processor.
Class must be non final in order for aspects to work.
T means the return value type.
void myMethod()T myMethod()Optional<T> myMethod()CompletionStage<T> myMethod()/CompletableFuture<T> myMethod()(CompletionStage)Mono<T> myMethod()(Project Reactor, requires dependency)Flux<T> myMethod()(Project Reactor, requires dependency)
Class must be open in order for aspects to work.
By T we mean the type of the return value, either T?, or Unit.
myMethod(): Tsuspend myMethod(): T(Kotlin Coroutines, requires dependency asimplementation)myMethod(): Flow<T>(Kotlin Coroutines, requires dependency asimplementation)