Metrics

Module for collecting application metrics using Micrometer. It creates a PrometheusMeterRegistry, connects Kora component metrics to it, and exposes the result in the Prometheus format through the private HTTP server. This lets you collect application, JVM, process, and built-in integration metrics in one place and scrape them with an external observability system.

Publishing metrics requires the private HTTP server, which exposes them in the Prometheus format.

For a step-by-step walkthrough before the reference details, see Observability.

Dependency

Java Kotlin

Dependency build.gradle:

implementation "ru.tinkoff.kora:micrometer-module"

Module:

@KoraApp
public interface Application extends MetricsModule { }

Dependency build.gradle.kts:

implementation("ru.tinkoff.kora:micrometer-module")

Module:

@KoraApp
interface Application : MetricsModule

Configuration

Example of private HTTP server path configuration for retrieving metrics described in the HttpServerConfig class (default values are specified):

Hocon YAML

httpServer {
    privateApiHttpMetricsPath = "/metrics" //(1)!
}

1. Path for retrieving metrics in the Prometheus format (default: "/metrics").

httpServer:
  privateApiHttpMetricsPath: "/metrics" #(1)!

1. Path for retrieving metrics in the Prometheus format (default: "/metrics").

Example of the complete configuration described in the MetricsConfig class (default values are specified):

Hocon YAML

metrics {
    opentelemetrySpec = "V120" //(1)!
}

1. Metrics format according to the OpenTelemetry standard (available values: V120 / V123, default: V120).

metrics:
  opentelemetrySpec: "V120" #(1)!

1. Metrics format according to the OpenTelemetry standard (available values: V120 / V123, default: V120).

Module Metrics

The metrics block above configures the registry globally. Each metric-collecting module additionally exposes a per-module telemetry.metrics block described in TelemetryConfig.MetricsConfig, letting you toggle metrics, tune histogram buckets, and attach extra tags for that module only. The example below uses the HTTP Server module as the host, but the same telemetry.metrics fields apply verbatim to HTTP Client, Database, Kafka, gRPC Server, gRPC Client, Scheduling, Cache, and every other integration that reports metrics:

Hocon YAML

httpServer {
    telemetry {
        metrics {
            enabled = true //(1)!
            slo = [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] //(2)!
            tags { //(3)!
                "key1" = "value1"
                "key2" = "value2"
            }
        }
    }
}

1. Enables metrics collection for the module (default: true)
2. SLO histogram buckets for DistributionSummary/Timer metrics (default: ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO in milliseconds for V120 / #DEFAULT_SLO_V123 in seconds for V123)
3. Extra common tags added to every metric the module reports (default: {})

httpServer:
  telemetry:
    metrics:
      enabled: true #(1)!
      slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(2)!
      tags: #(3)!
        key1: value1
        key2: value2

1. Enables metrics collection for the module (default: true)
2. SLO histogram buckets for DistributionSummary/Timer metrics (default: ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO in milliseconds for V120 / #DEFAULT_SLO_V123 in seconds for V123)
3. Extra common tags added to every metric the module reports (default: {})

Setting enabled = false disables metric creation for that module entirely (the module's MetricsFactory returns no metrics), which is the recommended way to silence a noisy integration. The default slo bucket values per standard are listed in the Personalization section.

Metrics collection configuration parameters are also described in the modules that collect metrics: HTTP Server, HTTP Client, gRPC Server, gRPC Client, Scheduling, Cache, and other integrations.

Usage

Kora follows the notation described in the Prometheus specification.

After the module is connected, PrometheusMeterRegistry is registered in Metrics.globalRegistry and used by all components that collect metrics. When the application stops, this registry is removed from Metrics.globalRegistry and closed.

The PrometheusMeterRegistryWrapper component is a Root component and implements Wrapped<PrometheusMeterRegistry>, so user code can inject either the generic MeterRegistry or the concrete PrometheusMeterRegistry:

Java Kotlin

@Component
public final class SomeService {
    private final MeterRegistry meterRegistry;

    public SomeService(MeterRegistry meterRegistry) {
        this.meterRegistry = meterRegistry;
    }
}

@Component
class SomeService(
    private val meterRegistry: MeterRegistry
)

The registry automatically gets standard Micrometer binders: ClassLoaderMetrics, JvmMemoryMetrics, JvmGcMetrics, JvmThreadMetrics, ProcessorMetrics, FileDescriptorMetrics, UptimeMetrics. Kora also registers the kora.up metric with value 1 and the version tag.

Kora additionally bridges the Micrometer registry to an OpenTelemetry MeterProvider (MicrometerMeterProvider from io.opentelemetry.contrib.metrics.micrometer), so libraries instrumented with the OpenTelemetry metrics API publish through the same PrometheusMeterRegistry.

A runnable baseline that wires MetricsModule alongside HoconConfigModule, LogbackModule, UndertowHttpServerModule, and the OpenTelemetry exporter is available in the kora-java-telemetry example.

Prometheus Export

Metrics are exposed in the Prometheus text format by the private HTTP server on the privateApiHttpMetricsPath (default /metrics) served at privateApiHttpPort. The private server must have a port configured for the endpoint to be reachable. With the example configuration (privateApiHttpPort = 8085), the current metric snapshot can be scraped like this:

curl http://localhost:8085/metrics

Point your Prometheus scrape target (or any compatible collector) at the same host, port, and path.

Custom Metric

For a custom metric, it is better to create a separate component, inject MeterRegistry, and reuse created Meter instances. Do not create a new metric on every method call: if the tag set depends on the operation, use a key with limited cardinality and cache the metric in ConcurrentHashMap. The register(...) call is needed for initial metric registration in MeterRegistry; on the hot path, prefer using an already created Timer / Counter / Gauge and only call record(...) or increment(...). Kora uses the same approach for its internal metrics.

For example, a duration metric for an external operation:

Java Kotlin

@Component
public final class ExternalOperationMetrics {
    private record Key(String operation, String status) {}

    private final MeterRegistry meterRegistry;
    private final ConcurrentHashMap<Key, Timer> timers = new ConcurrentHashMap<>();

    public ExternalOperationMetrics(MeterRegistry meterRegistry) {
        this.meterRegistry = meterRegistry;
    }

    public void record(String operation, String status, long durationNanos) {
        var key = new Key(operation, status);
        var timer = this.timers.computeIfAbsent(key, k -> Timer.builder("external.operation.duration")
            .tag("operation", k.operation())
            .tag("status", k.status())
            .register(this.meterRegistry));

        timer.record(durationNanos, TimeUnit.NANOSECONDS);
    }
}

@Component
class ExternalOperationMetrics(
    private val meterRegistry: MeterRegistry
) {
    private data class Key(
        val operation: String,
        val status: String
    )

    private val timers = ConcurrentHashMap<Key, Timer>()

    fun record(operation: String, status: String, durationNanos: Long) {
        val key = Key(operation, status)
        val timer = timers.computeIfAbsent(key) {
            Timer.builder("external.operation.duration")
                .tag("operation", it.operation)
                .tag("status", it.status)
                .register(meterRegistry)
        }

        timer.record(durationNanos, TimeUnit.NANOSECONDS)
    }
}

Tag values must have a limited number of variants. Do not use user identifiers, request numbers, full error text, or other high-cardinality values as tags.

Personalization

To change PrometheusMeterRegistry configuration, add a PrometheusMeterRegistryInitializer to the container. The initializer receives the created registry before standard system metrics are registered, so it can add common tags, MeterFilter, renaming rules, or custom PrometheusMeterRegistry settings.

Important, PrometheusMeterRegistryInitializer is applied only once when the application is initialized.

For example, we want to add a common tag for all metrics:

Java Kotlin

@Module
public interface MetricsConfigModule {
    default PrometheusMeterRegistryInitializer commonTagsInit() {
        return registry -> {
            registry.config().commonTags("tag", "value");
            return registry;
        };
    }
}

@Module
interface MetricsConfigModule {
    fun commonTagsInit(): PrometheusMeterRegistryInitializer? {
        return PrometheusMeterRegistryInitializer {
            it.config().commonTags("tag", "value")
            it
        }
    }
}

Standard metrics also have their own settings, for example slo histogram buckets for DistributionSummary/Timer metrics, configured per module under telemetry.metrics. When slo is not overridden, the defaults depend on the selected OpenTelemetry standard:

• V120 — DEFAULT_SLO in milliseconds: 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000
• V123 — DEFAULT_SLO_V123 in seconds: 0.001, 0.010, 0.050, 0.100, 0.200, 0.500, 1, 2, 5, 10, 20, 30, 60, 90

Both arrays are declared in ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig; the global registry field names are in ru.tinkoff.kora.micrometer.module.MetricsConfig.

Tag Providers

The tag set attached to framework metrics is produced by per-module tag providers registered as @DefaultComponent. To change which tags are emitted for a given integration, supply your own implementation of the corresponding interface as a @DefaultComponent override:

• MicrometerHttpServerTagsProvider (package ru.tinkoff.kora.micrometer.module.http.server.tag) — HTTP server metrics
• MicrometerHttpClientTagsProvider (package ru.tinkoff.kora.micrometer.module.http.client.tag) — HTTP client metrics
• MicrometerGrpcServerTagsProvider / MicrometerGrpcClientTagsProvider (packages ...grpc.server.tag / ...grpc.client.tag) — gRPC metrics
• MicrometerKafkaConsumerTagsProvider / MicrometerKafkaProducerTagsProvider (packages ...kafka.consumer.tag / ...kafka.producer.tag) — Kafka metrics

The default provider is selected from metrics.opentelemetrySpec, so an override replaces the tag mapping for both standards.

Standard

The original metrics format used the OpenTelemetry V120 standard; after Kora 1.1.0, metrics can also be provided in the OpenTelemetry V123 standard. A partial list of changes is available in the OpenTelemetry documentation and OpenTelemetry migration guidelines.

The metrics.opentelemetrySpec parameter affects some metric names, units, and tag sets. The reference below lists both V120 and V123 variants for such metrics; if no variant is specified, the name is the same for both standards.

Metrics Reference

All Kora metrics use OpenTelemetry semantic conventions for naming and tags.

Micrometer metric types used:

• DistributionSummary — used for collecting distributions of arbitrary values. This metric type enables efficient data visualization across buckets and percentile calculation.
• Counter — monotonically increasing counter
• Gauge — current metric value
• Timer — operation duration with count, sum, max, and buckets support

HTTP Server

Metric	Prometheus	Type	Description	Tags
http.server.duration (V120), http.server.request.duration (V123)	http_server_duration_milliseconds (V120) / http_server_request_duration_seconds (V123) / _count / _sum / _bucket / _max	DistributionSummary	HTTP server request processing duration	V120: http.request.method, http.response.status_code, http.route, server.address, url.scheme, http.target, http.method, http.status_code; V123: http.request.method, http.response.status_code, http.route, url.scheme, server.address, error.type
http.server.active_requests	http_server_active_requests	Gauge	Number of active HTTP requests	V120: http.route, http.request.method, server.address, url.scheme, http.target, http.method; V123: http.route, http.request.method, server.address, url.scheme

See HTTP Server module documentation for more details.

HTTP Client

Metric	Prometheus	Type	Description	Tags
http.client.duration (V120), http.client.request.duration (V123)	http_client_duration_milliseconds (V120) / http_client_request_duration_seconds (V123) / _count / _sum / _bucket / _max	DistributionSummary	HTTP client request duration	V120: http.request.method, http.response.status_code, server.address, url.scheme, http.route, http.status_code, http.method, http.target, error.type; V123: http.request.method, http.response.status_code, server.address, url.scheme, http.route, http.status_code, error.type

See HTTP Client module documentation for more details.

Database

Metric	Prometheus	Type	Description	Tags
database.client.request.duration (V120), db.client.request.duration (V123)	database_client_request_duration_milliseconds (V120) / db_client_request_duration_seconds (V123) / _count / _sum / _bucket / _max	DistributionSummary	Database operation/query duration	V120: pool, query.id, query.operation, error; V123: db.pool.name, db.statement, db.operation, error.type

See Database module documentation for more details.

Kafka

Metric	Prometheus	Type	Description	Tags
messaging.receive.duration	messaging_receive_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	Single message processing duration	messaging.system, messaging.destination, messaging.operation, error.type
messaging.publish.duration	messaging_publish_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	Message send duration	messaging.system, messaging.destination, messaging.partition_id, error.type
messaging.process.batch.duration	messaging_process_batch_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	Message batch processing duration	messaging.system, messaging.destination, error.type
messaging.kafka.consumer.lag	messaging_kafka_consumer_lag	Gauge	Consumer lag per partition	messaging.system, messaging.destination, messaging.partition_id, messaging.consumer_group

See Kafka module documentation for more details.

gRPC Server

Metric	Prometheus	Type	Description	Tags
rpc.server.duration	rpc_server_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	gRPC server call processing duration	rpc.service, rpc.method, rpc.status, error.type
rpc.server.requests_per_rpc	rpc_server_requests_per_rpc_total	Counter	Number of requests received per RPC	rpc.service, rpc.method
rpc.server.responses_per_rpc	rpc_server_responses_per_rpc_total	Counter	Number of responses sent per RPC	rpc.service, rpc.method

See gRPC Server module documentation for more details.

gRPC Client

Metric	Prometheus	Type	Description	Tags
rpc.client.duration	rpc_client_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	gRPC client call duration	rpc.service, rpc.method, rpc.status, error.type, server.address
rpc.client.requests_per_rpc	rpc_client_requests_per_rpc_total	Counter	Number of requests sent per RPC	rpc.service, rpc.method, server.address
rpc.client.responses_per_rpc	rpc_client_responses_per_rpc_total	Counter	Number of responses received per RPC	rpc.service, rpc.method, server.address

See gRPC Client module documentation for more details.

SOAP Client

Metric	Prometheus	Type	Description	Tags
rpc.client.duration	rpc_client_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	SOAP client call duration	rpc.system, rpc.service, rpc.method, rpc.result, server.address, server.port

See SOAP Client module documentation for more details.

Scheduling

Metric	Prometheus	Type	Description	Tags
scheduling.job.duration	scheduling_job_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	Scheduled job execution duration	code.class, code.function, error.type

See Scheduling module documentation for more details.

Cache

Metric	Prometheus	Type	Description	Tags
cache.duration	cache_duration_seconds / _count / _sum / _bucket / _max	Timer	Cache operation duration (GET, SET, DELETE, and others)	cache, operation, origin, status
cache.ratio	cache_ratio_total	Counter	Cache hit/miss counter	cache, origin, type
cache.hit, cache.miss	cache_hit_total, cache_miss_total	Counter	Deprecated hit/miss counters kept for compatibility	cache, origin

Standard Micrometer metrics are automatically registered when using Caffeine:

Metric	Prometheus	Type	Description
cache.gets	cache_gets_total	Counter	Number of cache requests
cache.puts	cache_puts_total	Counter	Number of cache writes
cache.evictions	cache_evictions_total	Counter	Number of cache evictions
cache.size	cache_size	Gauge	Current cache size

See Cache module documentation for more details.

Redis / Lettuce

Metric	Prometheus	Type	Description	Tags
lettuce.command.completion.duration	lettuce_command_completion_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	Redis command completion duration	type, remote, local, command, error.type
lettuce.command.firstresponse.duration	lettuce_command_firstresponse_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	Redis command first response duration	type, remote, local, command, error.type

Resilience

Metric	Prometheus	Type	Description	Tags
resilient.circuitbreaker.state	resilient_circuitbreaker_state	Gauge	Circuit breaker state (0=CLOSED, 1=HALF_OPEN, 2=OPEN)	name
resilient.circuitbreaker.transition	resilient_circuitbreaker_transition_total	Counter	Circuit breaker state transitions	name, state
resilient.circuitbreaker.call.acquire	resilient_circuitbreaker_call_acquire_total	Counter	Circuit breaker call acquire attempts/rejections	name, state, status
resilient.retry.attempts	resilient_retry_attempts_total	Counter	Number of retry attempts	name
resilient.retry.exhausted	resilient_retry_exhausted_total	Counter	Number of exhausted retries	name
resilient.timeout.exhausted	resilient_timeout_exhausted_total	Counter	Number of timeouts	name
resilient.fallback.attempts	resilient_fallback_attempts_total	Counter	Number of fallback invocations	name, type

See Resilience module documentation for more details.

JMS

Metric	Prometheus	Type	Description	Tags
messaging.receive.duration	messaging_receive_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	JMS message receive duration	messaging.system, messaging.destination.name, error.type

S3 Client

Metric	Prometheus	Type	Description	Tags
s3.client.duration	s3_client_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	S3 HTTP request duration	aws.s3.bucket, aws.operation.name, error.type
s3.kora.client.duration	s3_kora_client_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	Kora S3 client operation duration	aws.client.name, aws.s3.bucket, aws.operation.name, error.type

See S3 Client module documentation for more details.

Camunda 7 BPMN

Metric	Prometheus	Type	Description	Tags
camunda.engine.delegate.duration	camunda_engine_delegate_duration_milliseconds / _count / _sum / _bucket / _max	DistributionSummary	Camunda BPMN Java delegate execution duration	delegate, business.key, error.type
camunda.engine.delegate.active_requests	camunda_engine_delegate_active_requests	Gauge	Number of active delegate executions	delegate, business.key

See Camunda 7 BPMN module documentation for more details.

Camunda REST

Metric	Prometheus	Type	Description	Tags
camunda.rest.server.duration (V120), camunda.rest.server.request.duration (V123)	camunda_rest_server_duration_milliseconds (V120) / camunda_rest_server_request_duration_seconds (V123) / _count / _sum / _bucket / _max	DistributionSummary	Camunda REST request duration	V120: http.request.method, http.response.status_code, http.route, server.address, url.scheme, http.target, http.method, http.status_code; V123: http.request.method, http.response.status_code, http.route, url.scheme, server.address, error.type
camunda.rest.server.active_requests	camunda_rest_server_active_requests	Gauge	Number of active Camunda REST requests	http.route, http.request.method, server.address, url.scheme

See Camunda 7 REST module documentation for more details.

Camunda 8 Worker

Metric	Prometheus	Type	Description	Tags
zeebe.worker.handler (V120), zeebe.worker.handler.duration (V123)	zeebe_worker_handler_seconds (V120) / zeebe_worker_handler_duration_seconds (V123) / _count / _sum / _bucket / _max	DistributionSummary	Zeebe Worker job handler duration	job.name, job.type, status, error, error.code
zeebe.worker.handler	zeebe_worker_handler_total	Counter	Zeebe Worker error counter	job.name, job.type, status, error.code
zeebe.client.worker.job	zeebe_client_worker_job_total	Counter	Number of activated and handled Zeebe jobs	action, type

See Camunda 8 Worker module documentation for more details.

System

Metric	Prometheus	Type	Description	Tags
kora.up	kora_up	Gauge	Framework status indicator (value = 1)	version

JVM

Standard JVM metrics are collected automatically via Micrometer:

Metric	Prometheus	Type	Description	Tags
jvm.gc.pause	jvm_gc_pause_milliseconds / _count / _sum / _max	DistributionSummary	GC pause duration	action, cause
jvm.gc.memory.allocated	jvm_gc_memory_allocated_bytes_total	Counter	Allocated memory size	—
jvm.gc.memory.promoted	jvm_gc_memory_promoted_bytes_total	Counter	Memory promoted to old gen	—
jvm.gc.max.data.size	jvm_gc_max_data_size_bytes	Gauge	Max old gen size	—
jvm.gc.live.data.size	jvm_gc_live_data_size_bytes	Gauge	Old gen size after full GC	—
jvm.memory.used	jvm_memory_used_bytes	Gauge	Used memory	area, id
jvm.memory.committed	jvm_memory_committed_bytes	Gauge	Committed JVM memory	area, id
jvm.memory.max	jvm_memory_max_bytes	Gauge	Max available memory	area, id
jvm.threads.live	jvm_threads_live_threads	Gauge	Number of live threads	—
jvm.threads.daemon	jvm_threads_daemon_threads	Gauge	Number of daemon threads	—
jvm.threads.peak	jvm_threads_peak_threads	Gauge	Peak thread count	—
jvm.threads.states	jvm_threads_states_threads	Gauge	Thread count by state	state
process.cpu.usage	process_cpu_usage	Gauge	Process CPU usage	—
system.cpu.usage	system_cpu_usage	Gauge	System CPU usage	—
system.cpu.count	system_cpu_count	Gauge	Number of available processors	—
logback.events	logback_events_total	Counter	Logging event count	level
jvm.classes.loaded	jvm_classes_loaded_classes	Gauge	Number of loaded classes	—
jvm.classes.unloaded	jvm_classes_unloaded_classes_total	Counter	Number of unloaded classes	—
process.files.open	process_files_open_files	Gauge	Number of open file descriptors	—
process.files.max	process_files_max_files	Gauge	Max file descriptors	—
process.uptime	process_uptime_milliseconds	Gauge	Process uptime	—