Skip to content

Commit

Permalink
Address comments from Paolo
Browse files Browse the repository at this point in the history 
Signed-off-by: Federico Valeri <[email protected]>
  • Loading branch information
@fvaleri
fvaleri committed Jan 8, 2025
1 parent 29076de commit ed1bb17
Showing 1 changed file with 27 additions and 25 deletions.
52 changes: 27 additions & 25 deletions 090-integrate-bridge-with-metrics-reporter.md
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
# Integrate Bridge with Metrics Reporter

In [SIP-064](https://github.com/strimzi/proposals/blob/main/064-prometheus-metrics-reporter.md) we introduced the [Strimzi Metrics Reporter](https://github.com/strimzi/metrics-reporter).
This is a Kafka MetricsReporter plugin that directly exposes metrics in Prometheus format via an HTTP endpoint.
This is a Kafka plugin that directly exposes metrics in Prometheus format via an HTTP endpoint, and can be used as an alternative to the [Prometheus JMX Exporter](https://github.com/prometheus/jmx_exporter).
In the following paragraphs, we will refer to the former as *Reporter*, and the latter as *Exporter*.

## Current situation

The HTTP Bridge allows users to enable or disable the metrics endpoint using the `KAFKA_BRIDGE_METRICS_ENABLED` environment variable.
When this variable is set to true, the Bridge creates the JMX Exporter's JmxCollector class with hard coded configuration.
This configuration is a JMX metrics exporter YAML distributed as an embedded resource within the Bridge's JAR file.
The Bridge allows users to enable or disable the metrics endpoint using the `KAFKA_BRIDGE_METRICS_ENABLED` environment variable.
When this variable is set to true, the Bridge creates the Exporter's JmxCollector class with hard coded configuration for Kafka related metrics.
This configuration is a JMX metrics *Exporter* YAML distributed as an embedded resource within the Bridge's JAR file.
In addition to Kafka metrics, the Bridge also exposes JVM and HTTP metrics through Vert.x, that are also stored om JmxCollector's Prometheus registry.

When deploying the Bridge through the Cluster Operator, a similar configuration is available in the KafkaBridge's CRD:

Expand All @@ -16,30 +18,30 @@ spec:
enableMetrics: true
```

This is different from how the metrics endpoint is enabled in all the other major components deployed by the Cluster Operator.
The Cluster Operator exposes metrics through the [Prometheus JMX Exporter](https://github.com/prometheus/jmx_exporter), which can be configured using the shared `metricsConfig` schema.
This schema has a type property that only allows the `jmxPrometheusExporter` value, and a reference to a ConfigMap containing its configuration:
This is different from how the metrics endpoint is enabled in all the other components deployed by the Cluster Operator.
The Cluster Operator exposes metrics through the Exporter, which can be configured using the shared `metricsConfig` schema.
This schema has a type property that only allows the `jmxPrometheusExporter` value, and a reference to a ConfigMap containing its configuration.

```sh
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: $COMPONENT_NAME-metrics
name: my-metrics
key: metrics-config.yml
```

At runtime, the above configuration enables a Java agent which exposes Kafka JMX metrics in Prometheus format through an HTTP endpoint on port 9404.
Note that this agent depends on the [Kafka JMX Reporter](https://github.com/apache/kafka/blob/3.9.0/clients/src/main/java/org/apache/kafka/common/metrics/JmxReporter.java) plugin, which is enabled by default.
At runtime, the above configuration enables a Java agent that exposes Kafka JMX metrics in Prometheus format through an HTTP endpoint on port 9404.
Note that the *Exporter* requires the [Kafka JMX Reporter](https://github.com/apache/kafka/blob/3.9.0/clients/src/main/java/org/apache/kafka/common/metrics/JmxReporter.java).

## Motivation

We want to support the Strimzi Metrics Reporter as an alternative way of configuring metrics across all components.
When deploying configuring major components through the Cluster Operator, we want to provide a consistent user experience, so the Bridge should also support the `metricsConfig` schema.
We want to support the *Reporter* as an alternative way of configuring metrics across all Cluster Operator components.
When deploying any component through the Cluster Operator, we want to provide a consistent user experience, so the Bridge should also support the `metricsConfig` schema.

## Proposal

The Bridge will support both JMX Prometheus Exporter and Strimzi Metrics Reporter as metrics configuration types.
The Bridge will support both *Exporter* and *Reporter* as metrics configuration types.
It will also provide new configurations for the integration with Cluster Operator.

### Metrics configuration types
Expand All @@ -49,7 +51,7 @@ A new `bridge.metrics` property will be available within the application.propert
Any other value will raise an error and the application will fail to start with an appropriate error message.

When running in standalone mode with `strimziMetricsReporter`, the user will be able to configure any reporter property using the "kafka." prefix.
The following example will be provided as a comment in the default application.properties file:
The following example will be documented and also provided as a comment in the default application.properties file:

```sh
bridge.metrics=strimziMetricsReporter
Expand All @@ -58,19 +60,19 @@ kafka.prometheus.metrics.reporter.listener.enable=false
kafka.prometheus.metrics.reporter.allowlist=.*
```

The `KAFKA_BRIDGE_METRICS_ENABLED` environment variable will be deprecated and removed in a future release.
The `KAFKA_BRIDGE_METRICS_ENABLED` environment variable will be deprecated in 0.32.0 and removed in 0.34.0.
When set, the user will get a warning suggesting to use the `bridge.metrics` property.
In case they are both set, `bridge.metrics` will take precedence over `KAFKA_BRIDGE_METRICS_ENABLED`.

The MetricsReporter class will be updated to also include a new StrimziCollectorRegistry that will work similarly to the JmxCollectorRegistry.
The StrimziCollectorRegistry will include a reference to PrometheusRegistry.defaultRegistry, which is the same instance used by the Strimzi Metrics Reporter to collect metrics.
The StrimziCollectorRegistry will include a reference to PrometheusRegistry.defaultRegistry, which is the same instance used by the *Reporter* to collect metrics.

The kafka_bridge_config_generator.sh script is used to generate the image configuration based on environment variables.
This script will be also updated to include `bridge.metrics` and related configurations (see the following section).

The Bridge will try to load the JMX Prometheus Exporter configuration file from the path specified by the `bridge.metrics.jmx.exporter.config.path` property.
The Bridge will try to load the Exporter's configuration file from the path specified by the `bridge.metrics.jmx.exporter.config.path` property.
If the property is not specified or the file is not found, the Bridge will fall back to the hard coded configuration.
This feature is not strictly required to support the Strimzi Metrics Reporter, but will be used by the Cluster Operator.
This feature is not strictly required to support the *Reporter*, but will be used by the Cluster Operator.

### Cluster Operator integration

Expand All @@ -90,17 +92,17 @@ spec:
Three new environment variables will be introduced to pass the metrics configuration to the Bridge's container:

- `STRIMZI_METRICS`: This will contain the `metricsConfig` types to enable (one of `jmxPrometheusExporter` and `strimziMetricsReporter`).
- `KAFKA_BRIDGE_METRICS_JMX_CONFIG`: Used with JMX Prometheus Exporter to pass the configuration file path.
- `KAFKA_BRIDGE_METRICS_SMR_CONFIG`: Used with Strimzi Metrics Reporter to pass the plugin configuration.
- `KAFKA_BRIDGE_METRICS_JMX_CONFIG`: Used with *Exporter* to pass the configuration file path.
- `KAFKA_BRIDGE_METRICS_SMR_CONFIG`: Used with *Reporter* to pass the plugin configuration.

The `enableMetrics` property will be deprecated and removed in a future release.
The `enableMetrics` property will be deprecated in 0.46.0 and removed in 0.48.0.
When set, the user will get a warning suggesting to use the `metricsConfig` configuration.
In case they are both set, `metricsConfig` will take precedence over `enableMetrics`.

The JMX Prometheus Exporter configuration file will be stored in a ConfigMap and mounted in the Bridge's container.
The *Exporter* configuration file will be stored in a ConfigMap and mounted in the Bridge's container.
The full configuration file passed to the Bridge's container will be `/opt/strimzi/custom-config/metrics-config.yml`.

Metrics will be exposed through the Bridge's HTTP server, so the Strimzi Metrics Reporter's listener will be disabled.
Metrics will be exposed through the Bridge's HTTP server, so the *Reporter*'s listener will be disabled.
Other configurations will be locked down with the exception of the `prometheus.metrics.reporter.allowlist` property.

## Affected/not affected projects
Expand All @@ -113,5 +115,5 @@ All changes will be backwards compatible, but there will be some deprecations as

## Rejected alternatives

Lock down and automate the Strimzi Metrics Reporter configurations for the standalone Bridge.
This wa rejected because the Bridge allows user to customize Kafka clients and plugins using the application.properties file, which includes commented examples.
Lock down and automate the *Reporter* configurations for the standalone Bridge.
This was rejected because the Bridge allows user to customize Kafka clients and plugins using the application.properties file, which includes commented examples.

0 comments on commit ed1bb17

Please sign in to comment.