Add support for the Strimzi Metrics Reporter to Kafka brokers/controllers components #11051

OwenCorrigan76 · 2025-01-16T17:14:53Z

Type of change

Enhancement / new feature

Description

This patch adds support for the Strimzi Metrics Reporter to Kafka brokers/controllers components as described by the following proposal:

https://github.com/strimzi/proposals/blob/main/064-prometheus-metrics-reporter.md

Related to #10753

Support for Kafka Connect and MirrorMaker2 will be added in subsequent PRs.

Checklist

Write tests
Make sure all tests pass
Update documentation
Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
Reference relevant issue(s) and close them after merging
Update CHANGELOG.md

scholzj

I left some comments after an initial pass. You should also update the names / description / CHANGELOG to make it clear what this really does. Kafka compoenents are brokers/controllers, Connect and MM2. You add this only for brokers/controllers. It should be clear from the CHANGELOG and PR name / desc.

scholzj · 2025-01-16T17:19:04Z

CHANGELOG.md

@@ -6,6 +6,7 @@
 * Support for MirrorMaker 1 has been removed
 * Added support to configure `dnsPolicy` and `dnsConfig` using the `template` sections.
 * Store Kafka node certificates in separate Secrets, one Secret per pod.
+* Added support for Strimzi Metrics Reporter to the Kafka component.


What Kafka component?

Thanks for the feedback @scholzj. I will address this.

scholzj · 2025-01-16T17:21:48Z

packaging/examples/metrics/grafana-dashboards/strimzi-kafka.json

I don't think this makes really sense. It is over complicated, makes it harder to maintain the dashboards and probably makes it harder to use them as well. Just create separate dashboards in a subdirectory for the metrics reporter.

Ok, I will create separate dashboards as requested.

scholzj · 2025-01-16T17:22:11Z

packaging/examples/metrics/grafana-dashboards/strimzi-kraft.json

Same as above. Also, I guess Connect and MM2 dashboards are missing? Why?

Connect and MM2 dashboards will be created in subsequent PRs, as we have only allowed for Kafka brokers and controllers in the PR.

scholzj · 2025-01-16T17:28:32Z

api/src/main/java/io/strimzi/api/kafka/model/kafka/KafkaClusterSpec.java

@@ -62,7 +62,8 @@ public class KafkaClusterSpec implements HasConfigurableMetrics, HasConfigurable
            + "cruise.control.metrics.topic, cruise.control.metrics.reporter.bootstrap.servers, "
            + "node.id, process.roles, controller., metadata.log.dir, zookeeper.metadata.migration.enable, " // KRaft options
            + "client.quota.callback.static.kafka.admin., client.quota.callback.static.produce, client.quota.callback.static.fetch, "
-            + "client.quota.callback.static.storage.per.volume.limit.min.available., client.quota.callback.static.excluded.principal.name.list";
+            + "client.quota.callback.static.storage.per.volume.limit.min.available., client.quota.callback.static.excluded.principal.name.list, "
+            + "kafka.metric.reporters, prometheus.metrics.reporter.";


Why are we disabling kafka.metric.reporters? What if someone uses it? If we want to disable it, it is definitely worth adding to CHANGELOG. But maybe it should be kept configurable?

The naming here is a little unfortunate. We are not disabling metric.reporters but we are locking down kafka.metrics.reporters as the proposal requests. metric.reporters is a Kafka config that is configurable but kafka.metrics.reporters is a Reporter config that is not configurable.

There is no option metrics.reporters. You are disabling the option kafka.metric.reporters that is a standard broker configuration option and might be used by the users. If nothing else, this deserves more discussion as it has backward compatibility implications. Was it part of the proposal? I do not remember that.

From the Kafka docs, kafka.metric.reporters seems to be a list. So are there any reasons why we can't have custom and our reporters?

scholzj · 2025-01-16T17:29:09Z

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/ConfigMapUtils.java

@@ -67,7 +67,7 @@ public static Map<String, String> generateMetricsAndLogConfigMapData(Reconciliat
            data.put(supportsLogging.logging().configMapKey(), supportsLogging.logging().loggingConfiguration(reconciliation, metricsAndLogging.loggingCm()));
        }

-        if (model instanceof SupportsMetrics supportMetrics) {
+        if (model instanceof SupportsMetrics supportMetrics && supportMetrics.metrics() != null) {


Why do we need this?

scholzj · 2025-01-16T17:30:32Z

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/CruiseControl.java

@@ -394,7 +405,8 @@ public Deployment generateDeployment(Map<String, String> annotations, boolean is
    protected List<EnvVar> getEnvVars() {
        List<EnvVar> varList = new ArrayList<>();

-        varList.add(ContainerUtils.createEnvVar(ENV_VAR_CRUISE_CONTROL_METRICS_ENABLED, String.valueOf(metrics.isEnabled())));
+        String jmxMetricsEnabled = jmxExporterMetrics != null && jmxExporterMetrics.isEnabled() ? Boolean.TRUE.toString() : Boolean.FALSE.toString();


Maybe you can inline this?

scholzj · 2025-01-16T17:31:50Z

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaCluster.java

+    private MetricsModel jmxExporterMetrics;
+    private StrimziMetricsReporterModel strimziMetricsReporter;


We have MetricsModel for a reason. We should try to unify both metrics types into a single class (e.g. have MetricsModel as abstract class or interface and then two implementations for the different metric types).

scholzj · 2025-01-16T17:32:15Z

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java

@@ -286,7 +306,6 @@ public KafkaBrokerConfigurationBuilder withListeners(
        ////////////////////
        // Shared configurations with values dependent on all listeners
        ////////////////////
-


This is intentional here. Separates unrelated comments.

Clumsy error. Will fix

scholzj · 2025-01-16T17:33:32Z

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaCluster.java

+    /**
+     * The configuration field name for Kafka metric reporters.
+     */
+    public static final String KAFKA_METRIC_REPORTERS_CONFIG_FIELD = "metric.reporters";


It is not even used here. So why is it defined here?

I defined it here because this is where the other Kafka config variables were defined. I will move it into the class where it is used and make it private.

This patch adds support for the Strimzi Metrics Reporter to brokers and controllers as described by the following proposal: https://github.com/strimzi/proposals/blob/main/064-prometheus-metrics-reporter.md Signed-off-by: ocorriga <[email protected]>

Signed-off-by: ocorriga <[email protected]>

OwenCorrigan76 · 2025-01-17T12:01:44Z

@scholzj I am currently working on the changes I did not comment on yet.

Signed-off-by: ocorriga <[email protected]>

Signed-off-by: Federico Valeri <[email protected]>

scholzj · 2025-01-19T20:38:53Z

CHANGELOG.md

@@ -6,6 +6,8 @@
 * Support for MirrorMaker 1 has been removed
 * Added support to configure `dnsPolicy` and `dnsConfig` using the `template` sections.
 * Store Kafka node certificates in separate Secrets, one Secret per pod.
+* Added support for Strimzi Metrics Reporter to the Kafka brokers/controllers components. 
+  Support for Kafka Connect, MirrorMaker2 and Http Bridge will be added subsequently.


I would leave this sentence out. It is confusing.

If you do the others in the scope of 0.46, you will anyway remove this

If you do them later - e.g. in 0.48 - it will be forever unclear here if it was ever done or not without studying the later release notes.

scholzj · 2025-01-19T20:43:09Z

api/src/main/java/io/strimzi/api/kafka/model/common/metrics/StrimziMetricsReporterValues.java

+@EqualsAndHashCode()
+@ToString
+public class StrimziMetricsReporterValues implements UnknownPropertyPreserving {
+    private static final String DEFAULT_REGEX = ".*";


I think we should have the default tailor-made for each component. (Or it should be required) And the allow list should default to null to allow you to distinguish between not set and set to .*.

scholzj · 2025-01-19T20:51:41Z

api/src/main/java/io/strimzi/api/kafka/model/common/metrics/JmxPrometheusExporterMetrics.java

@@ -31,7 +30,6 @@ public class JmxPrometheusExporterMetrics extends MetricsConfig {
    private ExternalConfigurationReference valueFrom;

    @Description("ConfigMap entry where the Prometheus JMX Exporter configuration is stored. ")
-    @JsonProperty(required = true)


IIRC, you had to make it optional because of how the API is constructed. But it is required. So, we have to make sure:

It is handled in the code

Use the CEL validation (Use CEL validation in our CRDs #9417) to make this required that way (but this would need to be done right away). If you want, I can look at that when I get back to the office, but that would definitely delay this PR.

OwenCorrigan76 requested review from a team and mimaison January 16, 2025 17:15

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch 2 times, most recently from e569be2 to 1180ee2 Compare January 16, 2025 17:23

scholzj reviewed Jan 16, 2025

View reviewed changes

ocorriga added 2 commits January 17, 2025 11:56

Address initial feedback

f943007

Signed-off-by: ocorriga <[email protected]>

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch from 1180ee2 to f943007 Compare January 17, 2025 12:00

ocorriga and others added 3 commits January 17, 2025 15:40

Reverse dashboards and fix build issue

e145907

Signed-off-by: ocorriga <[email protected]>

Update derived resources

954a345

Signed-off-by: ocorriga <[email protected]>

Trigger build

35bc15d

Signed-off-by: Federico Valeri <[email protected]>

fvaleri added this to the 0.46.0 milestone Jan 17, 2025

OwenCorrigan76 changed the title ~~Add support for the Strimzi Metrics Reporter to Kafka component~~ Add support for the Strimzi Metrics Reporter to Kafka brokers/controllers components Jan 17, 2025

scholzj reviewed Jan 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for the Strimzi Metrics Reporter to Kafka brokers/controllers components #11051

Add support for the Strimzi Metrics Reporter to Kafka brokers/controllers components #11051

OwenCorrigan76 commented Jan 16, 2025 •

edited

Loading

scholzj left a comment

scholzj Jan 16, 2025

OwenCorrigan76 Jan 17, 2025

scholzj Jan 16, 2025

OwenCorrigan76 Jan 17, 2025

scholzj Jan 16, 2025

OwenCorrigan76 Jan 17, 2025

scholzj Jan 16, 2025

OwenCorrigan76 Jan 17, 2025 •

edited

Loading

scholzj Jan 19, 2025

scholzj Jan 16, 2025

scholzj Jan 16, 2025

OwenCorrigan76 Jan 17, 2025

scholzj Jan 16, 2025

scholzj Jan 16, 2025

OwenCorrigan76 Jan 17, 2025

scholzj Jan 16, 2025

OwenCorrigan76 Jan 17, 2025

OwenCorrigan76 commented Jan 17, 2025

scholzj Jan 19, 2025

scholzj Jan 19, 2025

scholzj Jan 19, 2025

		private MetricsModel jmxExporterMetrics;
		private StrimziMetricsReporterModel strimziMetricsReporter;

Add support for the Strimzi Metrics Reporter to Kafka brokers/controllers components #11051

Are you sure you want to change the base?

Add support for the Strimzi Metrics Reporter to Kafka brokers/controllers components #11051

Conversation

OwenCorrigan76 commented Jan 16, 2025 • edited Loading

Type of change

Description

Checklist

scholzj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OwenCorrigan76 Jan 17, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OwenCorrigan76 commented Jan 17, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OwenCorrigan76 commented Jan 16, 2025 •

edited

Loading

OwenCorrigan76 Jan 17, 2025 •

edited

Loading