Add proposal for integrating Bridge with Metrics Reporter

[fvaleri]

Add proposal for integrating Bridge with Metrics Reporter.

[fvaleri]

@ppatierno thanks for the review. I reworked the proposal to always present the Bridge changes first, and addressed the other comments.

[PaulRMellor]

The overall approach makes sense.
I left a few suggestions, including a couple of questions where I wasn't clear on parts of the proposal.

[fvaleri]

@ppatierno and @PaulRMellor your comments should be addressed, but let me know if you have any other concern. It would also be good to get some more feedback from other maintainers. Thanks.

[PaulRMellor]

Updates look good 👍

[ppatierno]

LGTM.

[scholzj]

Why does the user need to set the bridge.metrics when they need to also configure the rest of the details? I get the allow list. But why are the reporter and the listener not done automatically?

[ppatierno]

I guess this part of the properties file is an example when the bridge is used on bare metal/vm. In such a case, it's the user to provide the entire configuration (their own application.properties file). The bridge itself doesn't never change the config file on startup to add "something" to it.
One thing we could imagine is that the bridge doesn't update the file but add these value at runtime (in memory) when configuring the internal Kafka clients. My only doubt is that it won't be reflected in what the user configured so the user doesn't know about that.

[fvaleri]

We can use the same logic that we have in the Operator, with locked down Reporter configuration and a new bridge.metrics.reporter.allowlist property.

[scholzj]

Maybe I missed some of the arguments you are trying to make and maybe my comment was not clear, sorry. It of course does not matter when the operator configures it. But when I configure it myself, why do I need to set both

bridge.metrics=strimziMetricsReporter #1
kafka.metric.reporters=io.strimzi.kafka.metrics.KafkaPrometheusMetricsReporter #2

They seem to do the same - enable the Strimzi metrics reporter, or? So what is the difference between them?

[fvaleri]

The bridge.metrics property is the analogous of the metricsConfig.type field we have in the CO schemas. Maybe the property name is confusing, but it was inspired by the already existing bridge.tracing. It will be used to initialize the Bridge's MetricsRporter with the right CollectorRegistry.

The kafka.metric.reporters property is the Kafka metric.reporters configuration with the kafka. prefix that must be used to pass shared Kafka configurations (they are applied to all internal clients).

[scholzj]

But why do we need the user to configure both? You say it is analogous of the metricsConfig.type field -> but there you configure only the metricsConfig.type field.

[fvaleri]

I see what you mean. I think we can improve the design using default values for kafka.metric.reporters and kafka.prometheus.metrics.reporter.listener.enable to be used as fallback when not set. This is similar to what I describe for bridge.metrics.exporter.config.path. The example application.properties would be:

# uncomment the following line to enable JMX Prometehus Exporter, check the documentation for more details
#bridge.metrics=jmxPrometheusExporter
# optionally, you can also set a custom configuration file
#bridge.metrics.exporter.config.path=/path/to/my-exporter-config.yaml

# uncomment the following line to enable Strimzi Metrics Reporter, check the documentation for more details
#bridge.metrics=strimziMetricsReporter
# optionally, you can filter the exposed metrics using a list of regexes
#kafka.prometheus.metrics.reporter.allowlist=.*

[scholzj]

This should probably have some defaults to match what we / our sample dashabords need.

[fvaleri]

Yes, the default is .*. Example updated.

[scholzj]

That is not really what I meant. And it relates to the PR from @OwenCorrigan76 as well. I think we should choose one of the ways here:

1. Make the allowList required
2. Have a good default for them for each component. (And I guess .* is not the right default) This would be hardcoded in the code base and change as we for example update our dashabords.

Having a flat default .* everywhere is IMHO a bad option, because historically our experience is that users want to limit the metrics exporter:

• For performance reasons in Kafka (this might improve with our own reporter, but I expect it would still be an issue at some amount of metrics)
• For performance reasons in Prometheus (the metrics can be dropped by Prometheus after scraping, but that is inefficient and harder to configure)

[fvaleri]

I think having good defaults is better. I will look into that.

[fvaleri]

Bridge defaults added. We will address the other components in the related PRs.

[fvaleri]

@scholzj I think I addressed all your comments, let me know what you think. Thanks.

[scholzj]

Two nits. But LGTM otherwise. Thanks.

[scholzj]

Suggested change

	When `enableMetrics` will be removed, *Exporter* users will have to create a secret with its configuration, and reference the secret as shown in the following example:
	When `enableMetrics` will be removed, *Exporter* users will have to create a config map with its configuration, and reference it as shown in the following example:

[scholzj]

Suggested change

	A full example that include the *Exporter* configuration secret will be provided.
	A full example that include the *Exporter* configuration config map will be provided.