Skip to content

Configure ClustersProperties.AdminClient per ClustersProperties.Cluster #1713

@alkur-gh

Description

@alkur-gh

Issue submitter TODO list

  • I've searched for an already existing issues here
  • I'm running a supported version of the application which is listed here and the feature is not present there

Is your proposal related to a problem?

We have one Kafbat instance connected to old and less old Kafka clusters (2.8 & 3.6). In the old clusters, ReactiveAdminClient.listConsumerGroupOffsets is very slow because AdminClient performs sequential OffsetFetch requests in a single thread. Decreasing ClustersProperties.AdminClient.listConsumerGroupOffsetsPartitionSize to 1-2 helps with the old clusters, but makes ReactiveAdminClient.listConsumerGroupOffsets a bit slower for the new clusters. It would be nice to have the option to assign different partition sizes to different clusters.

Describe the feature you're interested in

kafka:
  clusters:
    - name: cluster-1
      bootstrap-servers: 127.0.0.1:9092
      admin-client:
        list-consumer-group-offsets-partition-size: 1
        list-consumer-group-offsets-concurrency: 16

    - name: cluster-2
      bootstrap-servers: 127.0.0.2:9092
      admin-client:
        list-consumer-group-offsets-partition-size: 200
        list-consumer-group-offsets-concurrency: 4

Describe alternatives you've considered

No response

Version you're running

dfa5a7e

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions