Skip to content

Commit f3b97ff

Browse files
pvannieroppvannierop
committed
feat(kafka): add jvmOptions and resource limits/requests
1 parent 5452633 commit f3b97ff

File tree

5 files changed

+75
-10
lines changed

5 files changed

+75
-10
lines changed

‎charts/radar-kafka/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ apiVersion: v2
44
appVersion: 3.9.0
55
description: "Apache Kafka for RADAR-base using the Strimzi Operator"
66
name: radar-kafka
7-
version: 0.3.0
7+
version: 0.5.0
88
keywords:
99
- kafka
1010
- queue

‎charts/radar-kafka/README.md‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
# radar-kafka
44
[![Artifact HUB](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/radar-kafka)](https://artifacthub.io/packages/helm/radar-base/radar-kafka)
55

6-
![Version: 0.3.0](https://img.shields.io/badge/Version-0.3.0-informational?style=flat-square) ![AppVersion: 3.9.0](https://img.shields.io/badge/AppVersion-3.9.0-informational?style=flat-square)
6+
![Version: 0.5.0](https://img.shields.io/badge/Version-0.5.0-informational?style=flat-square) ![AppVersion: 3.9.0](https://img.shields.io/badge/AppVersion-3.9.0-informational?style=flat-square)
77

88
Apache Kafka for RADAR-base using the Strimzi Operator
99

@@ -60,19 +60,24 @@ Consult the [documentation](https://github.com/lsst-sqre/strimzi-registry-operat
6060
| strimzi-registry-operator | object | `{"clusterName":"radar","operatorNamespace":"default"}` | Values for schema registry operator ref: https://github.com/lsst-sqre/strimzi-registry-operator |
6161
| strimzi-registry-operator.clusterName | string | `"radar"` | Keep in sync with 'fullnameOverride' |
6262
| strimzi-registry-operator.operatorNamespace | string | `"default"` | Keep in sync with namespace used by deployment |
63-
| kafka | object | `{"cruiseControl":{"addRebalanceTemplate":true,"enabled":false,"goals":[],"skipHardGoalCheck":true},"insyncReplicas":2,"metadataVersion":"3.9-IV0","partitions":9,"podSecurityContext":{},"replicas":3,"replicationFactor":3,"resources":{"requests":{"cpu":"100m","memory":"1Gi"}},"securityContext":{},"storage":{"size":"10Gi"}}` | Values for Kafka cluster deployed by Strimzi kafka operator |
63+
| kafka | object | `{"cruiseControl":{"addRebalanceTemplate":true,"enabled":false,"goals":[],"javaOptions":{"Xms":"128m","Xmx":"256m"},"skipHardGoalCheck":true},"insyncReplicas":2,"javaOptions":{"Xms":"512m","Xmx":"1G"},"metadataVersion":"3.9-IV0","nodePool":{"jvmOptions":{"Xms":"512m","Xmx":"1G"},"resources":{"limits":{"cpu":1,"memory":"2Gi"},"requests":{"cpu":"100m","memory":"1Gi"}}},"partitions":9,"podSecurityContext":{},"replicas":3,"replicationFactor":3,"resources":{"limits":{"cpu":1,"memory":"2Gi"},"requests":{"cpu":"100m","memory":"1Gi"}},"securityContext":{},"storage":{"size":"10Gi"},"topicOperator":{"jvmOptions":{"Xms":"64m","Xmx":"128m"}},"userOperator":{"jvmOptions":{"Xms":"64m","Xmx":"128m"}}}` | Values for Kafka cluster deployed by Strimzi kafka operator |
6464
| kafka.replicas | int | `3` | Number of Kafka brokers |
6565
| kafka.replicationFactor | int | `3` | Number of replicas for Kafka topics |
6666
| kafka.insyncReplicas | int | `2` | Number of in-sync kafka broker replicas |
6767
| kafka.partitions | int | `9` | Number of topic data partitions. Rule of thumb: 3 times the number of brokers. Headroom is used for future upscale of brokers. ref: https://learn.conduktor.io/kafka/kafka-topics-choosing-the-replication-factor-and-partitions-count/ |
6868
| kafka.metadataVersion | string | `"3.9-IV0"` | Metadata API version. Keep in sync with appVersion |
6969
| kafka.storage | object | `{"size":"10Gi"}` | Storage size for Kafka pods |
70-
| kafka.resources | object | `{"requests":{"cpu":"100m","memory":"1Gi"}}` | Resource requests for Kafka pods |
71-
| kafka.cruiseControl | object | `{"addRebalanceTemplate":true,"enabled":false,"goals":[],"skipHardGoalCheck":true}` | Config for the Cruise Control rebalancer (https://github.com/linkedin/cruise-control) |
70+
| kafka.resources.requests | object | `{"cpu":"100m","memory":"1Gi"}` | Resource requests for Kafka pods |
71+
| kafka.resources.limits | object | `{"cpu":1,"memory":"2Gi"}` | Resource limits for Kafka pods |
72+
| kafka.javaOptions | object | `{"Xms":"512m","Xmx":"1G"}` | JVM options for Kafka pods |
73+
| kafka.nodePool.resources.requests | object | `{"cpu":"100m","memory":"1Gi"}` | Resource requests for Kafka pods |
74+
| kafka.nodePool.resources.limits | object | `{"cpu":1,"memory":"2Gi"}` | Resource limits for Kafka pods |
75+
| kafka.cruiseControl | object | `{"addRebalanceTemplate":true,"enabled":false,"goals":[],"javaOptions":{"Xms":"128m","Xmx":"256m"},"skipHardGoalCheck":true}` | Config for the Cruise Control rebalancer (https://github.com/linkedin/cruise-control) |
7276
| kafka.cruiseControl.enabled | bool | `false` | Deploy the Cruise Control rebalancer operator. |
7377
| kafka.cruiseControl.addRebalanceTemplate | bool | `true` | Add a KafkaRebalance template CRD. Note that this does not automatically rebalance the cluster. For rebalancing, you need to create a KafkaRebalance resource using the template. |
7478
| kafka.cruiseControl.goals | list | `[]` | Goals for the Cruise Control rebalancer. If left empty, the default goals are used (see: https://github.com/linkedin/cruise-control#goals). |
7579
| kafka.cruiseControl.skipHardGoalCheck | bool | `true` | Skip hard goal check by Cruise Control rebalancer. |
80+
| kafka.cruiseControl.javaOptions | object | `{"Xms":"128m","Xmx":"256m"}` | JVM options for Cruise Control pods |
7681
| kafka.podSecurityContext | object | `{}` | Security Context for Kafka pods ref: https://strimzi.io/docs/operators/latest/deploying#assembly-security-providers-str |
7782
| kafka.securityContext | object | `{}` | Security Context for Kafka containers ref: https://strimzi.io/docs/operators/latest/deploying#assembly-security-providers-str |
7883
| schema-registry | object | check `values.yaml` | Values for schema registry deployed by strimzi-registry-operator ref: https://github.com/lsst-sqre/strimzi-registry-operator |

‎charts/radar-kafka/templates/kafka.yaml‎

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,13 @@ spec:
2525
default.replication.factor: {{ $topicReplicationFactor }}
2626
min.insync.replicas: {{ $insyncReplicas }}
2727
num.partitions: {{ $topicPartitions }}
28+
{{- if .Values.kafka.javaOptions }}
29+
jvmOptions: {{ .Values.kafka.javaOptions | toYaml | nindent 6 }}
30+
{{- end }}
31+
{{- if .Values.kafka.resources }}
32+
resources:
33+
{{ .Values.kafka.resources | toYaml | indent 4 | trim }}
34+
{{- end }}
2835
listeners:
2936
- name: plain
3037
port: 9092
@@ -71,10 +78,23 @@ spec:
7178
- mode: remove-brokers
7279
template:
7380
name: {{ template "common.names.fullname" . }}-rebalance-template
81+
{{- if .values.kafka.cruiseControl.javaOptions }}
82+
jvmOptions: {{ .values.kafka.cruiseControl.javaOptions | toYaml | nindent 6 }}
83+
{{- end }}
7484
{{- end }}
7585
kafkaExporter:
7686
{{- toYaml .Values.metrics.kafkaExporter | nindent 4 }}
7787
{{- end }}
7888
entityOperator:
79-
topicOperator: {}
80-
userOperator: {}
89+
topicOperator:
90+
{{- if .Values.kafka.topicOperator.jvmOptions }}
91+
jvmOptions: {{ .Values.kafka.topicOperator.jvmOptions | toYaml | nindent 8 }}
92+
{{- else }}
93+
{}
94+
{{- end }}
95+
userOperator:
96+
{{- if .Values.kafka.userOperator.jvmOptions }}
97+
jvmOptions: {{ .Values.kafka.userOperator.jvmOptions | toYaml | nindent 8 }}
98+
{{- else }}
99+
{}
100+
{{- end }}

‎charts/radar-kafka/templates/nodepool.yaml‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,12 @@ spec:
1212
roles:
1313
- controller
1414
- broker
15-
{{- if .Values.kafka.resources }}
15+
{{- if .Values.kafka.nodePool.javaOptions }}
16+
jvmOptions: {{ .Values.kafka.nodePool.javaOptions | toYaml | nindent 6 }}
17+
{{- end }}
18+
{{- if .Values.kafka.nodePool.resources }}
1619
resources:
17-
{{ .Values.kafka.resources | toYaml | indent 4 | trim }}
20+
{{ .Values.kafka.nodePool.resources | toYaml | indent 4 | trim }}
1821
{{- end }}
1922
storage:
2023
type: jbod

‎charts/radar-kafka/values.yaml‎

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,11 +67,44 @@ kafka:
6767
storage:
6868
size: 10Gi
6969

70-
# -- Resource requests for Kafka pods
7170
resources:
71+
# -- Resource requests for Kafka pods
7272
requests:
7373
cpu: 100m
7474
memory: 1Gi
75+
# -- Resource limits for Kafka pods
76+
limits:
77+
cpu: 1
78+
memory: 2Gi
79+
80+
# -- JVM options for Kafka pods
81+
javaOptions:
82+
Xms: 512m
83+
Xmx: 1G
84+
85+
nodePool:
86+
jvmOptions:
87+
Xms: 512m
88+
Xmx: 1G
89+
resources:
90+
# -- Resource requests for Kafka pods
91+
requests:
92+
cpu: 100m
93+
memory: 1Gi
94+
# -- Resource limits for Kafka pods
95+
limits:
96+
cpu: 1
97+
memory: 2Gi
98+
99+
userOperator:
100+
jvmOptions:
101+
Xms: 64m
102+
Xmx: 128m
103+
104+
topicOperator:
105+
jvmOptions:
106+
Xms: 64m
107+
Xmx: 128m
75108

76109
# -- Config for the Cruise Control rebalancer (https://github.com/linkedin/cruise-control)
77110
cruiseControl:
@@ -85,6 +118,10 @@ kafka:
85118
goals: []
86119
# -- Skip hard goal check by Cruise Control rebalancer.
87120
skipHardGoalCheck: true
121+
# -- JVM options for Cruise Control pods
122+
javaOptions:
123+
Xms: 128m
124+
Xmx: 256m
88125

89126
# -- Security Context for Kafka pods
90127
# ref: https://strimzi.io/docs/operators/latest/deploying#assembly-security-providers-str

0 commit comments

Comments
 (0)