-
Notifications
You must be signed in to change notification settings - Fork 237
Open
Description
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
- create cluster
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
name: mongodb-zedvuh
namespace: default
spec:
clusterDef: mongodb
topology: replicaset
terminationPolicy: Delete
componentSpecs:
- name: mongodb
serviceVersion: 4.4.29
replicas: 3
resources:
limits:
cpu: 100m
memory: 0.5Gi
requests:
cpu: 100m
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
kubectl get cluster
NAME CLUSTER-DEFINITION TERMINATION-POLICY STATUS AGE
etcd-eewrxk Delete Running 51m
kbcli cluster list-instances etcd-eewrxk
NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME
etcd-eewrxk-etcd-0 default etcd-eewrxk etcd Running leader cn-guangzhou-b 100m / 100m 512Mi / 512Mi data:21Gi 192.168.0.37/192.168.0.37 Sep 03,2025 16:14 UTC+0800
etcd-eewrxk-etcd-1 default etcd-eewrxk etcd Running follower cn-guangzhou-b 100m / 100m 512Mi / 512Mi data:21Gi 192.168.0.37/192.168.0.37 Sep 03,2025 16:14 UTC+0800
etcd-eewrxk-etcd-2 default etcd-eewrxk etcd Running follower cn-guangzhou-b 100m / 100m 512Mi / 512Mi data:21Gi 192.168.0.37/192.168.0.37 Sep 03,2025 16:14 UTC+0800
- scale-out
kbcli cluster scale-out etcd-eewrxk --components etcd --replicas 1 --auto-approve
OpsRequest etcd-eewrxk-horizontalscaling-ccpmp created successfully, you can view the progress:
kbcli cluster describe-ops etcd-eewrxk-horizontalscaling-ccpmp -n default
- See error
kubectl get cluster etcd-eewrxk
NAME CLUSTER-DEFINITION TERMINATION-POLICY STATUS AGE
etcd-eewrxk Delete Updating 92m
➜ ~ kubectl get pod
NAME READY STATUS RESTARTS AGE
etcd-eewrxk-etcd-0 2/2 Running 0 93m
etcd-eewrxk-etcd-1 2/2 Running 0 92m
etcd-eewrxk-etcd-2 2/2 Running 0 92m
etcd-eewrxk-etcd-3 2/2 Running 0 40m
➜ ~ kubectl get ops
NAME TYPE CLUSTER STATUS PROGRESS AGE
etcd-eewrxk-horizontalscaling-ccpmp HorizontalScaling etcd-eewrxk Running 0/1 41m
etcd-eewrxk-volumeexpansion-pjcdk VolumeExpansion etcd-eewrxk Succeed 3/3 90m
kbcli cluster list-instances etcd-eewrxk
NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME
etcd-eewrxk-etcd-0 default etcd-eewrxk etcd Running leader cn-guangzhou-b 100m / 100m 512Mi / 512Mi data:21Gi 192.168.0.37/192.168.0.37 Sep 03,2025 16:14 UTC+0800
etcd-eewrxk-etcd-1 default etcd-eewrxk etcd Running follower cn-guangzhou-b 100m / 100m 512Mi / 512Mi data:21Gi 192.168.0.37/192.168.0.37 Sep 03,2025 16:14 UTC+0800
etcd-eewrxk-etcd-2 default etcd-eewrxk etcd Running follower cn-guangzhou-b 100m / 100m 512Mi / 512Mi data:21Gi 192.168.0.37/192.168.0.37 Sep 03,2025 16:14 UTC+0800
etcd-eewrxk-etcd-3 default etcd-eewrxk etcd Running <none> cn-guangzhou-b 100m / 100m 512Mi / 512Mi data:21Gi 192.168.0.37/192.168.0.37 Sep 03,2025 17:06 UTC+0800
logs pod
kubectl logs etcd-eewrxk-etcd-3 --tail 20
Defaulted container "etcd" out of: etcd, kbagent, inject-bash (init), init-kbagent (init), kbagent-worker (init)
{"level":"warn","ts":"2025-09-03T09:48:48.530510Z","caller":"rafthttp/stream.go:652","msg":"request sent was ignored by remote peer due to cluster ID mismatch","remote-peer-id":"95d9be39c2616a56","remote-peer-cluster-id":"47c17342e3a1b4ec","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","error":"cluster ID mismatch"}
{"level":"warn","ts":"2025-09-03T09:48:48.613422Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"95d9be39c2616a56","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.613432Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"9a561d9e8f530bbc","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.613422Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"9a561d9e8f530bbc","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.613422Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"95d9be39c2616a56","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.613873Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"d0113665f403934f","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.613924Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"d0113665f403934f","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.619639Z","caller":"rafthttp/stream.go:652","msg":"request sent was ignored by remote peer due to cluster ID mismatch","remote-peer-id":"d0113665f403934f","remote-peer-cluster-id":"47c17342e3a1b4ec","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","error":"cluster ID mismatch"}
{"level":"warn","ts":"2025-09-03T09:48:48.619647Z","caller":"rafthttp/stream.go:652","msg":"request sent was ignored by remote peer due to cluster ID mismatch","remote-peer-id":"d0113665f403934f","remote-peer-cluster-id":"47c17342e3a1b4ec","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","error":"cluster ID mismatch"}
{"level":"warn","ts":"2025-09-03T09:48:48.619651Z","caller":"rafthttp/stream.go:652","msg":"request sent was ignored by remote peer due to cluster ID mismatch","remote-peer-id":"9a561d9e8f530bbc","remote-peer-cluster-id":"47c17342e3a1b4ec","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","error":"cluster ID mismatch"}
{"level":"warn","ts":"2025-09-03T09:48:48.630019Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"9a561d9e8f530bbc","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.630559Z","caller":"rafthttp/stream.go:652","msg":"request sent was ignored by remote peer due to cluster ID mismatch","remote-peer-id":"9a561d9e8f530bbc","remote-peer-cluster-id":"47c17342e3a1b4ec","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","error":"cluster ID mismatch"}
{"level":"warn","ts":"2025-09-03T09:48:48.630601Z","caller":"rafthttp/stream.go:652","msg":"request sent was ignored by remote peer due to cluster ID mismatch","remote-peer-id":"95d9be39c2616a56","remote-peer-cluster-id":"47c17342e3a1b4ec","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","error":"cluster ID mismatch"}
{"level":"warn","ts":"2025-09-03T09:48:48.630601Z","caller":"rafthttp/stream.go:652","msg":"request sent was ignored by remote peer due to cluster ID mismatch","remote-peer-id":"95d9be39c2616a56","remote-peer-cluster-id":"47c17342e3a1b4ec","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","error":"cluster ID mismatch"}
{"level":"warn","ts":"2025-09-03T09:48:48.713480Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"9a561d9e8f530bbc","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.713511Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"9a561d9e8f530bbc","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.713480Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"95d9be39c2616a56","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.713499Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"95d9be39c2616a56","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.713835Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"d0113665f403934f","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
{"level":"warn","ts":"2025-09-03T09:48:48.713899Z","caller":"rafthttp/http.go:502","msg":"request cluster ID mismatch","local-member-id":"fca7465b681861c4","local-member-cluster-id":"966fded6707566d3","local-member-server-version":"3.6.1","local-member-server-minimum-cluster-version":"3.0.0","remote-peer-server-name":"d0113665f403934f","remote-peer-server-version":"3.6.1","remote-peer-server-minimum-cluster-version":"3.0.0","remote-peer-cluster-id":"47c17342e3a1b4ec"}
kubectl logs etcd-eewrxk-etcd-3 --tail 20 kbagent
2025-09-03T09:06:20Z INFO create service Action {"actions": "switchover,memberJoin,memberLeave,dataDump,dataLoad,roleProbe"}
2025-09-03T09:06:20Z INFO create service Probe {"probes": "roleProbe"}
2025-09-03T09:06:20Z INFO create service Streaming {"actions": "dataDump,dataLoad"}
2025-09-03T09:06:20Z INFO service Action started...
2025-09-03T09:06:20Z INFO service Probe started...
2025-09-03T09:06:20Z INFO service Streaming started...
2025-09-03T09:06:20Z INFO starting the HTTP server
2025-09-03T09:06:20Z INFO register service to server {"service": "Action", "method": "POST", "uri": "/v1.0/action"}
2025-09-03T09:06:20Z INFO register service to server {"service": "Probe", "method": "POST", "uri": "/v1.0/probe"}
2025-09-03T09:06:20Z INFO register service to server {"service": "Streaming", "method": "POST", "uri": "/v1.0/streaming"}
2025-09-03T09:06:20Z INFO probe started {"probe": "roleProbe", "config": {"instance":"etcd-eewrxk-etcd","action":"roleProbe"}}
2025-09-03T09:06:20Z INFO starting the streaming server
2025-09-03T09:06:20Z INFO send probe event {"probe": "roleProbe", "probe": "roleProbe", "code": 0, "output": "", "message": ""}
2025-09-03T09:06:20Z INFO Action Executed {"action": "memberJoin", "result": ""}
2025-09-03T09:06:20Z INFO HTTP API Called {"user-agent": "Go-http-client/1.1", "method": "POST", "path": "/v1.0/action", "status code": 200, "cost": 13}
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
Additional context
Add any other context about the problem here.