Skip to content

Add graceful shutdown with connection cleanup #210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Run Tests

on:
push:
branches: [ main, master ]
pull_request:
branches: [ '**' ]

jobs:
test:
name: Run Tests
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.21'
cache: true

- name: Install dependencies
run: go mod vendor

- name: Run tests
run: make test

- name: Run race condition tests
run: make test.race
5 changes: 4 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
FROM golang:1.23-alpine3.21 AS builder
FROM --platform=${BUILDPLATFORM} golang:1.23-alpine3.21 AS builder

RUN apk add --no-cache alpine-sdk ca-certificates

ARG TARGETARCH
ARG TARGETOS
ARG VERSION

ENV CGO_ENABLED=0 \
Expand All @@ -12,6 +14,7 @@ WORKDIR /go/src/github.com/grepplabs/kafka-proxy
COPY . .

RUN mkdir -p build && \
GOOS=${TARGETOS} GOARCH=${TARGETARCH} \
go build -mod=vendor -o build/kafka-proxy -ldflags "${LDFLAGS}" .

FROM alpine:3.21
Expand Down
9 changes: 9 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ GOPKGS = $(shell go list ./... | grep -v /vendor/)
BUILD_FLAGS ?=
LDFLAGS ?= -X github.com/grepplabs/kafka-proxy/config.Version=$(VERSION) -w -s
TAG ?= "v0.4.3"
REPO ?= "grepplabs/kafka-proxy"

PROTOC_GO_VERSION ?= v1.33
PROTOC_GRPC_VERSION ?= v1.2
Expand Down Expand Up @@ -55,6 +56,14 @@ docker.build:
docker.build.all:
docker build --build-arg VERSION=$(VERSION) -t local/kafka-proxy -f Dockerfile.all .

docker.build.multiarch:
docker buildx build \
--platform linux/amd64,linux/arm64 \
--push \
--build-arg VERSION=$(VERSION) \
-t $(REPO):$(TAG) \
.

tag:
git tag $(TAG)

Expand Down
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,7 @@ You can launch a kafka-proxy container with auth-ldap plugin for trying it out w
--sasl-plugin-param stringArray Authentication plugin parameter
--sasl-plugin-timeout duration Authentication timeout (default 10s)
--sasl-username string SASL user name
--shutdown-timeout duration Maximum time to wait for graceful shutdown to complete (default 30s)
--tls-ca-chain-cert-file string PEM encoded CA's certificate file
--tls-client-cert-file string PEM encoded file with client certificate
--tls-client-key-file string PEM encoded file with private key for the client certificate
Expand Down Expand Up @@ -447,8 +448,25 @@ By setting `--proxy-listener-tls-client-cert-validate-subject true`, Kafka Proxy
--proxy-listener-tls-client-cert-validate-subject true \
--proxy-listener-tls-required-client-subject-country DE \
--proxy-listener-tls-required-client-subject-organization grepplabs

### Graceful Shutdown

Kafka-proxy implements graceful shutdown to ensure that active connections are properly closed when the proxy is terminated. When a termination signal (SIGINT or SIGTERM) is received, the proxy will:

1. Stop accepting new connections
2. Wait for existing connections to complete their current operations
3. Close all connections cleanly before exiting

You can configure the maximum time the proxy will wait during shutdown with the `--shutdown-timeout` parameter:

```
kafka-proxy server \
--bootstrap-server-mapping "kafka-0.example.com:9092,127.0.0.1:32500" \
--shutdown-timeout 60s
```

The default timeout is 30 seconds. If active connections take longer than the specified timeout to close, the proxy will force termination after the timeout period.

### Kubernetes sidecar container example

```yaml
Expand Down
Loading