enhancement(api)!: replace custom Health RPC with standard gRPC health service by pront · Pull Request #25139 · vectordotdev/vector

pront · 2026-04-07T18:20:47Z

Summary

Replace the custom (unreleased) ObservabilityService/Health RPC on the observability API (port 8686) with the standard grpc.health.v1.Health service. This enables native Kubernetes gRPC health probes, grpc-health-probe, and other standard gRPC health-checking tooling to work out of the box.

Added tonic_health standard health service to the gRPC server with reflection support
Removed the now-unused running: Arc<AtomicBool> parameter from the service and server

This change is required for vectordotdev/helm-charts#540,
which adds default gRPC readiness probes on port 8686. Kubernetes native gRPC probes call the standard
grpc.health.v1.Health/Check RPC with an empty service name, which tonic_health handles by default.

Vector configuration

No configuration changes required. The standard health service is automatically available on the
existing API address (default 0.0.0.0:8686).

How did you test this PR?

cargo check --features api,api-client passes
make check-clippy passes
make check-fmt passes
Verified tonic_health::server::health_reporter() responds to empty service name checks with
SERVING by default (same pattern already used and tested in the vector source)

Note: it's safe to ignore the failing proto check here since the affected proto file is not released yet.

Change Type

Is this a breaking change?

Yes
No

Does this PR include user facing changes?

Yes. Please add a changelog fragment based on our guidelines.
No. A maintainer will apply the no-changelog label to this PR.

References

Related: vectordotdev/helm-charts#540 - adds default gRPC readiness probes that depend on standard health service
Related: chore(api)!: replace GraphQL observability API with gRPC, remove async-graphql dependencies #24364 - moved observability API from GraphQL to gRPC
Related: fix(vector source): implement standard gRPC health checking protocol #24916 - added standard gRPC health service to the vector source

…h service Replace the custom ObservabilityService/Health RPC on the observability API with the standard grpc.health.v1.Health service. This enables native Kubernetes gRPC health probes, grpc-health-probe, and other standard tooling to work out of the box on port 8686. The vector-api-client now uses the standard health client for its health() method, maintaining the same API surface for vector top/tap. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Client::health() now checks response ServingStatus and returns NotServing error if not SERVING; uses named service "vector.observability.v1.ObservabilityService" instead of empty string - Store health_reporter handle in the server task and set NOT_SERVING when shutdown signal arrives, so in-flight probes see the transition - Fix stale doc example in lib.rs and grpcurl example in api.md - Add health integration test in tests/vector_api/health.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 447a821c92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

src/api/grpc_server.rs

Use only the empty service name ("") for health checks, which is the standard whole-server health slot. This is what Kubernetes gRPC probes, grpc-health-probe, and the Helm chart all query by default. Removes the named service registration since there is only one service on this port and no reason for service-specific health granularity today. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…grpc-health-check # Conflicts: # lib/vector-api-client/src/client.rs

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5cf0aeeeaf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-07T21:03:56Z

src/api/grpc_server.rs

                    rx.await.ok();
+                    // Signal NOT_SERVING before the server stops accepting requests,
+                    // so in-flight health probes see the transition during graceful drain.
+                    health_reporter
+                        .set_service_status("", tonic_health::ServingStatus::NotServing)


Flip gRPC health to NOT_SERVING before draining topology

This marks health as NOT_SERVING only inside the server shutdown future after rx.await, which is triggered when GrpcServer is dropped. On graceful shutdown, TopologyController::stop still awaits self.topology.stop().await before api_server is dropped, so the health endpoint stays SERVING during most of the drain window; with the old running-based health RPC removed in this commit, Kubernetes gRPC readiness probes can continue to pass until near process exit, keeping terminating pods routable longer than intended.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8584e75bd7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-07T21:11:28Z

src/api/grpc_server.rs

                    rx.await.ok();
+                    // Signal NOT_SERVING before the server stops accepting requests,
+                    // so in-flight health probes see the transition during graceful drain.
+                    health_reporter
+                        .set_service_status("", tonic_health::ServingStatus::NotServing)


Mark gRPC health NOT_SERVING before topology drain

The shutdown transition to NOT_SERVING is currently delayed until rx.await resolves inside serve_with_incoming_shutdown, which only happens when GrpcServer is dropped; on graceful shutdown, TopologyController::stop still awaits self.topology.stop().await before that drop, so readiness remains SERVING during most of drain and Kubernetes gRPC probes can keep routing to a terminating pod. Fresh evidence in this commit is that the previous running-based ObservabilityService::health path was removed, leaving no earlier readiness flip.

Useful? React with 👍 / 👎.

pront and others added 2 commits April 7, 2026 14:12

chore: fix formatting

7b5d74a

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

github-actions bot added the domain: topology Anything related to Vector's topology code label Apr 7, 2026

chore: update existing changelog to reference standard gRPC health check

f601fa3

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

pront added the no-changelog Changes in this PR do not need user-facing explanations in the release changelog label Apr 7, 2026

pront marked this pull request as ready for review April 7, 2026 18:57

pront requested review from a team as code owners April 7, 2026 18:57

chatgpt-codex-connector bot reviewed Apr 7, 2026

View reviewed changes

src/api/grpc_server.rs Outdated Show resolved Hide resolved

domalessi self-assigned this Apr 7, 2026

domalessi approved these changes Apr 7, 2026

View reviewed changes

pront and others added 2 commits April 7, 2026 16:54

Merge remote-tracking branch 'origin/master' into pavlos/standardize-…

8584e75

…grpc-health-check # Conflicts: # lib/vector-api-client/src/client.rs

chatgpt-codex-connector bot reviewed Apr 7, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enhancement(api)!: replace custom Health RPC with standard gRPC health service#25139

enhancement(api)!: replace custom Health RPC with standard gRPC health service#25139
pront wants to merge 6 commits intomasterfrom
pavlos/standardize-grpc-health-check

pront commented Apr 7, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Apr 7, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

pront commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Vector configuration

How did you test this PR?

Change Type

Is this a breaking change?

Does this PR include user facing changes?

References

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pront commented Apr 7, 2026 •

edited

Loading