Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

otel Datadog exporter is not propagating trace context from upstream spans #6560

Open
samuelAndalon opened this issue Jan 16, 2025 · 0 comments

Comments

@samuelAndalon
Copy link
Contributor

samuelAndalon commented Jan 16, 2025

Describe the bug

Datadog is not showing parent spans of the router callers, it only shows downstream spans

Image

in our distributed system there are 2 services on top of the router:

Image

after some debugging found that, ServiceB is properly sending x-datadog-trace-id and x-datadog-parent-id, the issue not there, it seems that problem is that the custom Apollo DatadogExporter is not assigning the x-datadog-parent-id value after it gets extracted to the span parent id.

if I send a graphQL request to the router with these DD headers.

Image

the headers are properly extracted:

Image

however here is where issue starts:

The custom Apollo oTel Datadog exporter is wrongly assigning x-datadog-parent-id as span-id of the incoming request span, see:
https://github.com/apollographql/router/blob/main/apollo-router/src/plugins/telemetry/tracing/datadog_exporter/mod.rs#L400

The SpanData struct has a parent_span_id field, and a span_context, this span context is the one that exposes the span_id and the trace_id.

Expected behavior

Datadog should show all the spans that belong to a given trace-id, included upstream services that are calling the router.

@samuelAndalon samuelAndalon changed the title otel Datadog exporter is not propagating context from upstream spans otel Datadog exporter is not propagating trace context from upstream spans Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant