Skip to content

Conversation

@codefromthecrypt
Copy link
Contributor

@codefromthecrypt codefromthecrypt commented Dec 7, 2025

Commit Message: access_log: add OTLP/HTTP exporter to OpenTelemetry access logger

Additional Description: Adds HTTP transport support to the OpenTelemetry access logger, complementing #29207 (OTLP/HTTP tracing). This enables direct OTLP log export to backends that only accept HTTP (Dynatrace, Datadog, Logfire), without requiring an intermediate collector sidecar.

Also refactors the config to match the tracer pattern:

  • Add top-level http_service and grpc_service fields
  • Add top-level log_name, buffer_flush_interval, buffer_size_bytes, filter_state_objects_to_log, and custom_tags fields
  • Add top-level resource_detectors field for dynamic resource attributes (e.g., environment detector)
  • Deprecate common_config field (still functional for backward compatibility)
  • Extract shared utilities (otlp_log_utils) from duplicated code between HTTP and gRPC implementations

Risk Level: Low (new optional feature, backward compatible)

Testing: Unit tests, integration test, manual testing with OTel collector

Docs Changes: Proto documentation updated

Release Notes: Added

Fixes #26352 (access logs portion)

Example Configuration (OTLP/HTTP with resource_detectors)

# Run with: OTEL_RESOURCE_ATTRIBUTES=service.name=my-service
access_log:
- name: envoy.access_loggers.open_telemetry
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.access_loggers.open_telemetry.v3.OpenTelemetryAccessLogConfig
    http_service:
      http_uri:
        uri: "http://localhost:4318/v1/logs"
        cluster: otel_collector
        timeout: 1s
    log_name: envoy_access_log
    resource_detectors:
    - name: envoy.tracers.opentelemetry.resource_detectors.environment
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.tracers.opentelemetry.resource_detectors.v3.EnvironmentResourceDetectorConfig
    body:
      string_value: "%REQ(:METHOD)% %REQ(:PATH)% %RESPONSE_CODE%"

Example Configuration (OTLP/gRPC - after this PR is similar to tracing)

access_log:
- name: envoy.access_loggers.open_telemetry
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.access_loggers.open_telemetry.v3.OpenTelemetryAccessLogConfig
    grpc_service:
      envoy_grpc:
        cluster_name: otel_collector
    log_name: envoy_access_log
    body:
      string_value: "%REQ(:METHOD)% %REQ(:PATH)% %RESPONSE_CODE%"

FAQ

Why add resource_detectors instead of propagating resource_attributes?

The resource_attributes field in the old common_config was a one-off pattern that no other signal uses. By adopting resource_detectors, access logs now follow the same configuration style as tracing and metrics:

How do I set service.name consistently across all signals?

Use the OTEL_RESOURCE_ATTRIBUTES environment variable with the environment resource detector:

OTEL_RESOURCE_ATTRIBUTES=service.name=my-envoy envoy -c config.yaml

Then configure all signals to use the environment detector:

resource_detectors:
- name: envoy.tracers.opentelemetry.resource_detectors.environment
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.tracers.opentelemetry.resource_detectors.v3.EnvironmentResourceDetectorConfig

This is the OTel-standard way to configure resource attributes and eliminates the need to duplicate service.name in multiple config sections.


Screenshots

Example was added and here are screen shots proving not just access logs are there, but linked to the right trace:
Screenshot 2025-12-07 at 2 03 23 PM
Screenshot 2025-12-07 at 2 03 12 PM

@repokitteh-read-only
Copy link

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #42445 was opened by codefromthecrypt.

see: more, trace.

@repokitteh-read-only
Copy link

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to (api/envoy/|docs/root/api-docs/).
envoyproxy/api-shepherds assignee is @mattklein123
CC @envoyproxy/api-watchers: FYI only for changes made to (api/envoy/|docs/root/api-docs/).

🐱

Caused by: #42445 was opened by codefromthecrypt.

see: more, trace.

@codefromthecrypt codefromthecrypt force-pushed the otlp-http-access-logs branch 3 times, most recently from 8831406 to 3664f4e Compare December 7, 2025 03:23
@codefromthecrypt codefromthecrypt marked this pull request as ready for review December 7, 2025 03:46
@codefromthecrypt
Copy link
Contributor Author

@wbpcode @Kludex fyi this is for extending otel logging from grpc-only in envoy to also http as many backends prefer or only support http OTLP

@codefromthecrypt
Copy link
Contributor Author

moving to draft as I noticed an inconsistency in how grpc access log worked wrt service names vs all other signals, and accidentally promoted that to a normalized practice across http when I should fix that goof

@codefromthecrypt codefromthecrypt marked this pull request as draft December 7, 2025 04:44
@codefromthecrypt codefromthecrypt force-pushed the otlp-http-access-logs branch 2 times, most recently from a50beca to 7adc68b Compare December 7, 2025 06:17
resource_detectors:
- name: envoy.tracers.opentelemetry.resource_detectors.environment
typed_config:
"@type": type.googleapis.com/envoy.extensions.tracers.opentelemetry.resource_detectors.v3.EnvironmentResourceDetectorConfig
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok this now uses the current practice with is resource detectors, not a one-off, even though the one-off in grpc+accesslog is retained for deprecation

Adds HTTP transport support to the OpenTelemetry access logger,
complementing OTLP/HTTP tracing. This enables direct OTLP log export to
backends that only accept HTTP (Dynatrace, Datadog, Logfire), without
requiring an intermediate collector sidecar.

Also refactors the config to match the tracer pattern:
- Add top-level http_service and grpc_service fields
- Add top-level log_name, buffer_flush_interval, buffer_size_bytes,
  filter_state_objects_to_log, and custom_tags fields
- Add top-level resource_detectors field for dynamic resource attributes
- Deprecate common_config field (still functional for backward compat)
- Extract shared utilities (otlp_log_utils) from duplicated code

Risk Level: Low (new optional feature, backward compatible)
Testing: Unit tests, integration test, manual testing with OTel collector

Fixes envoyproxy#26352 (access logs portion)

Signed-off-by: Adrian Cole <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for sending OpenTelemetry Tracing over HTTP/Protobuf endpoint

3 participants