access_log: add OTLP/HTTP exporter to OpenTelemetry access logger

Adds HTTP transport support to the OpenTelemetry access logger, complementing
#29207 (OTLP/HTTP tracing). This enables direct OTLP log export to backends
that only accept HTTP (Dynatrace, Datadog, Logfire), without requiring an
intermediate collector sidecar.

Changes:
- Add http_service field for OTLP/HTTP transport configuration
- Add top-level grpc_service, log_name, buffer_flush_interval, buffer_size_bytes,
  filter_state_objects_to_log, and custom_tags fields to match tracer config pattern
- Add field_migrate.oneof_promotion annotations for future oneof migration
- Add helper functions getLogName() and getGrpcService() with fallback to common_config
- Add transport validation (exactly one transport must be configured)
- Deprecate common_config field (still functional for backward compatibility)
- Extract shared utilities (otlp_log_utils) from duplicated HTTP/gRPC code

Fixes #26352 (access logs portion)

Signed-off-by: Adrian Cole <[email protected]>

Author: codefromthecrypt

TESTING.md

@@ -0,0 +1,150 @@
+# Running OTLP Access Logger Tests
+
+## macOS Setup
+
+On macOS, do NOT use `--config=clang` in `user.bazelrc`. The default Xcode clang works without additional configuration.
+
+## Running Tests
+
+Config tests:
+
+```bash
+bazel test //test/extensions/access_loggers/open_telemetry:config_test --test_output=errors
+```
+
+HTTP access log exporter tests:
+
+```bash
+bazel test //test/extensions/access_loggers/open_telemetry:http_access_log_impl_test --test_output=errors
+```
+
+HTTP access log integration test:
+
+```bash
+bazel test //test/extensions/access_loggers/open_telemetry:http_access_log_integration_test --test_output=errors
+```
+
+All OTLP access logger tests:
+
+```bash
+bazel test //test/extensions/access_loggers/open_telemetry/... --test_output=errors
+```
+
+## Building Envoy
+
+Build with crash recovery:
+
+```bash
+cd ~/oss/envoy
+nohup bazel build envoy 2>&1 | tee build.log &
+```
+
+Using `nohup` with `tee` ensures:
+- Build continues if your terminal/session crashes
+- Output is saved to `build.log` for review
+- You can resume monitoring with `tail -10 build.log`
+
+**WARNING**: A full build takes a long time (~7,000 actions with cache, 16,000+ from scratch).
+Expect 45-90 minutes on a modern M-series Mac, longer on slower machines.
+
+### Monitoring Progress
+
+Poll periodically with last 10 lines only (saves context):
+
+```bash
+tail -10 build.log
+```
+
+**DO NOT use `tail -f`** - it wastes context by streaming everything.
+
+Only read the full log if the build fails:
+
+```bash
+# If build failed, then read full log
+cat build.log
+```
+
+### If Session Crashes
+
+```bash
+# Check if build is still running
+pgrep -f "bazel build envoy"
+
+# Resume monitoring (poll, don't follow)
+tail -10 ~/oss/envoy/build.log
+```
+
+### Verify Build
+
+```bash
+bazel-bin/source/exe/envoy-static --version
+```
+
+Expected output format:
+```
+bazel-bin/source/exe/envoy-static  version: <commit>/<version>-dev/Modified/DEBUG/BoringSSL
+```
+
+## Manual Testing with OTLP HTTP Collector
+
+1. Start a local OTLP HTTP collector on port 4318 (e.g., otel-tui or OpenTelemetry Collector)
+
+2. Run Envoy with the example config (note the `OTEL_RESOURCE_ATTRIBUTES` env var for service name):
+
+```bash
+OTEL_RESOURCE_ATTRIBUTES=service.name=envoy-example bazel-bin/source/exe/envoy-static -c configs/envoy-otel-http.yaml
+```
+
+3. Send a test request:
+
+```bash
+curl localhost:10080/get
+```
+
+4. Wait at least 2 seconds before killing Envoy for **logs** to be exported. The HTTP access logger buffers logs and flushes every 1 second. If you kill Envoy immediately, buffered logs may not be sent. **Traces are sent immediately** (no buffering).
+
+```bash
+# Example: send request, wait for log flush, then stop
+curl localhost:10080/get
+sleep 2
+pkill -f envoy-static
+```
+
+5. Verify logs and traces are received by your collector.
+
+The example config (`configs/envoy-otel-http.yaml`) demonstrates:
+- HTTP transport for OTLP access logs
+- HTTP transport for OTLP traces with 100% sampling
+- Log body with request details
+- `resource_detectors` for consistent `service.name` across all signals via `OTEL_RESOURCE_ATTRIBUTES` env var
+
+## Coding Style Points from PR Reviews
+
+Key patterns learned from reviewing 25+ Envoy PRs with extensive review comments:
+
+### Proto Design
+- **Never use `oneof` for existing fields** - Adding `oneof` is a breaking change per API_VERSIONING.md. Use separate optional fields with runtime validation and `field_migrate.oneof_promotion` annotation. (PR #29207, #29289)
+- **Document mutual exclusivity clearly** - State "Only one of X or Y may be set" in both field comments. (PR #29207)
+- **Document behavior when fields are unset** - Required vs optional fields need clear defaults. (PR #39628, #28490)
+
+### Code Style
+- **Comments end with periods** - Envoy style requires punctuation on comments. (PR #37172)
+- **Use `constexpr` for constants** - Avoid runtime initialization with `const`. (PR #32490, #31507)
+- **Prefer `absl::string_view` over `const std::string&`** where appropriate. (PR #39628)
+- **Magic numbers should be named constants** - Use `constexpr` with descriptive names. (PR #36581)
+- **Log levels matter** - Use `warn` for recoverable failures, `error` for configuration issues, `debug` for routine events. (PR #29207, #37172)
+
+### Testing
+- **Add descriptive comments to tests** - Explain what each test verifies. (PR #37172, #30479)
+- **Test config rejection** - Always test that invalid configurations are properly rejected. (PR #29289)
+- **Verify all headers in HTTP tests** - Check method, path, content-type, custom headers. (PR #29207)
+
+### Async/HTTP Patterns
+- **Track in-flight requests** - Use `Http::AsyncClientRequestTracker` to cancel pending requests on destruction. (PR #29207)
+- **Parse headers at construction time** - Avoid copies per request. (PR #29207)
+- **Use `setReference()` for headers** - Not `setCopy()`, to avoid allocations. (PR #29207)
+
+### Avoiding Common Mistakes
+- **Don't hardcode values that appear elsewhere** - Use existing constants. (PR #36581)
+- **Reject bad config rather than silently fail** - Throw exception at config time. (PR #37172, #39628)
+- **Consider performance impact** - Run benchmarks for shared code paths. (PR #29709)

api/envoy/extensions/access_loggers/open_telemetry/v3/BUILD

@@ -8,6 +8,7 @@ api_proto_package(
     deps = [
         "//envoy/config/core/v3:pkg",
         "//envoy/extensions/access_loggers/grpc/v3:pkg",
+        "//envoy/type/tracing/v3:pkg",
         "@com_github_cncf_xds//udpa/annotations:pkg",
         "@opentelemetry_proto//:common_proto",
     ],

api/envoy/extensions/access_loggers/open_telemetry/v3/logs_service.proto

@@ -3,30 +3,61 @@ syntax = "proto3";
 package envoy.extensions.access_loggers.open_telemetry.v3;
 
 import "envoy/config/core/v3/extension.proto";
+import "envoy/config/core/v3/grpc_service.proto";
+import "envoy/config/core/v3/http_service.proto";
 import "envoy/extensions/access_loggers/grpc/v3/als.proto";
+import "envoy/type/tracing/v3/custom_tag.proto";
+
+import "google/protobuf/duration.proto";
+import "google/protobuf/wrappers.proto";
 
 import "opentelemetry/proto/common/v1/common.proto";
 
+import "udpa/annotations/migrate.proto";
 import "udpa/annotations/status.proto";
-import "validate/validate.proto";
 
 option java_package = "io.envoyproxy.envoy.extensions.access_loggers.open_telemetry.v3";
 option java_outer_classname = "LogsServiceProto";
 option java_multiple_files = true;
 option go_package = "github.com/envoyproxy/go-control-plane/envoy/extensions/access_loggers/open_telemetry/v3;open_telemetryv3";
 option (udpa.annotations.file_status).package_version_status = ACTIVE;
 
-// [#protodoc-title: OpenTelemetry (gRPC) Access Log]
+// [#protodoc-title: OpenTelemetry Access Log]
 
 // Configuration for the built-in ``envoy.access_loggers.open_telemetry``
 // :ref:`AccessLog <envoy_v3_api_msg_config.accesslog.v3.AccessLog>`. This configuration will
 // populate `opentelemetry.proto.collector.v1.logs.ExportLogsServiceRequest.resource_logs <https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/collector/logs/v1/logs_service.proto>`_.
 // In addition, the request start time is set in the dedicated field.
 // [#extension: envoy.access_loggers.open_telemetry]
-// [#next-free-field: 8]
+// [#next-free-field: 16]
 message OpenTelemetryAccessLogConfig {
   // [#comment:TODO(itamarkam): add 'filter_state_objects_to_log' to logs.]
-  grpc.v3.CommonGrpcAccessLogConfig common_config = 1 [(validate.rules).message = {required: true}];
+  // [#deprecated:]
+  // DEPRECATED: Use ``grpc_service`` and other top-level fields instead.
+  // This field is kept for backward compatibility with existing gRPC configurations.
+  // Note: Only one of ``common_config``, ``grpc_service``, or ``http_service`` may be used.
+  grpc.v3.CommonGrpcAccessLogConfig common_config = 1
+      [(udpa.annotations.field_migrate).oneof_promotion = "otlp_exporter"];
+
+  // The upstream HTTP cluster that will receive OTLP logs via
+  // `OTLP/HTTP <https://opentelemetry.io/docs/specs/otlp/#otlphttp>`_.
+  // Note: Only one of ``common_config``, ``grpc_service``, or ``http_service`` may be used.
+  //
+  // .. note::
+  //
+  //   The ``request_headers_to_add`` property in the OTLP HTTP exporter service
+  //   does not support the :ref:`format specifier <config_access_log_format>` as used for
+  //   :ref:`HTTP access logging <config_access_log>`.
+  //   The values configured are added as HTTP headers on the OTLP export request
+  //   without any formatting applied.
+  config.core.v3.HttpService http_service = 8
+      [(udpa.annotations.field_migrate).oneof_promotion = "otlp_exporter"];
+
+  // The upstream gRPC cluster that will receive OTLP logs.
+  // Note: Only one of ``common_config``, ``grpc_service``, or ``http_service`` may be used.
+  // This field is preferred over ``common_config.grpc_service``.
+  config.core.v3.GrpcService grpc_service = 9
+      [(udpa.annotations.field_migrate).oneof_promotion = "otlp_exporter"];
 
   // If specified, Envoy will not generate built-in resource labels
   // like ``log_name``, ``zone_name``, ``cluster_name``, ``node_name``.
@@ -57,4 +88,33 @@ message OpenTelemetryAccessLogConfig {
   // See the formatters extensions documentation for details.
   // [#extension-category: envoy.formatter]
   repeated config.core.v3.TypedExtensionConfig formatters = 7;
+
+  // Friendly identifier for the access log, used as a resource attribute.
+  // This field is preferred over ``common_config.log_name``.
+  string log_name = 10;
+
+  // The interval for flushing access logs to the transport. Default: 1 second.
+  // This field is preferred over ``common_config.buffer_flush_interval``.
+  google.protobuf.Duration buffer_flush_interval = 11;
+
+  // Soft size limit in bytes for the access log buffer. When the buffer exceeds
+  // this limit, logs will be flushed. Default: 16KB.
+  // This field is preferred over ``common_config.buffer_size_bytes``.
+  google.protobuf.UInt32Value buffer_size_bytes = 12;
+
+  // Additional filter state objects to log.
+  // This field is preferred over ``common_config.filter_state_objects_to_log``.
+  repeated string filter_state_objects_to_log = 13;
+
+  // Optional custom tags to include in the log record.
+  // This field is preferred over ``common_config.custom_tags``.
+  repeated type.tracing.v3.CustomTag custom_tags = 14;
+
+  // An ordered list of resource detectors for populating resource attributes.
+  // This enables using the ``OTEL_RESOURCE_ATTRIBUTES`` environment variable for
+  // ``service.name`` and other resource attributes, matching the tracer and
+  // metrics sink patterns. When configured, the detected resources are merged
+  // with any static ``resource_attributes``, with detector values taking precedence.
+  // [#extension-category: envoy.tracers.opentelemetry.resource_detectors]
+  repeated config.core.v3.TypedExtensionConfig resource_detectors = 15;
 }

changelogs/current.yaml

@@ -458,5 +458,26 @@ new_features:
 - area: attributes
   change: |
     added :ref:`attributes <arch_overview_attributes>` for looking up request or response headers bytes.
+- area: access_log
+  change: |
+    Added support for exporting OpenTelemetry access logs via HTTP and refactored the configuration
+    to match the tracer config pattern. New top-level fields ``http_service``, ``grpc_service``,
+    ``log_name``, ``buffer_flush_interval``, ``buffer_size_bytes``, ``filter_state_objects_to_log``,
+    and ``custom_tags`` provide a cleaner, transport-agnostic configuration.
+    The ``common_config`` field is deprecated but remains functional for backward compatibility.
+    See :ref:`http_service <envoy_v3_api_field_extensions.access_loggers.open_telemetry.v3.OpenTelemetryAccessLogConfig.http_service>`
+    and :ref:`grpc_service <envoy_v3_api_field_extensions.access_loggers.open_telemetry.v3.OpenTelemetryAccessLogConfig.grpc_service>`.
+- area: access_log
+  change: |
+    Added ``resource_detectors`` to the OpenTelemetry access logger configuration. This enables
+    consistent resource attribute configuration across all signals (tracing, metrics, access logs)
+    using the ``OTEL_RESOURCE_ATTRIBUTES`` environment variable via the environment resource detector.
+    See :ref:`resource_detectors <envoy_v3_api_field_extensions.access_loggers.open_telemetry.v3.OpenTelemetryAccessLogConfig.resource_detectors>`.
 
 deprecated:
+- area: access_log
+  change: |
+    The ``common_config`` field in
+    :ref:`OpenTelemetryAccessLogConfig <envoy_v3_api_msg_extensions.access_loggers.open_telemetry.v3.OpenTelemetryAccessLogConfig>`
+    is deprecated. Use the new top-level fields ``grpc_service``, ``log_name``, ``buffer_flush_interval``,
+    ``buffer_size_bytes``, ``filter_state_objects_to_log``, and ``custom_tags`` instead.