From 10cc05cdf81d6408a8c1d8c02130432ae7f4ee4c Mon Sep 17 00:00:00 2001 From: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> Date: Mon, 25 Aug 2025 14:52:05 +0200 Subject: [PATCH 1/8] docs(tracing): add OpenTelemetry span format guide Add documentation describing the new OpenTelemetry-based span format for tracing, including configuration, key differences from the legacy format, migration steps, and important considerations around privacy and performance. Also add a test script to verify Jaeger integration with NeMo-Guardrails using OpenTelemetry, demonstrating trace export and event-span correlation. --- docs/user-guides/tracing/index.md | 59 +++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/docs/user-guides/tracing/index.md b/docs/user-guides/tracing/index.md index c7771fc41..fd99bfd08 100644 --- a/docs/user-guides/tracing/index.md +++ b/docs/user-guides/tracing/index.md @@ -11,6 +11,65 @@ With tracing, you can: - Debug flow execution and identify bottlenecks. - Analyze conversation patterns and errors. +## Span Formats + +Starting with NeMo Guardrails v0.16.0, the tracing system has transitioned to OpenTelemetry semantic conventions for Generative AI (GenAI), moving away from the legacy span format. This change enhances observability standardization and improves monitoring capabilities for AI workloads. + +**Reference Documentation:** + +- [OpenTelemetry Semantic Conventions for GenAI](https://opentelemetry.io/docs/specs/semconv/gen-ai/) - Overview of GenAI semantic conventions +- [GenAI Events Specification](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/) - Details on capturing user inputs and model outputs +- [GenAI Spans Specification](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/) - Span naming and attribute conventions + +### Current Formats + +- **OpenTelemetry** (`opentelemetry`) - **Recommended default format** following OpenTelemetry semantic conventions for GenAI +- **Legacy** (`legacy`) - Previous format with simple metrics dictionary (deprecated) + +### Configuration + +The span format is controlled through the tracing configuration: + +```yaml +tracing: + enabled: true + span_format: "opentelemetry" # default + enable_content_capture: false # default, see privacy considerations below +``` + +### Key Differences + +**OpenTelemetry Format:** + +- Follows standardized [semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) +- Rich structured attributes and events (e.g., `gen_ai.request.model`, `gen_ai.usage.input_tokens`) +- Enhanced LLM call tracking with provider and model information +- Support for span events and error tracking +- Compatible with OpenTelemetry ecosystem tools + +**Legacy Format:** + +- Simple metrics dictionary +- Minimal overhead but limited observability +- **Deprecated** - maintained for backward compatibility only + +### Important Considerations + +**Development Status**: The [OpenTelemetry semantic conventions for GenAI](https://opentelemetry.io/docs/specs/semconv/gen-ai/) are currently in development and may undergo changes. Consider the following risks: + +- **Evolving Standards**: Conventions may change as they mature, potentially affecting existing implementations +- **Data Privacy**: The `enable_content_capture` option captures user inputs and model outputs, which may include sensitive information (PII). Only enable when necessary and ensure compliance with data protection regulations. See [GenAI Events documentation](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/) for details +- **Performance Impact**: Extensive telemetry collection may impact system performance, especially with large inputs/outputs + +### Migration Path + +Existing configurations will continue to work, but we strongly recommend migrating to the OpenTelemetry format: + +1. Update your configuration to use `span_format: "opentelemetry"` +2. Review your telemetry backends for compatibility with OpenTelemetry conventions +3. Test thoroughly, as span structures and attribute names have changed +4. Consider privacy implications if enabling content capture + ## Contents - [](quick-start.md) - Minimal setup to enable tracing using the OpenTelemetry SDK From 669f9d3c41ed1f50051bef19998aeef5b1fa2059 Mon Sep 17 00:00:00 2001 From: Pouyan <13303554+Pouyanpi@users.noreply.github.com> Date: Thu, 28 Aug 2025 10:09:00 +0200 Subject: [PATCH 2/8] Update docs/user-guides/tracing/index.md Co-authored-by: Miyoung Choi Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com> --- docs/user-guides/tracing/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guides/tracing/index.md b/docs/user-guides/tracing/index.md index fd99bfd08..192338882 100644 --- a/docs/user-guides/tracing/index.md +++ b/docs/user-guides/tracing/index.md @@ -28,7 +28,7 @@ Starting with NeMo Guardrails v0.16.0, the tracing system has transitioned to Op ### Configuration -The span format is controlled through the tracing configuration: +You can control the span format through the tracing configuration: ```yaml tracing: From e22b8ed92fb59f21e5ca0fec129d112b556a7a97 Mon Sep 17 00:00:00 2001 From: Pouyan <13303554+Pouyanpi@users.noreply.github.com> Date: Thu, 28 Aug 2025 10:09:05 +0200 Subject: [PATCH 3/8] Update docs/user-guides/tracing/index.md Co-authored-by: Miyoung Choi Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com> --- docs/user-guides/tracing/index.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/user-guides/tracing/index.md b/docs/user-guides/tracing/index.md index 192338882..096e14424 100644 --- a/docs/user-guides/tracing/index.md +++ b/docs/user-guides/tracing/index.md @@ -21,7 +21,9 @@ Starting with NeMo Guardrails v0.16.0, the tracing system has transitioned to Op - [GenAI Events Specification](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/) - Details on capturing user inputs and model outputs - [GenAI Spans Specification](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/) - Span naming and attribute conventions -### Current Formats +### Supported Formats + +NeMo Guardrails support the following formats. - **OpenTelemetry** (`opentelemetry`) - **Recommended default format** following OpenTelemetry semantic conventions for GenAI - **Legacy** (`legacy`) - Previous format with simple metrics dictionary (deprecated) From 0f67d972544d342984efdc15346d2fb22a392e31 Mon Sep 17 00:00:00 2001 From: Pouyan <13303554+Pouyanpi@users.noreply.github.com> Date: Thu, 28 Aug 2025 10:09:13 +0200 Subject: [PATCH 4/8] Update docs/user-guides/tracing/index.md Co-authored-by: Miyoung Choi Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com> --- docs/user-guides/tracing/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guides/tracing/index.md b/docs/user-guides/tracing/index.md index 096e14424..9878bb13b 100644 --- a/docs/user-guides/tracing/index.md +++ b/docs/user-guides/tracing/index.md @@ -43,7 +43,7 @@ tracing: **OpenTelemetry Format:** -- Follows standardized [semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) +- Follows the standardized [semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) - Rich structured attributes and events (e.g., `gen_ai.request.model`, `gen_ai.usage.input_tokens`) - Enhanced LLM call tracking with provider and model information - Support for span events and error tracking From 9beea159d053fa45ebf9ee64bfad2c588d3a5833 Mon Sep 17 00:00:00 2001 From: Pouyan <13303554+Pouyanpi@users.noreply.github.com> Date: Thu, 28 Aug 2025 10:09:25 +0200 Subject: [PATCH 5/8] Update docs/user-guides/tracing/index.md Co-authored-by: Miyoung Choi Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com> --- docs/user-guides/tracing/index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/user-guides/tracing/index.md b/docs/user-guides/tracing/index.md index 9878bb13b..bae2875c4 100644 --- a/docs/user-guides/tracing/index.md +++ b/docs/user-guides/tracing/index.md @@ -41,6 +41,8 @@ tracing: ### Key Differences +The following are the key differences between the supported span formats. + **OpenTelemetry Format:** - Follows the standardized [semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) From 989c0ada7c6196dcd52a531b63d171ca486af03b Mon Sep 17 00:00:00 2001 From: Pouyan <13303554+Pouyanpi@users.noreply.github.com> Date: Thu, 28 Aug 2025 10:09:36 +0200 Subject: [PATCH 6/8] Update docs/user-guides/tracing/index.md Co-authored-by: Miyoung Choi Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com> --- docs/user-guides/tracing/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guides/tracing/index.md b/docs/user-guides/tracing/index.md index bae2875c4..6708ca490 100644 --- a/docs/user-guides/tracing/index.md +++ b/docs/user-guides/tracing/index.md @@ -54,7 +54,7 @@ The following are the key differences between the supported span formats. **Legacy Format:** - Simple metrics dictionary -- Minimal overhead but limited observability +- Minimal overhead with limited observability - **Deprecated** - maintained for backward compatibility only ### Important Considerations From e9937ed52e48783e335ae338d1fe6edd7cad94fd Mon Sep 17 00:00:00 2001 From: Pouyan <13303554+Pouyanpi@users.noreply.github.com> Date: Thu, 28 Aug 2025 10:09:43 +0200 Subject: [PATCH 7/8] Update docs/user-guides/tracing/index.md Co-authored-by: Miyoung Choi Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com> --- docs/user-guides/tracing/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guides/tracing/index.md b/docs/user-guides/tracing/index.md index 6708ca490..4e7c7198e 100644 --- a/docs/user-guides/tracing/index.md +++ b/docs/user-guides/tracing/index.md @@ -46,7 +46,7 @@ The following are the key differences between the supported span formats. **OpenTelemetry Format:** - Follows the standardized [semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) -- Rich structured attributes and events (e.g., `gen_ai.request.model`, `gen_ai.usage.input_tokens`) +- Rich-structured attributes and events (e.g., `gen_ai.request.model`, `gen_ai.usage.input_tokens`) - Enhanced LLM call tracking with provider and model information - Support for span events and error tracking - Compatible with OpenTelemetry ecosystem tools From dc210fce7fef28fa99766614d6af445bc2328eee Mon Sep 17 00:00:00 2001 From: Pouyan <13303554+Pouyanpi@users.noreply.github.com> Date: Thu, 28 Aug 2025 10:09:50 +0200 Subject: [PATCH 8/8] Update docs/user-guides/tracing/index.md Co-authored-by: Miyoung Choi Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com> --- docs/user-guides/tracing/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guides/tracing/index.md b/docs/user-guides/tracing/index.md index 4e7c7198e..7b03e6437 100644 --- a/docs/user-guides/tracing/index.md +++ b/docs/user-guides/tracing/index.md @@ -67,7 +67,7 @@ The following are the key differences between the supported span formats. ### Migration Path -Existing configurations will continue to work, but we strongly recommend migrating to the OpenTelemetry format: +Existing configurations will continue to work. However, it is strongly recommended to migrate to the OpenTelemetry format. Migration steps are: 1. Update your configuration to use `span_format: "opentelemetry"` 2. Review your telemetry backends for compatibility with OpenTelemetry conventions