Nothing exported when I use tracing::subscriber::set_global_default

[joseph-henry]

Hello.

I'm trying to set the default subscriber for my entire program using set_global_default but am not getting the result that I expect. I can use set_default successfully, and I can use .with_default in a closure successfully.

When I use method 1 or 2 in the code below, my tracing events are sent to my collector, but when I use method 1 (set_global_default) nothing is sent to my collector and no error is emitted when I check its state. Any help would be greatly appreciated. Thanks!

Someone recently suggested adding a call to the provider's .shutdown() which makes sense to me, but that unfortunately makes everything stop working.

I have already raised the issue with the tracing team as I initially thought the problem was with that, but I think now we know it is not.

Any help would be greatly appreciated!

use opentelemetry::trace::TracerProvider as _;
use opentelemetry::KeyValue;
use opentelemetry_otlp::WithExportConfig;
use opentelemetry_sdk::trace::TracerProvider;
use opentelemetry_sdk::{runtime, Resource};
use opentelemetry_semantic_conventions::resource::SERVICE_NAME;
use tracing::{error, info, span, trace};
use tracing_subscriber::layer::SubscriberExt;
use tracing_subscriber::Registry;

#[tokio::main]
async fn main() {
    let exporter = opentelemetry_otlp::SpanExporter::builder()
        .with_tonic()
        .with_endpoint("http://localhost:4317")
        .build()
        .unwrap();

    let provider = TracerProvider::builder()
        .with_batch_exporter(exporter, runtime::Tokio)
        .with_resource(Resource::new(vec![KeyValue::new(
            SERVICE_NAME,
            "test-service",
        )]))
        .build();

    let tracer = provider.tracer("global_tracer");
    let telemetry = tracing_opentelemetry::layer().with_tracer(tracer.clone());
    let subscriber = Registry::default().with(telemetry);

    // Method 1: Set subscribe as global default (Doesn't work)
    tracing::subscriber::set_global_default(subscriber).expect("setting default subscriber failed");
    let root = span!(tracing::Level::TRACE, "app_start", work_units = 2);
    let _enter = root.enter();
    trace!("This is a trace event.");
    error!("This is an error event.");
    info!("This is an info event.");

    // Method 2: Set subscriber as default (Works)
    /*
    let _guard = tracing::subscriber::set_default(subscriber);
    let root = span!(tracing::Level::TRACE, "app_start", work_units = 2);
    let _enter = root.enter();
    trace!("This is a trace event.");
    error!("This is an error event.");
    info!("This is an info event.");
    */

    // Method 3: Trace executed code inside of a closure (Works)
    /*
    tracing::subscriber::with_default(subscriber, || {
        // Spans will be sent to the configured OpenTelemetry exporter
        let root = span!(tracing::Level::TRACE, "app_start", work_units = 2);
        let _enter = root.enter();
        trace!("This is a trace event");
    });
    */

    let current_span = tracing::Span::current();
    println!("Current span: {:?}", current_span);
}

Similar working example

It seems like this example although different from what I'm trying to do does demonstrate that set_global_default can work, but why would the subscriber make any difference here? The subscriber clearly works in my original example when I use set_default.

use tracing::{info, Level};
use tracing_subscriber::FmtSubscriber;

fn main() {
    // a builder for `FmtSubscriber`.
    let subscriber = FmtSubscriber::builder()
        // all spans/events with a level higher than TRACE (e.g, debug, info, warn, etc.)
        // will be written to stdout.
        .with_max_level(Level::TRACE)
        // completes the builder.
        .finish();

    tracing::subscriber::set_global_default(subscriber)
        .expect("setting default subscriber failed");

    let number_of_yaks = 3;
    // this creates a new event, outside of any spans.
    info!(number_of_yaks, "preparing to shave yaks");

    let number_shaved = yak_shave::shave_all(number_of_yaks);
    info!(
        all_yaks_shaved = number_shaved == number_of_yaks,
        "yak shaving completed."
    );
}

[lalitb]

From your code, it seems you are not using the latest opentelemetry-* (v0.28.0) and tracing-opentelemetry(v0.29.0) versions. Can you please try with latest versions and update here. Also try with stdout exporter instead of otlp exporter, would be helpful to debug the issue.

[joseph-henry]

Thank you, however when I do upgrade it breaks my code in several ways. I've been banging my head on this issue for a while now and could use some more specific guidance if you don't mind.

If I were to try to write this targeting the latest 0.28.0 what needs to change specifically?

EDIT: To narrow things down a bit: Say I use the example here. How do I go about setting up a global subscriber? The way I'm doing it above seems incompatible with the example code.

Thanks.

[cijothomas]

The example in the repo is already setting global subscriber: https://github.com/open-telemetry/opentelemetry-rust/blob/main/opentelemetry-otlp/examples/basic-otlp-http/src/main.rs#L106

 // Initialize the tracing subscriber with the OpenTelemetry layer and the
    // Fmt layer.
    tracing_subscriber::registry()
        .with(otel_layer)
        .with(fmt_layer)
        .init();

[joseph-henry]

Thank you, in testing I can see that now. I'll close this issue but this seems counterintuitive for there to be a tracing::subscriber::set_global_default() in tracing crate that is and is not respected by OT for unknown reasons, and then:

 // Initialize the tracing subscriber with the OpenTelemetry layer and the
    // Fmt layer.
    tracing_subscriber::registry()
        .with(otel_layer)
        .with(fmt_layer)
        .init();

which apparently sets the default subscriber but doesn't seem to claim that anywhere in the documentation for that module.

What in the documentation should cause me to think this would accomplish what I'm trying to do?

Thanks.