Instrumentation scope is application-centric, exluding system scopes?

[thediveo]

URL

https://opentelemetry.io/docs/concepts/instrumentation-scope/

Description

To cite the current state of documentation https://opentelemetry.io/docs/concepts/instrumentation-scope/:

The Instrumentation scope represents a logical unit within the application code with which the emitted telemetry can be associated.

Developers can decide what denotes a reasonable instrumentation scope. For example, they can select a module, a package, or a class as the instrumentation scope. In the case of a library or framework, a common approach is to use an identifier as scope that is unique to the library or framework, such as a fully qualified name and version of the library or framework.

This is all fine from a perspective of applications (including libraries/frameworks). Unfortunately, it isn't inclusive yet of OTel applications tackling system ("host") scope and lacks advice of what an instrumentation scope preferably might be in host/system scope. For instance, does the scope identify the piece of software that emits, say, metrics? Or does it scope the piece of system information instead?

The aforementioned documentation also could be in slight conflict with https://opentelemetry.io/docs/specs/otel/glossary/#instrumentation-scope:

It is typically the developer’s choice to decide what denotes a reasonable instrumentation scope. The most common approach is to use the instrumentation library as the scope, however other scopes are also common, e.g. a module, a package, or a class can be chosen as the instrumentation scope.

Preferably, the concepts doc should be in sync with the glossary; to me, it looks as if the concepts took should adopt the glossary approach: under the glossary approach, the instrument scope for a system instrument would be the implementation package, instead of a system scope. Thus, under this regime, different implementations of system instruments might emit the same instrument metrics, but in separate scopes. Is this what OTel envisioned?

[svrnm]

Hey @thediveo,

I am not exactly sure if I understand your concern/feedback here: the instrumentation scope is a scope within an application. It is here to denote the boundaries between libraries, modules, classes, etc.

For host/system scopes OpenTelemetry uses the concept of resources, which you will find here:

https://opentelemetry.io/docs/concepts/resources/

[thediveo]

When I need to expose system/host metrics using the OTel SDK where the metrics are coming from outside the SDK I need to implement a metric.Producer which produces []metricdata.ScopeMetrics.

These scope(d) metrics are

// ScopeMetrics is a collection of Metrics Produces by a Meter.
type ScopeMetrics struct {
	// Scope is the Scope that the Meter was created with.
	Scope instrumentation.Scope
	// Metrics are a list of aggregations created by the Meter.
	Metrics []Metrics
}

and the instrumentation scope is:

// Scope represents the instrumentation scope.
type Scope struct {
	// Name is the name of the instrumentation scope. This should be the
	// Go package name of that scope.
	Name string
	// Version is the version of the instrumentation scope.
	Version string
	// SchemaURL of the telemetry emitted by the scope.
	SchemaURL string
	// Attributes of the telemetry emitted by the scope.
	Attributes attribute.Set
}

So instrumentation metrics producers need to provide an instrumentation scope in any case, unless there is something that I totally missed in the past weeks looking through the OTel docs over and over again, or simply missing.

Is it allowed to zero-value the instrumentation.Scope? If so, I would like to ask you to clearly document this, both on the levels of SDK and of concepts.

[thediveo]

Any official guidance here on whether instrumentation.Scope should be a zero value?

[svrnm]

@thediveo I moved this from the website repo to the spec repo since this way there is a higher chance to provide you with an answer. If we get to the point that the wording either in the spec glossary or on the website needs a rewrite we can move it back there.

A few thoughts from my end after thinking about this for a little bit, there are existing implementations of what you are trying to achive (if I understand you correctly here), e.g. resource detectors in different languages that also collect metrics, an example in Go:

https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/instrumentation/host/host.go#L25

or the JMX Metrics Insights module in Java:

https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/jmx-metrics/library/src/main/java/io/opentelemetry/instrumentation/jmx/engine/JmxMetricInsight.java#L22

Both provide (or can provide) host/system metrics. The instrumentation scope donates the class/package/... name that deliveres that instrumentation. This is within the definition ("developer's choice" and "use the instrumentation library as scope"), and is also a useful setting: if a metric reported is not as expected or if someone wants to add/remove/update metrics the scope is indicative where to look in your source code.

So in your own use case, you should look up from the code where you collect the measurements and pick the class/package/... name closest to it.

I hope that helps

[thediveo]

Initially, you wrote:

[...] the instrumentation scope is a scope within an application. It is here to denote the boundaries between libraries, modules, classes, etc.

For host/system scopes OpenTelemetry uses the concept of resources [...]

The Go "host instrumentation" example you referenced defines an "instrumentation scope name". Based on your initial definition, "[an] instrumentation scope is a scope within an application." So how am I to understand your example now? Is it a contradiction to what you've said? Or is it acceptable practice (or rather, a precedent) and the documentation needs to be changed accordingly?

Seemingly in the same contradiction, the Java instumentation scope example also seems to contradict your original intention.

Did I miss setting the resource information?

[svrnm]

I think I understand your initial question now, and I apologize for the confusion I might have created. Let me go back to the top and take the specific question you had

For instance, does the scope identify the piece of software that emits, say, metrics? Or does it scope the piece of system information instead?

• The scope that identifies the piece of software that emits metrics is the instrumentation scope, e.g. go.opentelemetry.io/contrib/instrumentation/host
• The scope of the piece of system information is a resource, e.g. a host or process

You might find the following useful: Semantic Conventions for Common Hardware Metrics

Note: Based on the discussion I see that the documentation is misleading here and it also makes the assumption about the instrumentation scope that it also can be used as a scope within an application, which is not universally true. For example, if I use an instrumentation library for the javascript library "express" the instrumentation scope will be "@opentelemetry/instrumentation-express" and not some express internal value. This needs to be clarified.

[thediveo]

• The scope that identifies the piece of software that emits metrics is the instrumentation scope, e.g. go.opentelemetry.io/contrib/instrumentation/host
• The scope of the piece of system information is a resource, e.g. a host or process

That very clearly nails it and is easily actionable from devs perspective. Thank you very much!