add document defining an OpenTelemetry Collector

[codeboten]

Changes

Adds a definition of an OpenTelemetry Collector

• Related issues Propose a definition of Collector to be included in the specification #4309

[cyrille-leclerc]

Should this specification define what an OpenTelemetry Collector plugin is? Is it any component of type receiver, processor, exporter, extension, or config map provider?

[yurishkuro]

Specification needs to define all the terms used in this definition, otherwise it does not remove ambiguity.

[codeboten]

Renamed plugin to component and added a section to define OpenTelemetry Collector component.

[mx-psi]

The Collector also accepts confmap.Providers and confmap.Converters, which do not accept this interface. Do we consider those out of scope?

[Aneurysm9]

I think they do need to be considered in scope. Interoperability of those components is important.

[codeboten]

The Collector also accepts confmap.Providers and confmap.Converters, which do not accept this interface. Do we consider those out of scope?

I wonder if including them would allow us to avoid having to include a definition for a config file, wdyt?

[mx-psi]

Providers handle configuration abstractly so they help remove the need for how configuration should be represented, but they don't solve the schema part (which I don't feel like we need to solve tbh)

[Aneurysm9]

I'm not sure I understand this goal. If a vendor produces a collector distribution that has a subset of available components because those are the components relevant to their service offerings and that they're willing to support, where do any other components (whether hosted in an OTel repo or not) fit into that picture? Do we mean that a distribution must offer end users the ability to modify its source and create their own build? We should be explicit about that if that is the case.

Given that the licensing of the collector's source code does not require that distribution of derivative works happen in source form I'm not sure that we have much ability here to enforce such a requirement. We can certainly try to use the "OpenTelemetry" mark as a cudgel, but I'm not sure it'll be as effective as may be desirable since the terms "collector" and "distribution" are very broad. It could perhaps be argued that "OpenTelemetry Collector" is a protectable mark and maybe even that "Collector" has acquired secondary meaning in this limited scope, but protecting such a mark against genericization is going to be a Sisyphean task.

[TylerHelmuth]

I see this definition as separation from the term Distribution defined below. A Distribution is a specific compiled OpenTelemetry Collector with a specific set of OpenTelemetry Collector Components that the maintainer (the user in this case) decided to add. It is a OpenTelemetry Collector bc the maintainer was able to bring their chosen OpenTelemetry Collector components to it.

Something is not an OpenTelemetry Collector if it cannot support OpenTelemetry Collector Components. Maybe the word additional below is unnecessary and could be removed?

[mx-psi]

Given that the licensing of the collector's source code does not require that distribution of derivative works happen in source form I'm not sure that we have much ability here to enforce such a requirement

We potentially have leverage over:

• Trademark usage if "OpenTelemetry Collector" becomes a trademark
• What we list on our registry and website and what we promote
• What wording can be used in 'official' OTel events

I think we have enough leverage here to make this worth it

[Aneurysm9]

I'm not sure that having a registered "OpenTelemetry Collector" mark is sufficient here as nominative fair use would allow anyone preparing a distribution (in the colloquial sense, not Distribution however we seek to define it) to identify it as such. The Linux Foundation trademark usage guidelines also call out specifically this sort of usage as acceptable for indicating products are related to or based on the project that produces the product bearing their marks.

Obviously the project can control what it puts on its website and what marketing collateral is used in conjunction with events operated by LF/CNCF, but that doesn't seem like effective leverage over an actor who has no need or interest in such things.

[Aneurysm9]

I think they do need to be considered in scope. Interoperability of those components is important.

[yurishkuro]

OpenTelemetry Collector is never defined. Is it a source code artifact? A binary?

[Aneurysm9]

That's one of the things I was trying to get at here. Since there's no binary plugin mechanism it seems that the source would need to be available for it to be extended in the manner contemplated, but that's not clear or explicit in the current state.

[reyang]

Since there's no binary plugin mechanism it seems that the source would need to be available for it to be extended in the manner contemplated, but that's not clear or explicit in the current state.

Is the lack of binary plugin mechanism something that the OpenTelemetry Collector SIG wants to solve? Are there technical blockers?

Binary and dynamic loading plugin seem to be an established pattern. For example:

• https://github.com/fluent/fluent-bit-go

• https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/yaml/plugins-section

  While Fluent Bit comes with a variety of built-in plugins, it also supports loading external plugins at runtime. This feature is especially useful for loading Go or Wasm plugins that are built as shared object files (.so).

[Aneurysm9]

The Fluent Bit example is not necessarily apposite as it involves a C application dynamically loading shared libraries built with the Go toolchain using CGO (which is generally prohibited in the Collector codebase).

Go does have a native plugin mechanism, though it comes with many caveats and is widely regarded as a bad idea that can't be dropped due to compatibility guarantees. Its documentation sums up its litany of restrictions in this way, which sounds a lot like a suggestion to use something like ocb:

Together, these restrictions mean that, in practice, the application and its plugins must all be built together by a single person or component of a system. In that case, it may be simpler for that person or component to generate Go source files that blank-import the desired set of plugins and then compile a static executable in the usual way.

[mx-psi]

Given that the licensing of the collector's source code does not require that distribution of derivative works happen in source form I'm not sure that we have much ability here to enforce such a requirement

We potentially have leverage over:

• Trademark usage if "OpenTelemetry Collector" becomes a trademark
• What we list on our registry and website and what we promote
• What wording can be used in 'official' OTel events

I think we have enough leverage here to make this worth it

[jaronoff97]

two minor suggestions, otherwise this looks great! Thank you Alex!

[jaronoff97]

Thank you Alex 🙇

[tedsuo]

Thanks Alex, this definition really captures the heart of it. A+

[carlosalberto]

Overall LGTM, but I suggest there's a (minor) clarification around this comment:

Do we mean that a distribution must offer end users the ability to modify its source and create their own build? We should be explicit about that if that is the case.

As the bottom part mentions that, for vendor distros, this is a SHOULD instead.

[codeboten]

Overall LGTM, but I suggest there's a (minor) clarification around this comment:

Do we mean that a distribution must offer end users the ability to modify its source and create their own build? We should be explicit about that if that is the case.

As the bottom part mentions that, for vendor distros, this is a SHOULD instead.

Updated should to must, PTAL

[Aneurysm9]

This seems problematic to me as a MUST. This is, in effect, a requirement that distributions be made available in source form with a license that permits modification (and presumably distribution, though that's not clear). The license under which the Collector is released does not require this and I'm suspicious of the ability to use trademark protections to prevent someone from using the phrasing "Foo Distribution for OpenTelemetry Collector" given that's literally the first "Correct" example in the Linux Foundation trademark guidelines and is a textbook case of nominative fair use.

I think if we want an identifier for compatible distributions that can be effectively controlled we will need a distinctive mark for a compatibility certification that can be granted to distributions that satisfy its requirements, similar to what @tedsuo seems to be describing here.

Beyond those concerns, this requirement also seems excessively vague. What qualifies as "tools and/or documentation"? Is a link to https://go.dev/dl/ sufficient? This probably requires a definition similar to "Corresponding Source" from AGPL-3, which again reinforces the limitations that come from not having this be part of the license under which the collector source code is made available.

[jpkrohling]

The spirit here is to allow users to reuse their components when they move from one distribution to another: the engineering investments made should not be lost. If the distribution is open source and there's clear documentation how to add a new component to it, that's good enough for me. If the distribution is not open source but allows me to enter the Go module name on a web interface somewhere and get a binary out, that's also fine.

I'd see that binary as "tainted" (to use the kernel terminology) and the final binary might not be officially supported (with SLAs) by a service provider, but as an end-user, I'm not locked in.

[Aneurysm9]

I'm not sure that having a registered "OpenTelemetry Collector" mark is sufficient here as nominative fair use would allow anyone preparing a distribution (in the colloquial sense, not Distribution however we seek to define it) to identify it as such. The Linux Foundation trademark usage guidelines also call out specifically this sort of usage as acceptable for indicating products are related to or based on the project that produces the product bearing their marks.

Obviously the project can control what it puts on its website and what marketing collateral is used in conjunction with events operated by LF/CNCF, but that doesn't seem like effective leverage over an actor who has no need or interest in such things.

[Aneurysm9]

The Fluent Bit example is not necessarily apposite as it involves a C application dynamically loading shared libraries built with the Go toolchain using CGO (which is generally prohibited in the Collector codebase).

Go does have a native plugin mechanism, though it comes with many caveats and is widely regarded as a bad idea that can't be dropped due to compatibility guarantees. Its documentation sums up its litany of restrictions in this way, which sounds a lot like a suggestion to use something like ocb:

Together, these restrictions mean that, in practice, the application and its plugins must all be built together by a single person or component of a system. In that case, it may be simpler for that person or component to generate Go source files that blank-import the desired set of plugins and then compile a static executable in the usual way.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.