add document defining an OpenTelemetry Collector #4313
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| <!--- Hugo front matter used to generate the website version of this page: | ||
| path_base_for_github_subdir: | ||
| from: tmp/otel/specification/collector/_index.md | ||
| to: collector/README.md | ||
| ---> | ||
|
|
||
| # OpenTelemetry Collector | ||
codeboten marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| The goal of this document is for users to be able to easily switch between | ||
| OpenTelemetry Collector Distros while also ensuring that components produced by | ||
| the OpenTelemetry Collector SIG are able to work with any vendor who claims | ||
| support for an OpenTelemetry Collector. | ||
|
Comment on lines
+11
to
+14
There was a problem hiding this comment. I'm not sure I understand this goal. If a vendor produces a collector distribution that has a subset of available components because those are the components relevant to their service offerings and that they're willing to support, where do any other components (whether hosted in an OTel repo or not) fit into that picture? Do we mean that a distribution must offer end users the ability to modify its source and create their own build? We should be explicit about that if that is the case. Given that the licensing of the collector's source code does not require that distribution of derivative works happen in source form I'm not sure that we have much ability here to enforce such a requirement. We can certainly try to use the "OpenTelemetry" mark as a cudgel, but I'm not sure it'll be as effective as may be desirable since the terms "collector" and "distribution" are very broad. It could perhaps be argued that "OpenTelemetry Collector" is a protectable mark and maybe even that "Collector" has acquired secondary meaning in this limited scope, but protecting such a mark against genericization is going to be a Sisyphean task. There was a problem hiding this comment. I see this definition as separation from the term Something is not an OpenTelemetry Collector if it cannot support OpenTelemetry Collector Components. Maybe the word There was a problem hiding this comment.
We potentially have leverage over:
I think we have enough leverage here to make this worth it There was a problem hiding this comment. I'm not sure that having a registered "OpenTelemetry Collector" mark is sufficient here as nominative fair use would allow anyone preparing a distribution (in the colloquial sense, not Obviously the project can control what it puts on its website and what marketing collateral is used in conjunction with events operated by LF/CNCF, but that doesn't seem like effective leverage over an actor who has no need or interest in such things. |
||
|
|
||
| - An OpenTelemetry Collector _MUST_ accept an [OpenTelemetry Collector configuration | ||
| file](#opentelemetry-collector-configuration-file). | ||
| - An OpenTelemetry Collector _MUST_ be able to include additional compatible | ||
| [Collector components](#opentelemetry-collector-components) that | ||
| the user wishes to include. | ||
|
|
||
| ## OpenTelemetry Collector configuration file | ||
jpkrohling marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| An OpenTelemetry Collector configuration file is defined as YAML and _MUST_ support | ||
| the following [minimum structure](https://pkg.go.dev/go.opentelemetry.io/collector/otelcol#Config): | ||
|
|
||
| ```yaml | ||
| receivers: | ||
| processors: | ||
| exporters: | ||
| connectors: | ||
| extensions: | ||
| service: | ||
codeboten marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ``` | ||
codeboten marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## OpenTelemetry Collector components | ||
|
|
||
| For a library to be considered an OpenTelemetry Collector component, it _MUST_ | ||
| implement a [Component interface](https://pkg.go.dev/go.opentelemetry.io/collector/component#Component) | ||
| defined by the OpenTelemetry Collector SIG. | ||
|
|
||
| Components require a unique identfier as a `type` string to be included in an OpenTelemetry | ||
| Collector. It is possible that multiple components use the same identifier, in which | ||
| case the two components cannot be used simultaneously in a single OpenTelemetry Collector. In | ||
| order to resolve this, the clashing components must use a different identifier. | ||
codeboten marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### Compatibility requirements | ||
|
|
||
| A component is defined as compatible with an OpenTelemetry Collector when its dependencies are | ||
| source- and version-compatible with the Component interfaces of that Collector. | ||
|
|
||
| For example, a Collector derived from version tag v0.100.0 of the [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector) _MUST_ support all components that | ||
| are version-compatible with the Golang Component API defined in the `github.com/open-telemetry/opentelemetry-collector/component` module found in that repository for that version tag. | ||
|
|
||
codeboten marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ## OpenTelemetry Collector Distribution | ||
codeboten marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| An OpenTelemetry Collector Distribution (Distro) is a compiled instance | ||
| of an OpenTelemetry Collector with a specific set of components and features. A | ||
| Distribution _SHOULD_ provide users with tools and/or documentation for adding | ||
| their own components to the Distribution. | ||
reyang marked this conversation as resolved.
Show resolved
Hide resolved
codeboten marked this conversation as resolved.
Show resolved
Hide resolved
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OpenTelemetry Collector is never defined. Is it a source code artifact? A binary?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's one of the things I was trying to get at here. Since there's no binary plugin mechanism it seems that the source would need to be available for it to be extended in the manner contemplated, but that's not clear or explicit in the current state.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the lack of binary plugin mechanism something that the OpenTelemetry Collector SIG wants to solve? Are there technical blockers?
Binary and dynamic loading plugin seem to be an established pattern. For example:
https://github.com/fluent/fluent-bit-go
https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/yaml/plugins-section
While Fluent Bit comes with a variety of built-in plugins, it also supports loading external plugins at runtime. This feature is especially useful for loading Go or Wasm plugins that are built as shared object files (.so).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Fluent Bit example is not necessarily apposite as it involves a C application dynamically loading shared libraries built with the Go toolchain using CGO (which is generally prohibited in the Collector codebase).
Go does have a native plugin mechanism, though it comes with many caveats and is widely regarded as a bad idea that can't be dropped due to compatibility guarantees. Its documentation sums up its litany of restrictions in this way, which sounds a lot like a suggestion to use something like
ocb: