Skip to content

pipeline: outputs: observe: general cleanup #1979

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 22, 2025
+20 −22
Merged

pipeline: outputs: observe: general cleanup #1979

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a156ea0
pipeline: outputs: observe: general cleanup
alexakreizinger Jul 22, 2025
80fca82
Update pipeline/outputs/observe.md
alexakreizinger Jul 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 20 additions & 22 deletions pipeline/outputs/observe.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,31 +1,28 @@
# Observe

Observe employs the **http** output plugin, allowing you to flush your records [into Observe](https://docs.observeinc.com/en/latest/content/data-ingestion/forwarders/fluentbit.html).
Use the [HTTP output plugin](../pipeline/outputs/http) to flush your records [into Observe](https://docs.observeinc.com/en/latest/content/data-ingestion/forwarders/fluentbit.html). It issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format.

For now the functionality is pretty basic, and it issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format.
## Configuration parameters

The following are the specific HTTP parameters to employ:
The following HTTP configuration parameters are relevant to Observe:

## Configuration Parameters
| Key | Description | Default |
| --- | ----------- | ------- |
| `host` | IP address or hostname of the Observe data collection endpoint. Replace `$(OBSERVE_CUSTOMER)` with your [Customer ID](https://docs.observeinc.com/en/latest/content/common-topics/HelpfulHints.html?highlight=customer%20id#customer-id). | `OBSERVE_CUSTOMER.collect.observeinc.com` |
| `port` | TCP port to use when sending data to Observe. | `443` |
| `tls` | Specifies whether to use TLS. | `on` |
| `uri` | Specifies the HTTP URI for Observe. | `/v1/http/fluentbit` |
| `format` | The data format to be used in the HTTP request body. | `msgpack` |
| `header` | The specific header that provides the Observe token needed to authorize sending data [into a data stream](https://docs.observeinc.com/en/latest/content/data-ingestion/datastreams.html?highlight=ingest%20token#create-a-datastream). | 'Authorization Bearer ${OBSERVE_TOKEN}' |
| `header` | The specific header that instructs Observe how to decode incoming payloads. | `X-Observe-Decoder fluent` |
| `compress` | Sets the payload compression mechanism. Possible values: `gzip`, `false`. | `gzip` |
| `tls.ca_file` | For Windows only: the path to the root cert. | _none_ |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |

| Key | Description | default |
|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|
| host | IP address or hostname of Observe's data collection endpoint. $(OBSERVE_CUSTOMER) is your [Customer ID](https://docs.observeinc.com/en/latest/content/common-topics/HelpfulHints.html?highlight=customer%20id#customer-id) | OBSERVE_CUSTOMER.collect.observeinc.com |
| port | TCP port of to employ when sending to Observe | 443 |
| tls | Specify to use tls | on |
| uri | Specify the HTTP URI for the Observe's data ingest | /v1/http/fluentbit |
| format | The data format to be used in the HTTP request body | msgpack |
| header | The specific header that provides the Observe token needed to authorize sending data [into a data stream](https://docs.observeinc.com/en/latest/content/data-ingestion/datastreams.html?highlight=ingest%20token#create-a-datastream). | Authorization Bearer ${OBSERVE_TOKEN} |
| header | The specific header to instructs Observe how to decode incoming payloads | X-Observe-Decoder fluent |
| compress | Set payload compression mechanism. Option available is 'gzip' | gzip |
| tls.ca_file | **For use with Windows**: provide path to root cert | |
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | 0 |

### Configuration File
### Configuration file

In your main configuration file append the following:


{% tabs %}
{% tab title="fluent-bit.yaml" %}

Expand All @@ -37,16 +34,17 @@ pipeline:
match: '*'
host: my-observe-customer-id.collect.observeinc.com
port: 443
tls: on
tls: on
uri: /v1/http/fluentbit
format: msgpack
header:
header:
- 'Authorization Bearer ${OBSERVE_TOKEN}'
- 'X-Observe-Decoder fluent'
compress: gzip
# For Windows: provide path to root cert
#tls.ca_file C:\fluent-bit\isrgrootx1.pem
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

Expand All @@ -70,4 +68,4 @@ pipeline:
```

{% endtab %}
{% endtabs %}
{% endtabs %}