Skip to content

pipeline: output: tls tcp: style: #1983

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 23, 2025
+46 −33
Merged

pipeline: output: tls tcp: style: #1983

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4d4004d
pipeline: output: tls tcp: style:
esmerel Jul 23, 2025
3a0cf5e
Apply suggestions from code review
esmerel Jul 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 45 additions & 33 deletions pipeline/outputs/tcp-and-tls.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,64 +1,70 @@
# TCP & TLS
# TCP and TLS

The **tcp** output plugin allows to send records to a remote TCP server. The payload can be formatted in different ways as required.
The _TCP_ output plugin lets you send records to a remote TCP server. The payload can be formatted in different ways as required.

## Configuration Parameters
## Configuration parameters

| Key | Description | default |
|:-------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------|
| Host | Target host where Fluent-Bit or Fluentd are listening for Forward messages. | 127.0.0.1 |
| Port | TCP Port of the target service. | 5170 |
| Format | Specify the data format to be printed. Supported formats are _msgpack_ _json_, _json\_lines_ and _json\_stream_. | msgpack |
| json\_date\_key | Specify the name of the time key in the output record. To disable the time key just set the value to `false`. | date |
| json\_date\_format | Specify the format of the date. Supported formats are _double_, _epoch_, _iso8601_ (eg: _2018-05-30T09:39:52.000681Z_) and _java_sql_timestamp_ (eg: _2018-05-30 09:39:52.000681_) | double |
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `2` |
This plugin supports the following parameters:

## TLS Configuration Parameters
| Key | Description | Default |
|:--- |:----------- |:------- |
| `Host` | Target host where Fluent Bit or Fluentd are listening for Forward messages. | `127.0.0.1` |
| `Port` | TCP Port of the target service. | `5170` |
| `Format` | Specify the data format to be printed. Supported formats: `msgpack`, `json`, `json_lines`, `json_stream`. | `msgpack` |
| `json_date_key`| Specify the name of the time key in the output record. To disable the time key, set the value to `false`. | `date` |
| `json_date_format` | Specify the format of the date. Supported formats: `double`, `epoch`, `iso8601`, `java_sql_timestamp`. | `double` |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `2` |

## TLS configuration parameters

The following parameters are available to configure a secure channel connection through TLS:

| Key | Description | Default |
|:----------------|:--------------------------------------------------------------------------------------------------------------------------------------------------|:--------|
| tls | Enable or disable TLS support | Off |
| tls.verify | Force certificate validation | On |
| tls.debug | Set TLS debug verbosity level. It accept the following values: 0 \(No debug\), 1 \(Error\), 2 \(State change\), 3 \(Informational\) and 4 Verbose | 1 |
| tls.ca\_file | Absolute path to CA certificate file | |
| tls.crt\_file | Absolute path to Certificate file. | |
| tls.key\_file | Absolute path to private Key file. | |
| tls.key\_passwd | Optional password for tls.key\_file file. | |
| Key | Description | Default |
|:--- |:----------- |:------- |
| `tls` | Enable or disable TLS support. | `Off` |
| `tls.verify` | Force certificate validation. | `On` |
| `tls.debug` | Set TLS debug verbosity level. Allowed values: `0` (No debug), `1` (Error), `2` (State change), `3` (Informational), `4` (Verbose) | `1` |
| `tls.ca_file` | Absolute path to CA certificate file. | _none_ |
| `tls.crt_file` | Absolute path to Certificate file. | _none_ |
| `tls.key_file` | Absolute path to private Key file. | _none_ |
| `tls.key_passwd` | Optional password for `tls.key_file` file. | _none_ |

### Command Line
### Command line

#### JSON format

This example specifies gathering [CPU](https://docs.fluentbit.io/manual/pipeline/inputs/cpu-metrics) usage metrics and send them in JSON lines mode to a remote endpoint using netcat service.

```shell
fluent-bit -i cpu -o tcp://127.0.0.1:5170 -p format=json_lines -v
```

We have specified to gather [CPU](https://github.com/fluent/fluent-bit-docs/tree/16f30161dc4c79d407cd9c586a0c6839d0969d97/pipeline/input/cpu.md) usage metrics and send them in JSON lines mode to a remote end-point using netcat service.

Run the following in a separate terminal, `netcat` will start listening for messages on TCP port 5170.
Once it connects to Fluent Bit ou should see the output as above in JSON format:
Run the following in a separate terminal, `netcat` will start listening for messages on TCP port `5170`. After it connects to Fluent Bit you should see the output in JSON format:

```shell
$ nc -l 5170
nc -l 5170
```

Which returns results similar to:

```text
...
{"date":1644834856.905985,"cpu_p":1.1875,"user_p":0.5625,"system_p":0.625,"cpu0.p_cpu":0.0,"cpu0.p_user":0.0,"cpu0.p_system":0.0,"cpu1.p_cpu":1.0,"cpu1.p_user":1.0,"cpu1.p_system":0.0,"cpu2.p_cpu":4.0,"cpu2.p_user":2.0,"cpu2.p_system":2.0,"cpu3.p_cpu":1.0,"cpu3.p_user":0.0,"cpu3.p_system":1.0,"cpu4.p_cpu":1.0,"cpu4.p_user":0.0,"cpu4.p_system":1.0,"cpu5.p_cpu":1.0,"cpu5.p_user":1.0,"cpu5.p_system":0.0,"cpu6.p_cpu":0.0,"cpu6.p_user":0.0,"cpu6.p_system":0.0,"cpu7.p_cpu":3.0,"cpu7.p_user":1.0,"cpu7.p_system":2.0,"cpu8.p_cpu":0.0,"cpu8.p_user":0.0,"cpu8.p_system":0.0,"cpu9.p_cpu":1.0,"cpu9.p_user":0.0,"cpu9.p_system":1.0,"cpu10.p_cpu":1.0,"cpu10.p_user":0.0,"cpu10.p_system":1.0,"cpu11.p_cpu":0.0,"cpu11.p_user":0.0,"cpu11.p_system":0.0,"cpu12.p_cpu":0.0,"cpu12.p_user":0.0,"cpu12.p_system":0.0,"cpu13.p_cpu":3.0,"cpu13.p_user":2.0,"cpu13.p_system":1.0,"cpu14.p_cpu":1.0,"cpu14.p_user":1.0,"cpu14.p_system":0.0,"cpu15.p_cpu":0.0,"cpu15.p_user":0.0,"cpu15.p_system":0.0}
...
```

#### Msgpack format
#### `msgpack` format

Repeat the JSON approach but using the `msgpack` output format.

```shell
fluent-bit -i cpu -o tcp://127.0.0.1:5170 -p format=msgpack -v
```

We could send this to stdout but as it is a serialized format you would end up with strange output.
This should really be handled by a msgpack receiver to unpack as per the details in the developer documentation [here](../../development/msgpack-format.md).
As an example we use the [Python msgpack library](https://msgpack.org/#languages) to deal with it:
You could send this to stdout but as it's a serialized format you would end up with strange output.

This should be handled by a [`msgpack`](../../development/msgpack-format.md) receiver to unpack. The following example uses the [Python `msgpack` library](https://msgpack.org/#languages) handle it:

```python
#Python3
Expand All @@ -81,11 +87,17 @@ while True:
print(unpacked)
```

Run the following commands:

```shell
$ pip install msgpack
$ python3 test.py
pip install msgpack
python3 test.py
```

Which return results like:

```text
...
(ExtType(code=0, data=b'b\n5\xc65\x05\x14\xac'), {'cpu_p': 0.1875, 'user_p': 0.125, 'system_p': 0.0625, 'cpu0.p_cpu': 0.0, 'cpu0.p_user': 0.0, 'cpu0.p_system': 0.0, 'cpu1.p_cpu': 0.0, 'cpu1.p_user': 0.0, 'cpu1.p_system': 0.0, 'cpu2.p_cpu': 1.0, 'cpu2.p_user': 0.0, 'cpu2.p_system': 1.0, 'cpu3.p_cpu': 0.0, 'cpu3.p_user': 0.0, 'cpu3.p_system': 0.0, 'cpu4.p_cpu': 0.0, 'cpu4.p_user': 0.0, 'cpu4.p_system': 0.0, 'cpu5.p_cpu': 0.0, 'cpu5.p_user': 0.0, 'cpu5.p_system': 0.0, 'cpu6.p_cpu': 0.0, 'cpu6.p_user': 0.0, 'cpu6.p_system': 0.0, 'cpu7.p_cpu': 0.0, 'cpu7.p_user': 0.0, 'cpu7.p_system': 0.0, 'cpu8.p_cpu': 0.0, 'cpu8.p_user': 0.0, 'cpu8.p_system': 0.0, 'cpu9.p_cpu': 1.0, 'cpu9.p_user': 1.0, 'cpu9.p_system': 0.0, 'cpu10.p_cpu': 0.0, 'cpu10.p_user': 0.0, 'cpu10.p_system': 0.0, 'cpu11.p_cpu': 0.0, 'cpu11.p_user': 0.0, 'cpu11.p_system': 0.0, 'cpu12.p_cpu': 0.0, 'cpu12.p_user': 0.0, 'cpu12.p_system': 0.0, 'cpu13.p_cpu': 0.0, 'cpu13.p_user': 0.0, 'cpu13.p_system': 0.0, 'cpu14.p_cpu': 0.0, 'cpu14.p_user': 0.0, 'cpu14.p_system': 0.0, 'cpu15.p_cpu': 0.0, 'cpu15.p_user': 0.0, 'cpu15.p_system': 0.0})
...
```
```
1 change: 1 addition & 0 deletions vale-styles/FluentBit/Headings.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ exceptions:
- HTTP
- I
- InfluxDB
- JSON
- Kafka
- Kafka Producer
- Kafka REST
Expand Down