You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: administration/configuring-fluent-bit/yaml/pipeline-section.md
+13-13Lines changed: 13 additions & 13 deletions
Original file line number
Diff line number
Diff line change
@@ -1,17 +1,17 @@
1
-
# Pipeline Section
1
+
# Pipeline section
2
2
3
3
The `pipeline` section defines the flow of how data is collected, processed, and sent to its final destination. It encompasses the following core concepts:
4
4
5
5
| Name | Description |
6
-
|---|---|
6
+
|---- |----------- |
7
7
|`inputs`| Specifies the name of the plugin responsible for collecting or receiving data. This component serves as the data source in the pipeline. Examples of input plugins include `tail`, `http`, and `random`. |
8
-
|`processors`|**Unique to YAML configuration**, processors are specialized plugins that handle data processing directly attached to input plugins. Unlike filters, processors are not dependent on tag or matching rules. Instead, they work closely with the input to modify or enrich the data before it reaches the filtering or output stages. Processors are defined within an input plugin section. |
8
+
|`processors`|**Unique to YAML configuration**, processors are specialized plugins that handle data processing directly attached to input plugins. Unlike filters, processors aren't dependent on tag or matching rules. Instead, they work closely with the input to modify or enrich the data before it reaches the filtering or output stages. Processors are defined within an input plugin section. |
9
9
|`filters`| Filters are used to transform, enrich, or discard events based on specific criteria. They allow matching tags using strings or regular expressions, providing a more flexible way to manipulate data. Filters run as part of the main event loop and can be applied across multiple inputs and filters. Examples of filters include `modify`, `grep`, and `nest`. |
10
10
|`outputs`| Defines the destination for processed data. Outputs specify where the data will be sent, such as to a remote server, a file, or another service. Each output plugin is configured with matching rules to determine which events are sent to that destination. Common output plugins include `stdout`, `elasticsearch`, and `kafka`. |
11
11
12
-
## Example Configuration
12
+
## Example configuration
13
13
14
-
Here's a simple example of a pipeline configuration:
14
+
Here's an example of a pipeline configuration:
15
15
16
16
```yaml
17
17
pipeline:
@@ -33,13 +33,13 @@ pipeline:
33
33
match: '*'
34
34
```
35
35
36
-
## Pipeline Processors
36
+
## Pipeline processors
37
37
38
-
Processors operate on specific signals such as logs, metrics, and traces. They are attached to an input plugin and must specify the signal type they will process.
38
+
Processors operate on specific signals such as logs, metrics, and traces. They're attached to an input plugin and must specify the signal type they will process.
39
39
40
40
### Example of a Processor
41
41
42
-
In the example below, the content_modifier processor inserts or updates (upserts) the key my_new_key with the value 123 for all log records generated by the tail plugin. This processor is only applied to log signals:
42
+
In the following example, the `content_modifier` processor inserts or updates (upserts) the key `my_new_key` with the value `123` for all log records generated by the tail plugin. This processor is only applied to log signals:
43
43
44
44
```yaml
45
45
parsers:
@@ -110,9 +110,9 @@ pipeline:
110
110
end
111
111
```
112
112
113
-
You might noticed that processors not only can be attached to input, but also to an output.
113
+
Processors can be attached to inputs and outputs.
114
114
115
-
### How Are Processors Different from Filters?
115
+
### How Processors are different from Filters
116
116
117
117
While processors and filters are similar in that they can transform, enrich, or drop data from the pipeline, there is a significant difference in how they operate:
118
118
@@ -122,11 +122,11 @@ While processors and filters are similar in that they can transform, enrich, or
122
122
123
123
## Running Filters as Processors
124
124
125
-
You can configure existing [Filters](https://docs.fluentbit.io/manual/pipeline/filters) to run as processors. There are no specific changes needed; you simply use the filter name as if it were a native processor.
125
+
You can configure existing [Filters](https://docs.fluentbit.io/manual/pipeline/filters) to run as processors. There are no specific changes needed; you use the filter name as if it were a native processor.
126
126
127
-
### Example of a Filter Running as a Processor
127
+
### Example of a Filter running as a Processor
128
128
129
-
In the example below, the grep filter is used as a processor to filter log events based on a pattern:
129
+
In the following example, the `grep` filter is used as a processor to filter log events based on a pattern:
0 commit comments