You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pipeline/filters/grep.md
+18-32Lines changed: 18 additions & 32 deletions
Original file line number
Diff line number
Diff line change
@@ -1,33 +1,28 @@
1
1
---
2
-
description: Select or exclude records using patterns
2
+
description: Select or exclude records using regular expressions.
3
3
---
4
4
5
5
# Grep
6
6
7
-
The _Grep Filter_ plugin lets you match or exclude specific records based on
8
-
regular expression patterns for values or nested values.
7
+
The _Grep_ filter plugin lets you match or exclude specific records based on regular expression patterns for values or nested values.
9
8
10
9
## Configuration parameters
11
10
12
11
The plugin supports the following configuration parameters:
13
12
14
13
| Key | Value Format | Description |
15
14
| ------------ | ------------ | ----------- |
16
-
|`Regex`| KEY REGEX | Keep records where the content of KEY matches the regular expression. |
17
-
|`Exclude`| KEY REGEX | Exclude records where the content of KEY matches the regular expression. |
18
-
|`Logical_Op`| Operation | Specify a logical operator: `AND`, `OR` or `legacy` (default). In `legacy` mode the behaviour is either `AND` or `OR` depending on whether the `grep` is including (uses AND) or excluding (uses OR). Available from 2.1 or higher. |
15
+
|`Regex`|`KEY REGEX`| Keep records where the content of `KEY` matches the regular expression. |
16
+
|`Exclude`|`KEY REGEX`| Exclude records where the content of `KEY` matches the regular expression. |
17
+
|`Logical_Op`|`Operation`| Specify a logical operator: `AND`, `OR` or `legacy` (default). In `legacy` mode the behaviour is either `AND` or `OR` depending on whether the `grep` is including (uses `AND`) or excluding (uses OR). Available from 2.1 or higher. |
19
18
20
19
### Record Accessor Enabled
21
20
22
-
Enable the [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md)
23
-
feature to specify the KEY. Use the record accessor to match values against nested
24
-
values.
21
+
Enable the [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) feature to specify the `KEY`. Use the record accessor to match values against nested values.
25
22
26
23
## Filter records
27
24
28
-
To start filtering records, run the filter from the command line or through the
29
-
configuration file. The following example assumes that you have a file named
30
-
`lines.txt` with the following content:
25
+
To start filtering records, run the filter from the command line or through the configuration file. The following example assumes that you have a file named `lines.txt` with the following content:
31
26
32
27
```text
33
28
{"log": "aaa"}
@@ -42,16 +37,12 @@ configuration file. The following example assumes that you have a file named
42
37
43
38
### Command line
44
39
45
-
When using the command line, pay close attention to quote the regular expressions.
46
-
Using a configuration file might be easier.
40
+
When using the command line, pay close attention to quote the regular expressions. Using a configuration file might be easier.
47
41
48
-
The following command loads the [tail](../../pipeline/inputs/tail) plugin and
49
-
reads the content of `lines.txt`. Then the `grep` filter applies a regular
50
-
expression rule over the `log` field created by the `tail` plugin and only passes
51
-
records with a field value starting with `aa`:
42
+
The following command loads the [tail](../../pipeline/inputs/tail) plugin and reads the content of `lines.txt`. Then the `grep` filter applies a regular expression rule over the `log` field created by the `tail` plugin and only passes records with a field value starting with `aa`:
To match or exclude records based on nested values, you can use [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) format as the `KEY` name.
115
102
116
103
Consider the following record example:
117
104
@@ -132,8 +119,7 @@ Consider the following record example:
132
119
}
133
120
```
134
121
135
-
For example, to exclude records that match the nested field `kubernetes.labels.app`,
136
-
use the following rule:
122
+
For example, to exclude records that match the nested field `kubernetes.labels.app`, use the following rule:
137
123
138
124
{% tabs %}
139
125
{% tab title="fluent-bit.conf" %}
@@ -162,15 +148,15 @@ use the following rule:
162
148
163
149
You might want to drop records that are missing certain keys.
164
150
165
-
One way to do this is to `exclude` with a regex that matches anything. A missing
151
+
One way to do this is to `exclude` with a regular expression that matches anything. A missing
166
152
key fails this check.
167
153
168
-
The followinfg example checks for a specific valid value for the key:
154
+
The following example checks for a specific valid value for the key:
169
155
170
156
{% tabs %}
171
157
{% tab title="fluent-bit.conf" %}
172
158
173
-
```text
159
+
```python
174
160
# Use Grep to verify the contents of the iot_timestamp value.
175
161
# If the iot_timestamp key does not exist, this will fail
176
162
# and exclude the row.
@@ -201,7 +187,7 @@ or is missing or empty, then it will be excluded.
201
187
### Multiple conditions
202
188
203
189
If you want to set multiple `Regex` or `Exclude`, use the `Logical_Op` property
204
-
to use a logical conjuction or disjunction.
190
+
to use a logical conjunction or disjunction.
205
191
206
192
If `Logical_Op` is set, setting both `Regex` and `Exclude` results in an error.
0 commit comments