1
1
# Record Modifier
2
2
3
- The _ Record Modifier Filter_ plugin allows to append fields or to exclude specific fields.
3
+ The _ Record Modifier_ [ filter] ( pipeline/filters.md ) plugin lets you append
4
+ fields to a record, or exclude specific fields.
4
5
5
- ## Configuration Parameters
6
+ ## Configuration parameters
6
7
7
- The plugin supports the following configuration parameters: _ Remove \_ key _ and _ Allowlist \_ key _ are exclusive.
8
+ The plugin supports the following configuration parameters:
8
9
9
10
| Key | Description |
10
11
| :--- | :--- |
11
- | Record | Append fields. This parameter needs key and value pair. |
12
- | Remove \_ key | If the key is matched, that field is removed. |
13
- | Allowlist \_ key | If the key is ** not ** matched, that field is removed. |
14
- | Whitelist \_ key | An alias of ` Allowlist_key ` for backwards compatibility. |
15
- | Uuid \_ key | If set, the plugin appends uuid to each record. The value assigned becomes the key in the map.|
12
+ | ` Record ` | Append fields. This parameter needs a key/ value pair. |
13
+ | ` Remove_key ` | If the key is matched, that field is removed. You can this or ` Allowlist_key ` . |
14
+ | ` Allowlist_key ` | If the key isn't matched, that field is removed. You can this or ` Remove_key ` . |
15
+ | ` Whitelist_key ` | An alias of ` Allowlist_key ` for backwards compatibility. |
16
+ | ` Uuid_key ` | If set, the plugin appends Uuid to each record. The value assigned becomes the key in the map. |
16
17
17
- ## Getting Started
18
+ ## Get started
18
19
19
- In order to start filtering records, you can run the filter from the command line or through the configuration file.
20
+ To start filtering records, run the filter from the command line or through a
21
+ configuration file.
20
22
21
- This is a sample in \_ mem record to filter.
23
+ This is a sample ` in_mem ` record to filter.
22
24
23
25
``` text
24
26
{"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724}
25
27
```
26
28
27
29
### Append fields
28
30
29
- The following configuration file is to append product name and hostname \( via environment variable\) to record.
31
+ The following configuration file appends a product name and hostname to a record
32
+ using an environment variable:
30
33
31
34
{% tabs %}
32
35
{% tab title="fluent-bit.conf" %}
33
- ``` python
36
+
37
+ ``` python copy
34
38
[INPUT ]
35
39
Name mem
36
40
Tag mem.local
@@ -45,48 +49,50 @@ The following configuration file is to append product name and hostname \(via en
45
49
Record hostname $ {HOSTNAME }
46
50
Record product Awesome_Tool
47
51
```
52
+
48
53
{% endtab %}
49
54
50
55
{% tab title="fluent-bit.yaml" %}
51
- ``` yaml
56
+
57
+ ``` yaml copy
52
58
pipeline :
53
59
inputs :
54
60
- name : mem
55
61
tag : mem.local
56
62
filters :
57
63
- name : record_modifier
58
64
match : ' *'
59
- record :
65
+ record :
60
66
- hostname ${HOSTNAME}
61
67
- product Awesome_Tool
62
68
outputs :
63
69
- name : stdout
64
70
match : ' *'
65
71
` ` `
72
+
66
73
{% endtab %}
67
74
{% endtabs %}
68
75
76
+ You can run the filter from command line:
69
77
70
- You can also run the filter from command line.
71
-
72
- ` ` ` text
73
- $ fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
78
+ ` ` ` shell copy
79
+ fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
74
80
```
75
81
76
- The output will be
82
+ The output looks something like:
77
83
78
- ``` python
84
+ ``` python copy
79
85
[0 ] mem.local: [1492436882.000000000 , {" Mem.total" => 1016024 , " Mem.used" => 716672 , " Mem.free" => 299352 , " Swap.total" => 2064380 , " Swap.used" => 32656 , " Swap.free" => 2031724 , " hostname" => " localhost.localdomain" , " product" => " Awesome_Tool" }]
80
86
```
81
87
82
- ### Remove fields with Remove\_ key
83
-
84
- The following configuration file is to remove 'Swap.\* ' fields.
88
+ ### Remove fields with ` Remove_key `
85
89
90
+ The following configuration file removes ` Swap.* ` fields:
86
91
87
92
{% tabs %}
88
93
{% tab title="fluent-bit.conf" %}
89
- ``` python
94
+
95
+ ``` python copy
90
96
[INPUT ]
91
97
Name mem
92
98
Tag mem.local
@@ -102,47 +108,51 @@ The following configuration file is to remove 'Swap.\*' fields.
102
108
Remove_key Swap.used
103
109
Remove_key Swap.free
104
110
```
111
+
105
112
{% endtab %}
106
113
107
114
{% tab title="fluent-bit.yaml" %}
108
- ``` yaml
115
+
116
+ ``` yaml copy
109
117
pipeline :
110
118
inputs :
111
119
- name : mem
112
120
tag : mem.local
113
121
filters :
114
122
- name : record_modifier
115
123
match : ' *'
116
- remove_key :
124
+ remove_key :
117
125
- Swap.total
118
126
- Swap.used
119
127
- Swap.free
120
128
outputs :
121
129
- name : stdout
122
130
match : ' *'
123
131
` ` `
132
+
124
133
{% endtab %}
125
134
{% endtabs %}
126
135
127
136
You can also run the filter from command line.
128
137
129
- ` ` ` text
130
- $ fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
138
+ ` ` ` shell copy
139
+ fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
131
140
```
132
141
133
- The output will be
142
+ The output looks something like:
134
143
135
144
``` python
136
145
[0 ] mem.local: [1492436998.000000000 , {" Mem.total" => 1016024 , " Mem.used" => 716672 , " Mem.free" => 295332 }]
137
146
```
138
147
139
- ### Remove fields with Allowlist \_ key
148
+ ### Retain fields with ` Allowlist_key `
140
149
141
- The following configuration file is to remain ' Mem.\* ' fields.
150
+ The following configuration file retains ` Mem.* ` fields.
142
151
143
152
{% tabs %}
144
153
{% tab title="fluent-bit.conf" %}
145
- ``` python
154
+
155
+ ``` python copy
146
156
[INPUT ]
147
157
Name mem
148
158
Tag mem.local
@@ -158,37 +168,39 @@ The following configuration file is to remain 'Mem.\*' fields.
158
168
Allowlist_key Mem.used
159
169
Allowlist_key Mem.free
160
170
```
171
+
161
172
{% endtab %}
162
173
163
174
{% tab title="fluent-bit.yaml" %}
164
- ``` yaml
175
+
176
+ ``` yaml copy
165
177
pipeline :
166
178
inputs :
167
179
- name : mem
168
180
tag : mem.local
169
181
filters :
170
182
- name : record_modifier
171
183
match : ' *'
172
- Allowlist_key :
184
+ Allowlist_key :
173
185
- Mem.total
174
186
- Mem.used
175
187
- Mem.free
176
188
outputs :
177
189
- name : stdout
178
190
match : ' *'
179
191
` ` `
192
+
180
193
{% endtab %}
181
194
{% endtabs %}
182
195
183
- You can also run the filter from command line.
196
+ You can also run the filter from command line:
184
197
185
- ` ` ` text
186
- $ fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
198
+ ` ` ` shell copy
199
+ fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
187
200
```
188
201
189
- The output will be
202
+ The output looks something like:
190
203
191
204
``` python
192
205
[0 ] mem.local: [1492436998.000000000 , {" Mem.total" => 1016024 , " Mem.used" => 716672 , " Mem.free" => 295332 }]
193
206
```
194
-
0 commit comments