@@ -12,6 +12,9 @@ data. Any option that supports this syntax will be clearly documented as such in
1212Let's partition data on AWS S3 by "application_id" and "date". We can accomplish this with the ` key_prefix ` option in
1313the ` aws_s3 ` sink:
1414
15+ {{< tabs default="YAML" >}}
16+ {{< tab title="YAML" >}}
17+
1518``` yaml
1619sinks :
1720 backup :
@@ -20,6 +23,19 @@ sinks:
2023 key_prefix : " application_id={{ application_id }}/date=%F/"
2124` ` `
2225
26+ {{< /tab >}}
27+ {{< tab title="TOML" >}}
28+
29+ ` ` ` toml
30+ [sinks.backup]
31+ type = "aws_s3"
32+ bucket = "all_application_logs"
33+ key_prefix = "application_id={{ application_id }}/date=%F/"
34+ ```
35+
36+ {{< /tab >}}
37+ {{< /tabs >}}
38+
2339Notice that Vector allows direct field references as well as "strftime" specifiers. If we were to run the following log
2440event through Vector:
2541
@@ -46,18 +62,44 @@ enables dynamic partitioning, something fundamental to storing log data in files
4662
4763Individual [ log event] [ log ] fields can be accessed using ` {{ ... }} ` to wrap a VRL [ path expression] [ path_expression ] :
4864
65+ {{< tabs default="YAML" >}}
66+ {{< tab title="YAML" >}}
67+
4968``` yaml
5069option : " {{ .parent.child }}"
5170` ` `
5271
72+ {{< /tab >}}
73+ {{< tab title="TOML" >}}
74+
75+ ` ` ` toml
76+ option = "{{ .parent.child }}"
77+ ```
78+
79+ {{< /tab >}}
80+ {{< /tabs >}}
81+
5382### Strftime specifiers
5483
5584In addition to directly accessing fields, Vector offers a shortcut for injecting [ strftime specifiers] [ strftime ] :
5685
86+ {{< tabs default="YAML" >}}
87+ {{< tab title="YAML" >}}
88+
5789``` yaml
5890option : " year=%Y/month=%m/day=%d/"
5991` ` `
6092
93+ {{< /tab >}}
94+ {{< tab title="TOML" >}}
95+
96+ ` ` ` toml
97+ option = "year=%Y/month=%m/day=%d/"
98+ ```
99+
100+ {{< /tab >}}
101+ {{< /tabs >}}
102+
61103{{< info >}}
62104The value is derived from the [ ` timestamp ` field] ( /docs/architecture/data-model/log/#timestamps )
63105and the name of this field can be changed via the [ global ` timestamp_key ` option] ( /docs/reference/configuration/schema/#log_schema.timestamp_key ) .
@@ -68,16 +110,42 @@ and the name of this field can be changed via the [global `timestamp_key` option
68110You can escape this syntax by prefixing the character with a ` \ ` . For example, you can escape the event field syntax
69111like this:
70112
113+ {{< tabs default="YAML" >}}
114+ {{< tab title="YAML" >}}
115+
71116``` yaml
72117option : ' \{{ field_name }}'
73118` ` `
74119
120+ {{< /tab >}}
121+ {{< tab title="TOML" >}}
122+
123+ ` ` ` toml
124+ option = "\{{ field_name }}"
125+ ```
126+
127+ {{< /tab >}}
128+ {{< /tabs >}}
129+
75130And [ strftime] specified like so:
76131
132+ {{< tabs default="YAML" >}}
133+ {{< tab title="YAML" >}}
134+
77135``` yaml
78136option : " year=\\ %Y/month=\\ %m/day=\\ %d/"
79137` ` `
80138
139+ {{< /tab >}}
140+ {{< tab title="TOML" >}}
141+
142+ ` ` ` toml
143+ option = "year=\%Y/month=\%m/day=\%d/"
144+ ```
145+
146+ {{< /tab >}}
147+ {{< /tabs >}}
148+
81149Each of the values above would be treated literally.
82150
83151## How it works
@@ -92,6 +160,9 @@ You can find additional examples for accessing fields in the
92160Vector doesn't currently support fallback values, [ issue 1692] [ 1692 ] is open to add this functionality. In the interim,
93161you can use the [ ` remap ` transform] [ remap ] to set a default value:
94162
163+ {{< tabs default="YAML" >}}
164+ {{< tab title="YAML" >}}
165+
95166``` yaml
96167transforms :
97168 set_defaults :
@@ -104,6 +175,23 @@ transforms:
104175 }
105176` ` `
106177
178+ {{< /tab >}}
179+ {{< tab title="TOML" >}}
180+
181+ ` ` ` toml
182+ [transforms.set_defaults]
183+ type = "remap"
184+ inputs = ["my-source-id"]
185+ source = '''
186+ if !exists(.my_field) {
187+ .my_field = "default"
188+ }
189+ ' ''
190+ ```
191+
192+ {{< /tab >}}
193+ {{< /tabs >}}
194+
107195### Missing fields
108196
109197If a field is missing, an error is logged and Vector drops the event. The `component_errors_total` internal
0 commit comments