feat(om2.0): Add UTF-8 metric names and labels (#2746)

Part of prometheus/prometheus#16093

Signed-off-by: Owen Williams <[email protected]>

Author: ywwg

docs/specs/om/open_metrics_spec_2_0.md

@@ -92,7 +92,7 @@ Strings MUST only consist of valid UTF-8 characters and MAY be zero length. NULL
 
 Labels are key-value pairs consisting of strings.
 
-Label names beginning with underscores are RESERVED and MUST NOT be used unless specified by this standard. Label names MUST follow the restrictions in the ABNF section.
+Label names beginning with two underscores are RESERVED and MUST NOT be used unless specified by this standard. Label names SHOULD follow the restrictions in the ABNF section under the `label-name` section. Label names MAY be any quoted escaped UTF-8 string as described in the ABNF section. Be aware that exposing UTF-8 metrics is still experimental and may reduce usability.
 
 Empty label values SHOULD be treated as if the label was not present.
 
@@ -128,7 +128,7 @@ A MetricFamily MAY have zero or more Metrics. A MetricFamily MUST have a name, H
 
 ##### Name
 
-MetricFamily names are a string and MUST be unique within a MetricSet. Names SHOULD be in snake_case. Metric names MUST follow the restrictions in the ABNF section.
+MetricFamily names are a string and MUST be unique within a MetricSet. Names SHOULD be in snake_case. Names SHOULD follow the restrictions in the ABNF section under `metricname`. Metric names MAY be any quoted and escaped UTF-8 string as described in the ABNF section. Be aware that exposing UTF-8 metrics is still experimental and may reduce usability, especially when suffixes are not included.
 
 Colons in MetricFamily names are RESERVED to signal that the MetricFamily is the result of a calculation or aggregation of a general purpose monitoring system.
 
@@ -329,22 +329,24 @@ metricset = *metricfamily
 
 metricfamily = *metric-descriptor *metric
 
-metric-descriptor = HASH SP type SP metricname SP metric-type LF
-metric-descriptor =/ HASH SP help SP metricname SP escaped-string LF
-metric-descriptor =/ HASH SP unit SP metricname SP *metricname-char LF
+metric-descriptor = HASH SP type SP (metricname / metricname-utf8) SP metric-type LF
+metric-descriptor =/ HASH SP help SP (metricname / metricname-utf8) SP escaped-string LF
+metric-descriptor =/ HASH SP unit SP (metricname / metricname-utf8) SP *metricname-char LF
 
 metric = *sample
 
 metric-type = counter / gauge / histogram / gaugehistogram / stateset
 metric-type =/ info / summary / unknown
 
-sample = metricname [labels] SP number [SP timestamp] [SP created] [exemplar] LF
+sample = metricname-and-labels SP number [SP timestamp] [SP created] [exemplar] LF
 
-exemplar = SP HASH SP labels SP number [SP timestamp]
+exemplar = SP HASH SP labels-in-braces SP number [SP timestamp]
 
-labels = "{" [label *(COMMA label)] "}"
+metricname-and-labels = metricname [labels-in-braces] / name-and-labels-in-braces
+labels-in-braces = "{" [label *(COMMA label)] "}"
+name-and-labels-in-braces = "{" metricname-utf8 *(COMMA label) "}"
 
-label = label-name EQ DQUOTE escaped-string DQUOTE
+label = label-key EQ DQUOTE escaped-string DQUOTE
 
 number = realnumber
 ; Case insensitive
@@ -386,13 +388,16 @@ metricname = metricname-initial-char 0*metricname-char
 
 metricname-char = metricname-initial-char / DIGIT
 metricname-initial-char = ALPHA / "_" / ":"
+metricname-utf8 = DQUOTE escaped-string-non-empty DQUOTE
 
+label-key = label-name / DQUOTE escaped-string-non-empty DQUOTE
 label-name = label-name-initial-char *label-name-char
 
 label-name-char = label-name-initial-char / DIGIT
 label-name-initial-char = ALPHA / "_"
 
 escaped-string = *escaped-char
+escaped-string-non-empty = 1*escaped-char
 
 escaped-char = normal-char
 escaped-char =/ BS ("n" / DQUOTE / BS)
@@ -448,6 +453,8 @@ A double backslash SHOULD be used to represent a backslash character.
 A single backslash SHOULD NOT be used for undefined escape sequences.
 As an example, `\\\\a` is equivalent and preferable to `\\a`.
 
+Escaping MUST also be applied to quoted UTF-8 strings.
+
 ##### Numbers
 
 Integer numbers MUST NOT have a decimal point. Examples are `23`, `0042`, and `1341298465647914`.
@@ -751,7 +758,7 @@ Metric labels and MetricPoint value labels MAY be in any order.
 
 ##### Summary
 
-If present, the MetricPoint's Sum Value Sample MetricName MUST have the suffix `_sum`. If present, the MetricPoint's Count Value MetricName MUST have the suffix `_count`. If present, the MetricPoint's Quantile Values MUST specify the quantile measured using a label with a label name of "quantile" and with a label value of the quantile measured. 
+If present, the MetricPoint's Sum Value Sample MetricName MUST have the suffix `_sum`. If present, the MetricPoint's Count Value MetricName MUST have the suffix `_count`. If present, the MetricPoint's Quantile Values MUST specify the quantile measured using a label with a label name of "quantile" and with a label value of the quantile measured.
 
 If present the MetricPoint's Created Timestamp MUST be inlined with the Metric point with a `ct@` prefix. If the value's timestamp is present, the Created Timestamp MUST be added right after it. If exemplar is present, the Created Timestamp MUST be added before it. Created Timestamp MUST be appended to all Quantile Values, to the MetricPoint's Sum and MetricPoint's Count.
 
@@ -775,7 +782,7 @@ Quantiles MAY be in any order.
 
 ##### Histogram
 
-The MetricPoint's Bucket Values Sample MetricNames MUST have the suffix `_bucket`. If present, the MetricPoint's Sum Value Sample MetricName MUST have the suffix `_sum`. 
+The MetricPoint's Bucket Values Sample MetricNames MUST have the suffix `_bucket`. If present, the MetricPoint's Sum Value Sample MetricName MUST have the suffix `_sum`.
 
 If present the MetricPoint's Created Timestamp MUST be inlined with the Metric point with a `ct@` prefix. If the value's timestamp is present, the Created Timestamp MUST be added right after it. If exemplar  is present, the Created Timestamp MUST be added before it. Created Timestamp MUST be appended to all Bucket Values, to the MetricPoint's Sum and MetricPoint's Count.
 
@@ -980,7 +987,7 @@ For high availability and ad-hoc access a common approach is to have multiple in
 # EDITOR’S NOTE:  This section might be good for a BCP paper.
 -->
 
-We aim for a balance between understandability, avoiding clashes, and succinctness in the naming of metrics and label names. Names are separated through underscores, so metric names end up being in “snake_case”.
+We aim for a balance between understandability, avoiding clashes, and succinctness in the naming of metrics and label names. Names are separated through underscores, so metric names end up being in “snake_case”. While we strongly recommend the practices recommended in this document, other metric systems have different philosophies regarding naming conventions. OpenMetrics allows these metrics to be exposed, but without the conventions and suffixes recommended here there is an increased risk of collisions and incompatibilities along the chain of services in a metrics system. Users wishing to use alternative conventions will need to take special care and expend additional effort to ensure that the entire system is consistent.
 
 To take an example "http_request_seconds" is succinct but would clash between large numbers of applications, and it's also unclear exactly what this metric is measuring. For example, it might be before or after auth middleware in a complex system.
 
@@ -1020,7 +1027,7 @@ While there is metadata about metric names such as HELP, TYPE and UNIT there is
 
 ### Metric Names versus Labels
 
-There are situations in which both using multiple Metrics within a MetricFamily or multiple MetricFamilies seem to make sense. Summing or averaging aMetricFamily should be meaningful even if it's not always useful. For example, mixing voltage and fan speed is not meaningful.
+There are situations in which both using multiple Metrics within a MetricFamily or multiple MetricFamilies seem to make sense. Summing or averaging a MetricFamily should be meaningful even if it's not always useful. For example, mixing voltage and fan speed is not meaningful.
 
 As a reminder, OpenMetrics is built with the assumption that ingestors can process and perform aggregations on data.