Support specifying log levels for the attributes in an OpenAPI spec

[ae-ou]

Problem

Currently, when I define an OpenAPI spec, there is no clear-cut way for me to specify (or consistently control) which attributes can/should be logged by the users of the spec. This means that there has to be an implicit level of trust that the user of the API spec will log the data appropriately - which presents a couple of problems:

1. A developer using an OpenAPI spec (or the bindings generated from said spec) can easily log attributes (or an entire object) that may contain sensitive information. This could be avoided by allowing for the creation of sensible defaults.
   1. The developer may not know the API schema/bindings well enough to make an informed decision about which attributes can/should be logged. This could be caused by inexperience, or by poor developer documentation.
   2. A lot of us have left a print() statement in code that we only intended for use during debugging.
2. A number of popular API Gateways can be configured using OpenAPI specs, but since these specs don’t contain any information about the loggability of attributes, the log outputs tend to be very minimal (to avoid leaking sensitive request info) - which can complicate debugging.
   1. AWS API Gateway allows you to use an OpenAPI spec to directly configure an API gateway.
   2. GCP API Gateway allows you to use an OpenAPI spec to directly configure an API gateway.
   3. Kong allows you to convert an OpenAPI spec into a Kong spec using their openapi2kong tool.

TLDR: In the absence of proper information about which attributes are loggable, users of OpenAPI specs have to practice self-restraint with logging. Whilst a cautious approach is good, it directly impacts observability - which can hinder debugging efforts. Furthermore, since this self-restraint falls to individual developers, there are inherent inconsistencies in approach, and there’s an ever-present risk of a slip-up resulting in the leaking of sensitive information.

Request

I would like to be able to specify logging expectations as part of an OpenAPI spec. This would allow us to move from implicitly trusting developers/services to handle logging (and any redaction that may need to occur), to explicitly declaring expectations around logging. I think that this would provide a few benefits:

1. The maintainers of an API spec are already the authority on that specific API, and they (hopefully) have the most domain knowledge about it. These people are therefore the best suited for saying which attributes are sensitive, and which can/should be logged.
   1. Allowing maintainers to specify logging expectations would effectively just extend their existing authority.
2. We would effectively be adding [log-specific] guardrails. By allowing the maintainers of an API spec to define logging expectations, we can minimise the risk of an inexperienced developer logging sensitive information.
   1. The information about attribute loggability would be visible in the API spec, and it could be displayed as part of any documentation that's based on the spec.
   2. When we generate bindings from an OpenAPI spec, we could also generate helper functions to control the logging/serialisation of objects and attributes.
      1. This would mean that individual developers no longer have to make their own helper functions (or manually add object attributes to log fields).
3. If the OpenAPI spec already defines the loggability of individual attributes, then API Gateways won't have to default to minimising/quashing logs in an effort to avoid outputting sensitive information. This should help with observability and debugging.
4. If the full stack (e.g. JS client bindings, Go server bindings, AWS API Gateway) uses the same OpenAPI spec, then we could consistently output the same log fields. This consistency helps with observability and log correlation.
   1. Obviously, trace headers help with correlation, but tooling to make use of these headers isn't always available. Even when trace headers are available, there could still be a need to set log fields using attributes.

Potential Approach

Suggested Changes to OpenAPI

I think that we should be able to specify logging expectations for individual object attributes inside of an OpenAPI schema - this could be done by adding a field called loggability. This loggability field could then be
used to specify when an attribute can appear in logs (i.e. the log levels at which it can be added as a log field).

Potential Values for the loggability Field

Log levels differ between programming languages and log packages, so we could base the values for loggability on the OpenTelemetry severity levels. Since OpenTelemetry is a platform-agnostic open standard (backed by the CNCF), there shouldn't be any concerns about favouring a specific language/log package/service provider.

It would make sense to only log an attribute when the loggability field is explicitly defined. This default behaviour is beneficial for two reasons:

1. If we logged every attribute by default, then the size of each log could get out of hand very quickly - which would create a lot of operational overheads.
2. Logging everything by default would mean logging sensitive information by default.

In addition to not logging attributes by default, we could add a value to represent explicitly denying the logging of a field (e.g. DO_NOT_LOG). This would allow the maintainers of an OpenAPI spec to explicitly communicate that they do not want a field to be logged.

Given the aforementioned considerations, the values for loggability could look like this

Value	Source	Notes
TRACE	OpenTelemetry severities	This is the lowest severity log level as it will output a lot of information. These logs are only visible when an application is configured to output logs at the TRACE level.
DEBUG	OpenTelemetry severities	Typically only visible when a program is configured to output logs at the DEBUG level or lower (i.e. DEBUG or TRACE).
INFO	OpenTelemetry severities	When a log is output in code, INFO is the default log level. In comparison, the default for the loggability field is to be unset, and when this field is unset, the corresponding attribute should not be logged at all.
WARN	OpenTelemetry severities
ERROR	OpenTelemetry severities	If a validation error occurs (i.e. when a request payload is malformed) then any attributes with loggability: ERROR could be added to the error log (assuming that the malformed payload is still parseable).
FATAL	OpenTelemetry severities
/ unset		This is an empty string that represents a unset loggability field (which is the default). If an attribute has no loggability field defined, the attribute shouldn't be logged.
DO_NOT_LOG		A way to explixcitly denote that an attribute should never be logged, and that there was an intentional decision to not log the attribute. We could use SENSITIVE (or another value) instead, but this could be misinterpreted as an instruction to redact data.

Examples

Error

    lunch:
      type: object
      description: Information about lunch
      properties:
        fruit:
          type: string
          loggability: ERROR
          description: Which piece of fruit was selected
          enum:
          - Apple
          - Orange
          - Banana

In the above example, the fruit attribute has loggability: ERROR set on it - which means that the attribute can be added to logs that are ERROR level or lower.

With this configuration, we may log the attribute if a payload [containing the attribute] fails validation (e.g. if somebody specified "fruit": "guava").

Not Set/Default

    sweater:
      type: object
      description: Information about someone's sweater
      properties:
        colour:
          type: string
          description: The colour of the sweater
          enum:
          - It's not just blue
          - It's not turquoise
          - It's not lapis
          - It's actually cerulean

In the above example, the loggability field hasn't been set on the colour attribute - so colour should never be output to logs.

Do Not Log

    contactInfo:
      type: object
      description: Information used to contact a person
      required:
        - email
        - mobileNumber
      properties:
        email:
          type: string
          loggability: DO_NOT_LOG
          description: An email address
          format: email          
        mobileNumber:
          type: string
          loggability: DO_NOT_LOG
          description: A UK-based mobile number
          pattern: ^(\+44\s?7\d{3}|\(?07\d{3}\)?)\s?\d{3}\s?\d{3}$

In the above example, both email and mobileNumber have loggability: DO_NOT_LOG. This shows that a conscious decision has been made to prevent the logging of the attributes under any circumstances (since both attributes contain PII).

Possible Language-specific Usages

Go

The loggability field could be used in multiple places when generating Go bindings for an OpenAPI spec:

1. We could conditionally set struct tags based on the loggability field. Struct tags are relatively free form, so purpose-specific struct tags could be set that suit individual needs.
   1. We could set json:"-" on a struct field if there is a corresponding loggability: DO_NOT_LOG in the OpenAPI spec
   2. These tags could be used by internal tooling (e.g. something could be written to enforce redaction).
2. The log/slog package defines a LogValuer interface which allows developers to control how a struct should be logged - we could autogenerate implementations of this interface based on the loggability fields within the OpenAPI spec.
   1. This LogValuer interface can be used to hide sensitive fields.
   2. This interface doesn't receive information about the log level - so it wouldn't be possible to conditionally alter the output.
   3. Likewise, Zerolog defines a LogObjectMarshaler interface which operates in a similar manner.
3. We could autogenerate implementations of the Marshaler interface in order to control how a struct is marshalled into JSON.

Java

The loggability field could be used in multiple places when generating Java bindings for an OpenAPI spec:

1. We could autogenerate the toString() method to control which fields are output/masked.
   1. This method doesn't receive information about the log level - so it wouldn't be possible to conditionally alter the output.
2. We could generate a Data Transfer Object (DTO) for each object within the OpenAPI spec - to provide a restricted view of the attributes from the object.
   1. This could be expanded so that there are multiple DTOs per object (i.e. one DTO per log level).

Considerations

OpenAPI Extension

I was initially going to suggest this as an OpenAPI Extension since logging isn't fully related to OpenAPI, however, there are a few reasons that I'm suggesting it as part of the wider OpenAPI spec:

1. The values for the loggability field (e.g. ERROR, DEBUG, DO_NOT_LOG) would extend the OpenTelemetry specification. Extending/altering a specification usually triggers debate, and I figured that suggesting this loggability field as a part of the wider OpenAPI specification would be more conducive to such a debate.
2. My suggestion focuses on standardising logging across all consumers of a given OpenAPI spec. The implementation of OpenAPI extensions seems to vary across languages and providers - which would be counterproductive given my goal of having a common/standard approach.
3. A growing number of teams employ a design-first approach to API development - which means that OpenAPI specs increasingly act as a source of truth for a company's data models. Since OpenAPI specs are often elevated to such a point of authority, I think it makes sense to allow OpenAPI specs to say how the data (that they model) should be logged.
4. Whilst OpenAPI and OpenTelemetry are concerned with two different domains, there are already touchpoints/areas of common interest between the two (e.g. the traceparent, tracestate, and X-Request-ID headers).

Related Work/Issues

#4330 requested an x-sensitive-data field and suggested some excellent approaches for redacting sensitive data.

My loggability: DO_NOT_LOG field would behave similarly to @cristigirbovan's full masking proposal:

    x-sensitive-data: 
        mask: full

Both of our suggestions would seek to completely eliminate specified fields from logs, but the difference is that my
loggability: DO_NOT_LOG field isn't specific to sensitive data - it's for any field that we explicitly don't want to log.

Whilst their proposal focuses on known sensitive data (and the most effective ways to redact it), my proposal
is focused on logging in general (including attributes that aren't sensitive). Realistically, our proposals are complimentary
since my proposal focuses on when to log attributes, and theirs focuses on how to log [known sensitive] attributes.

Hypothetically, our proposals could be combined into one overarching spec change that governs logging and redaction as
a whole. For example:

    merchantTransaction:
      type: object
      description: Information about a purchase from a store
      required:
        - fullCardNumber
        - merchantID
        - amount
      properties:
         fullCardNumber:
          type: string
          description: A full 16 digit card number. We only output it for DEBUG/TRACE logs, and when we do log it, we redact everything but the last 4 digits.
          logging:
            loggability: DEBUG
            sensitive-data:
               mask: regex
               pattern: "^[\d]{12}"
         merchantID:
          type: string
          logging:
             loggability: INFO
          description: The ID of the merchant where the transaction was made.
         amount:
            type: integer
            description: An int-based representation of the transaction amount. We don't want to log this because it could be commercially sensitive.
            logging:
               loggability: DO_NOT_LOG

The above sample shows my proposal in conjunction with @cristigirbovan's x-sensitive-data proposal. Both of our
proposed fields have been centralised under the logging field, the x- prefix has been removed, and I'm using my DO_NOT_LOG in lieu of mask: full.