Describe expected response times for responses

[agardnerIT]

I can describe expected return codes, response types and even expected responses via the spec: https://swagger.io/docs/specification/describing-responses/

What I feel is missing right now is the ability to document expected response times. As a developer, I know my endpoint should return in <100ms with a 201.

Is there a way to achieve this today (perhaps via some custom annotation?) or would this require a spec update. If so, how do I formally submit that as a request?

[rafalkrupinski]

That's an interesting idea.

I guess it would make sense to put it under either operation or server, you could use description or add a custom attribute e.g. x-timeout-ms.

The only problem is, there can always be a delay on the way, so if the operation, including the network transmission, takes 101ms, and client developer imposes that 100ms as a timeout, they lose the response.

The typical default timeout is 30sec for the reason that the internet doesn't guarantee response times.

[agardnerIT]

My intention isn't to fail the response but more use it as a hint. Ie. when the request hits the API to the backend and back - that's the time I'm interested in hinting at.

Other actors can then infer expected behaviour. It would eliminate false callouts as I can see that actually, the dev intended it to take 5 seconds so no point in moaning that it is taking 3.

[agardnerIT]

@rafalkrupinski how do I progress this? Can this be achieved now and if so, how? If not possible today, I'm more than happy to jump on calls / present this proposal. Just let me know where and when and I'll turn up.

Looking forward to PoCing htis if it is possible (I'm an OpenAPI newbie so a noob guide would be great for me :) )

[rafalkrupinski]

@agardnerIT
OAS allows arbitrary extension fields so I don't think you need to start a long process to change it for your particular needs.

How you support timeouts depends on your tooling.
If you use a code generator to create the client code, you can add an extension field and handle it in the generator (You say you don't want to fail the request, but I guess you'd add a warning).

If the client is coded by hand, you can either use an extension or simply the description field and let the client developer handle it in their code.

It's also a matter of how you create the schema. If it's generated from the server code, it also needs to support adding extension fields.

[darrelmiller]

I would start by suggesting some x- extensions and let's see whether there is interest from a broader group of people. If it becomes popular we can considering incorporating the values into a future spec update.

[agardnerIT]

Thanks. x-expected-response-time-ms

[AMorgaut]

I join this request

We're working on a framework that reads many information from a mandatory Open API specification file including:

• request validation rules
• success http status code
• caching rules based on the defined response cache-control header

We also have mandatory response time requirements that we need to enforce in the specification as all routes don't have the same response time criticity based on business use cases.

I take this occasion to remind that this request also exists in fact in a still opened issue from 2016 ;-)

• Ability to define a API-level and method-level timeouts #695
  Which was then linked to another issue also opened since 2016:
• Adding optional SLA definitions with the Spec #541

As no-standard field exists yes in OAS, we'll for sure start defining a vendor specific x-timeout, x-expected-response-time-ms, or x-max-response-time related field, but that will not help getting a better native support by existing open api tools

We expect such field, once standardized, to be interpreted & used by:

• Spec-based test automation tools like portman, Robot Framework OpenApiDriver, Ready API, ...
• HTTP frameworks like Nest-js / Express on node via tools like express-openapi
• API Managers like the AWS API Gateway, Kong, Apigee, Gravitee, ...

Moreover we could define rules in tools like Spectral to expect a 408 response from routes that define such timeout to detect such kind of potential issues before running service contact tests

[rafalkrupinski]

Does any of the open-source tools you've mentioned support timeout via a x- property so far?

[AMorgaut]

I still have to dig

The first I can mention is the timeoutInMillis property of the x-amazon-apigateway-integration vendor specific object property from the AWS API Gateway

• https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions-integration.html
• https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions-any-method.html

On its side Apigee natively handles different types of timeouts in its products, but while you can publish Open API based services, their timeouts cannot be defined through that Open API specification. It only supports standard Open API fields

• https://docs.apigee.com/api-platform/reference/endpoint-properties-reference#targetendpointtransportpropertyspecification
• https://cloud.google.com/apigee/docs/api-platform/publish/api-design-overview

[agardnerIT]

Costs Codes

Suggestions: x-cost and related x-cost-code

x-cost: "0.03"
x-cost-code: "USD"

Examples

Endpoint Evaluation

I have a system which calls a third party API. I am evaluating between two different third parties for this APIs. All the usual SLOs apply (response time etc.) but cost is also a major factor. If endpointA costs 3 "whatevers" per call and endpointB costs 30 "whatevers" per call - that's a major plus point for endpointA.

Direct cost tracking

If I have the above, my tooling can aggregate the header data and show:

You have spent 123.45USD over the last 24hrs with endpointA. This is x% of your budget.

Different Definitions of Cost

Cost of course can be monetary, but a "cost" could also indicate other quantities (like a carbon emissions costs).

[GabenGar]

Doesn't this fall apart once proxy/reverse-proxy servers enter into the picture? They can add to a response time which can't be tracked by the application returning the header, and the client will not be able to tell apart a proxy/app response of 2/5 from 5/2.