fastapi: fix wrapping of middlewares

[adriangb]

MRE:

import anyio
import fastapi
import httpx
import uvicorn
from opentelemetry import trace
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter

resource = Resource(attributes={SERVICE_NAME: '🐛'})

traceProvider = TracerProvider(resource=resource)
processor = BatchSpanProcessor(ConsoleSpanExporter())
traceProvider.add_span_processor(processor)
trace.set_tracer_provider(traceProvider)

app = fastapi.FastAPI()

@app.get('/ok')
async def ok() -> int:
    return 123

@app.get('/error')
async def error() -> int:
    raise Exception('An error occurred!')

FastAPIInstrumentor.instrument_app(app)


async def main() -> None:
    config = uvicorn.Config(app=app, port=9999, log_config=None)
    server = uvicorn.Server(config)

    async with anyio.create_task_group() as tg:
        tg.start_soon(server.serve)
        await anyio.sleep(1)  # Wait for the server to start
        async with httpx.AsyncClient(base_url='http://localhost:9999') as client:
            response = await client.get('/ok')
            assert response.status_code == 200, f'{response.status_code}: {response.text}'
            response = await client.get('/error')
            assert response.status_code == 500,  f'{response.status_code}: {response.text}'
            tg.cancel_scope.cancel()


if __name__ == '__main__':
    anyio.run(main)

With this change it shows up properly

Run this and you'll see that there is no "http.status_code": 500 in the logs.

[codefromthecrypt]

nice comment and test!

[xrmx]

@adriangb Please take a look at the failing tests

[Kludex]

fixes #795

[adriangb]

@adriangb Please take a look at the failing tests

@xrmx see #3012 (comment)

[adriangb]

@Kludex could you update the Starlette instrumentations after we finish up this PR? I'm assuming they have the same bugs.

[Kludex]

The pipeline has been running for some hours... 👀

[vanyae-cqc]

Hello! Just wondering if you'd expect this to enable getting the trace_id in a FastAPI exception_handler?

This has been discussed in more detail here: open-telemetry/opentelemetry-python#3477, but currently OpenTelemetryMiddleware's context seems to be removed by the time an exception gets to the ServerErrorMiddleware.

Would be great to enable users to report back a trace_id when they get an unexpected 500 status response. Have tried to verify this PR provides this by running it locally but haven't had any luck - could be something wrong in my setup though.

[Update]: Have now verified this achieves the above behaviour - this fix is greatly appreciated!

[adriangb]

Fixed issue with test shutdown hanging. All checks passing now.

[xrmx]

I trust you on the fastapi knowledge but would be nice to add a test to check that the change does what it's supposed to fix

[adriangb]

Ups looks like that got lost, I had added it in 4220b09

[adriangb]

Added back and updated the branch

[xrmx]

ICYMI the added test is red

[adriangb]

Thanks. I hadn't understood how the tests are parametrized and that _create_app is overridden in a subclass.

[adriangb]

@xrmx @lzchen can we get this merged please? I believe this is a major issue that deserves attention.

[lzchen]

@adriangb

I believe I still have an outstanding question

[adriangb]

@adriangb

I believe I still have an outstanding question

I've answered it. Please re-review.

[outergod]

adriangb#2

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: adriangb / name: Adrian Garcia Badaracco (a5db685, c75b280, c1c4300, 14d8159, 834392f, 128cf89, d635026, d8ca85e, af0bf2c, 8a2bfc7, 83d4863, bfe110c, a979966, 4c4ac68, 00840d7, 0da827b, d3b9910, 3d7d537, f75dbef, fdba820, 8057faa, 04cf5d5, d2cd389, 121813e, b3bea3c, d7f7cc6, 0c32f4d, 2498309, 289f788, 887475b, 7ac54ae, 23a04a5, f6dd589, 18b8962, fcc62f4)
• ✅ login: emdneto / name: Emídio Neto (8fcf4ec, faf0cef, 17e08fa, 5a84bcc, 808d42a, 87aa3f6, 60e1706, 2a150fd)
• ✅ login: xrmx / name: Riccardo Magliocchetti (1a485a8, 0870986, a648666)
• ✅ login: outergod / name: Alexander Dorn (5fa64a5)

[outergod]

I've just signed the CLA, any way to re-trigger the check?

[emdneto]

@adriangb CI should be OK now

[outergod]

Anything else in the way of merging now?

[emdneto]

@adriangb I fixed the changelog conflict and added this note:
opentelemetry-instrumentation-fastapi: Drop support for FastAPI versions earlier than 0.92

[emdneto]

@outergod @adriangb can you please fix the conflict?

[adriangb]

I gave it a shot, lets see what CI says

[xrmx]

@adriangb we merged a PR stopping using SpanAttributes in favor of semconv._incubating.attributes.*

[adriangb]

Ok sounds like this 2 year old PR now needs another round of adjusting for upstream changes 😢. Any chance you can make the change here since you're familiar with the implications?

[alexmojaki]

Does @codefromthecrypt need to approve?

[xrmx]

@alexmojaki github was saying so but your approval was good enough 😅

[alexmojaki]

Thanks everyone for getting this across!

[adriangb]

Amazing folks thanks so much!!