Prompt Flow incomplete documentation

[mennolaan]

Hi!

Im trying to wrap my head around the prompt flow logic as I expected more logic in the openapi.json and policy.xml

The animated gif creates an assumption that we make a request to Apim to one api endpoint. Within api management this request gets send to prompt flow hosted in a container and than upon retrieval is send to openai endpoint with compressed prompt.

However the policy.xml is empty and the openapi.json only displays a score result. Also the descriptions says this " The Prompt Flow OpenAI connection will be facilitated by APIM, enabling load balancing, token counting, and other features. " So one would at least expect some aspects in the policy.xml.

Im mostly interested how the Prompt Flow hosted in a container related to api management.

But I have a feeling you're creating a prompt flow that does the compression and also passes the info to openai. Thus doing the load balancing within prompt flow?

Maybe my expectation is not correct, but it feels like this lab isn't complete.

[simonkurtz-MSFT]

Hi @mennolaan, thank you for submitting this issue! I'm assigning to @vieiraae to have a look as he is the originator of that lab and has the most context.