fix(llm): cost metrics calculation for unsupport litellm prefix

[xingyaoww]

End-user friendly description of the problem this fixes or functionality that this introduces

• Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

---

Give a summary of what the PR does, explaining any non-trivial design decisions

Currently, our metric cost calculation doesn't work if the model is served through LiteLLM proxy.

This PR

• first try to grab llm_provider-x-litellm-response-cost from the litellm proxy response https://docs.litellm.ai/docs/proxy/cost_tracking#how-to-track-spend-with-litellm
• adds a fallback (eg remove the litellm_proxy/ prefix for cost) -- so as long as the model name is the same, it will get the cost correctly.

---

Link of any specific issues this addresses

---

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:40ca3d5-nikolaik   --name openhands-app-40ca3d5   docker.all-hands.dev/all-hands-ai/openhands:40ca3d5

[enyst]

Is this the header name, or just x-litellm-response-cost?
https://github.com/BerriAI/litellm/blob/3230c902c9d9c4c933773e946b744965573a14f9/litellm/proxy/proxy_server.py#L738

[xingyaoww]

yeah i think this is the actual key in that dict 🤔

[enyst]

Strange! 😅

OK, let's go with it

[xingyaoww]

Double checked again to make sure!

[enyst]

Looks reasonable, thanks!

(I think sometime we could move this to the logging mixin or something, we seem to have more and more code unrelated to the main purpose of this class to keep track of in the same place 😅)