Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] Using litellm Message object not supported with instrumentation #1303

Open
AdamRobertson-Cognasim opened this issue Feb 21, 2025 · 1 comment
Labels
bug Something isn't working triage Issues that require triage

Comments

@AdamRobertson-Cognasim
Copy link

AdamRobertson-Cognasim commented Feb 21, 2025

Describe the bug
When using LiteLLMInstrumentor, you cannot use the Message objects provided by LiteLLM.
You have to use dictionaries.

This is unfortunate since we use the Message objects extensively.

To Reproduce

from phoenix.otel import register
import litellm 
from openinference.instrumentation.litellm import LiteLLMInstrumentor

# Setup tracer provider
tracer_provider = register(
    project_name="test",
    endpoint="http://localhost:6006/v1/traces",
    )

# Setup LiteLLM prebuilt instrument
LiteLLMInstrumentor().instrument(tracer_provider=tracer_provider)

# This works
model_response = litellm.completion(model='anthropic/claude-3-haiku-20240307',
                               messages=[{'role':'user',  'content':'hello'}])

# This throws an exception: "Object of type Message is not JSON serializable"
model_response = litellm.completion(model='anthropic/claude-3-haiku-20240307',
                               messages=[litellm.Message(role='user',  content='hello'])

Expected behavior
Calling litellm methods with litellm.Message objects and getting traces without exceptions.

Desktop

  • OS: Windows
  • Version openinference-instrumentation-litellm==0.1.11
@AdamRobertson-Cognasim AdamRobertson-Cognasim added bug Something isn't working triage Issues that require triage labels Feb 21, 2025
@Arize-ai Arize-ai deleted a comment from dosubot bot Feb 21, 2025
@axiomofjoy
Copy link
Contributor

Thanks for reporting @AdamRobertson-Cognasim! We will take a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage Issues that require triage
Projects
Status: No status
Development

No branches or pull requests

2 participants