How can I see the LLM Judge evaluation in Phoenix UI?

[ralucaginga]

Hello, I am still struggling to follow the documentation from the website and see a lot of changes are happening with the versions of Arize Phoenix, but could you please assist on how to setup an evaluator (correctness/relevance) and show the evaluation results in the Phoenix UI as well?

Unfortunately, there is not anything shown in the UI and tried to follow the open issues, but nothing helped.

Tried the following pieces of code, but unfortunately nothing is shown in the UI:

# Convert evaluation results to annotation format
correctness_annotations = to_annotation_dataframe(dataframe=correcteness_df)

px_client.spans.log_span_annotations_dataframe(
    dataframe=correctness_annotations,
    annotator_kind="LLM"
)

Thanks.

[anticorrelator]

hi @ralucaginga how are you creating your correctness_annotations? are you using evals 2? If you show an example of your workflow up until that point I can help explain how to get up and running

[anticorrelator]

I'll add that to_annotation_dataframe is intended for use with the output of evaluate_dataframe from evals 2.0 and not the legacy llm_classify output

[ralucaginga]

from phoenix.evals import ClassificationEvaluator
from phoenix.evals.llm import LLM
from phoenix.evals.evaluators import async_evaluate_dataframe
from phoenix.evals.utils import to_annotation_dataframe

eval_model = LLM(
    provider="google",
    model="gemini-2.5-flash",
)

CATEGORICAL_TEMPLATE = '''You are comparing a reference text to a question and trying to determine if the reference text
contains information relevant to answering the question. Here is the data:
    [BEGIN DATA]
    ************
    [Question]: {query}
    ************
    [Reference text]: {reference}
    [END DATA]

Compare the Question above to the Reference text. You must determine whether the Reference text
contains information that can answer the Question. Please focus on whether the very specific
question can be answered by the information in the Reference text.
"irrelevant" means that the reference text does not contain an answer to the Question.
"relevant" means the reference text contains an answer to the Question. '''

choices = {"irrelevant": 0, "relevant": 1}

relevance_classifier = ClassificationEvaluator(
    name="relevance",
    prompt_template=CATEGORICAL_TEMPLATE,
    llm=eval_model,
    choices=choices
)
results = relevance_classifier.evaluate({"query": "paris is the capital of france", 
                                         "reference": "Paris, the capital of France, is a world-renowned city for its history, art, and architecture. Located in the heart of the Paris Basin"})

print(results)

df_adapted = df.copy()
df_adapted = df_adapted.rename({"input": "query"}, axis=1)

relevance_classifier_df = await async_evaluate_dataframe(
        dataframe=df_adapted,
        evaluators=[relevance_classifier],
    )

relevance_annotations = to_annotation_dataframe(relevance_classifier_df)

await px_client.spans.log_span_annotations_dataframe(
    dataframe=relevance_annotations,
    annotator_kind="LLM"
)

It also doesn't display anything when running a simple example:

from phoenix.client import AsyncClient, Client

client = Client()

df1 = pd.DataFrame({
                "name": ["sentiment", "toxicity"],
                "span_id": ["span_123", "span_456"],
                "annotator_kind": ["HUMAN", "LLM"],
                "label": ["positive", "low"],
                "score": [0.9, 0.1]
            })
client.spans.log_span_annotations_dataframe(dataframe=df1)

Could you please assist in how to do that? I cannot access the Slack community unfortunately, the link seems to be expired.

Versions:
Arize Phoenix = 12.32.0
Arize Phoenix Evals = 2.8.0
Arize Phoenix Client = 1.28.0
Arize Phoenix Otel = 0.14.0

[anticorrelator]

Hi @ralucaginga

For your evaluation example: I don't see where df comes from. For annotations to work, the DataFrame needs to contain valid span IDs that exist in Phoenix. Our recommended workflow is:

1. Start by fetching spans from Phoenix

spans_df = client.spans.get_spans_dataframe(project_name="your-project")

2. Prepare DataFrame while preserving span_id

eval_df = spans_df[["context.span_id", "attributes.input.value"]].copy()
eval_df = eval_df.rename(columns={"attributes.input.value": "query"})
eval_df["reference"] = "..." # add your reference data
eval_df = eval_df.set_index("context.span_id") # span_id must be index or column

3. Run evaluation

results_df = await async_evaluate_dataframe(eval_df, evaluators=[relevance_classifier])

4. Convert and log with sync=True to see any errors

annotation_df = to_annotation_dataframe(results_df)
result = client.spans.log_span_annotations_dataframe(
dataframe=annotation_df,
sync=True # Important: returns 404 if span IDs don't exist
)
print(f"Created {len(result)} annotations")

For your simple example: The span IDs "span_123" and "span_456" need to be real span IDs from traces in Phoenix. With sync=False (the default), invalid span IDs silently fail with no error.

To debug:

1. Can you share where your df comes from in the evaluation example?
2. Try adding sync=True to your calls - this will raise an error if span IDs don't exist
3. Verify you have spans: print(client.spans.get_spans_dataframe().index[:5])