Skip to content

Commit fd47a3e

Browse files
committed
add docs for evals submission
1 parent 144c26f commit fd47a3e

File tree

1 file changed

+56
-0
lines changed
  • content/en/llm_observability/setup/sdk

1 file changed

+56
-0
lines changed

content/en/llm_observability/setup/sdk/java.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -484,6 +484,62 @@ public class MyJavaClass {
484484
}
485485
```
486486

487+
## Evaluations
488+
489+
The LLM Observability SDK provides the methods `LLMObs.SubmitEvaluation()` to help your traced LLM application submit evaluations to LLM Observability.
490+
491+
### Submit evaluations
492+
493+
`LLMObs.SubmitEvaluation()` can be used to submit your custom evaluation associated with a given span.
494+
495+
#### Arguments
496+
497+
The `LLMObs.SubmitEvaluation()` method accepts the following arguments:
498+
499+
`llmObsSpan`
500+
: required - _LLMObsSpan_
501+
<br />The span context to associate the evaluation with.
502+
503+
`label`
504+
: required - _string_
505+
<br />The name of the evaluation.
506+
507+
`categoricalValue` or `scoreValue`
508+
: required - _string or double_
509+
<br />The value of the evaluation. Must be a string (for categorical evaluations) or a double (for score evaluations).
510+
511+
`tags`
512+
: optional - _dictionary_
513+
<br />A dictionary of string key-value pairs that users can add as tags regarding the evaluation. For more information about tags, see [Getting Started with Tags][3].
514+
515+
516+
517+
### Example
518+
519+
```java
520+
import datadog.trace.api.llmobs.LLMObs;
521+
522+
public class MyJavaClass {
523+
public String invokeChat(String userInput) {
524+
LLMObsSpan llmSpan = LLMObs.startLLMSpan("my-llm-span-name", "my-llm-model", "my-company", "maybe-ml-app-override", "session-141");
525+
String chatResponse = "N/A";
526+
try {
527+
chatResponse = ... // user application logic to invoke LLM
528+
} catch (Exception e) {
529+
llmSpan.addThrowable(e);
530+
throw new RuntimeException(e);
531+
} finally {
532+
llmSpan.finish();
533+
534+
// submit evaluations
535+
LLMObs.SubmitEvaluation(llmSpan, "toxicity", "toxic", Map.of("language", "english"));
536+
LLMObs.SubmitEvaluation(llmSpan, "f1-similarity", 0.02, Map.of("provider", "f1-calculator"));
537+
}
538+
return chatResponse;
539+
}
540+
}
541+
```
542+
487543

488544
[1]: /account_management/api-app-keys/#add-an-api-key-or-client-token
489545
[2]: /llm_observability/terms/

0 commit comments

Comments
 (0)