Skip to content

v0.53.0

Choose a tag to compare

@chris-colinsky chris-colinsky released this 09 Jun 02:05
· 452 commits to main since this release
bd2d782

Added

  • LlmFailedEvent — second typed event variant on the observer event union. Carves LLM provider failures into a spec-normatively-typed event variant alongside LlmCompletionEvent (proposal 0049 / 0057). Mirrors LlmCompletionEvent's identity / scoping / request-side field set 1:1 (17 fields — invocation_id, correlation_id, node_name, namespace, attempt_index, fan_out_index, branch_name, provider, model, latency_ms, caller_invocation_metadata, input_messages, request_params, request_extras, active_prompt, active_prompt_group, call_id) plus three failure-specific fields: error_category (always-present; one of the llm-provider §7 normative categories — provider_authentication, provider_unavailable, provider_invalid_model, provider_model_not_loaded, provider_rate_limit, provider_invalid_response, provider_invalid_request, provider_unsupported_content_block, structured_output_invalid), error_type (OPTIONAL impl-level / vendor-specific error type or code — null when no impl-side type is available; two acceptable styles: vendor error code or upstream exception class name), error_message (always-present human-readable message from the raised exception; empty string when the exception carried no message). Response-side fields (response_id, response_model, usage, output_content, finish_reason) are absent from the failure variant — no response was received. (proposal 0058)
  • Mutual exclusion + exception-flow preservation. The two variants are mutually exclusive on a given provider.complete() call; implementations MUST NOT emit both for the same call (codified in graph-engine §6 + locked down by a dedicated conformance fixture). The provider exception-flow contract is preserved per proposal 0049's alternative-3 framing — failures still raise the §7 category exception out of provider.complete(); the typed event is dispatched on the observer delivery queue alongside the exception, not in place of it. Lands the LlmCallFailedEvent typed-variant follow-on proposal 0049's Out of scope section anticipated by name.
  • graph-engine §6 Typed LLM completion event + observability §5.5.7 — extended with the failure-side variant. A new Typed LLM failure event paragraph block in graph-engine §6 (alongside the existing Typed LLM completion event paragraph from proposal 0049) carries the LlmFailedEvent field table and dispatch contract. A new Typed LLM failure event paragraph in observability §5.5.7 frames the failure-side typed event as the structured form of LLM-call failure observability and notes that with both LlmCompletionEvent and LlmFailedEvent defined, the impl-current sentinel-namespace NodeEvent convention for LLM observability can retire fully across both outcome sides.
  • Five new conformance fixtures under spec/observability/conformance/: 069-llm-failure-event-dispatch-on-provider-unavailable (dispatch contract for the canonical transport-error category; verifies the exception still raises out of provider.complete() alongside the typed event), 070-llm-failure-event-dispatch-on-provider-invalid-request (companion variant on a pre-send-validation category to verify dispatch consistency across §7 categories), 071-llm-failure-event-call-id-distinct-from-completion-event (per-call call_id mint contract — a failed call gets its own fresh call_id, distinct from any success-event call_id from a different call in the same invocation), 072-llm-failure-event-mutual-exclusion-with-completion-event (dedicated lockdown of the mutual-exclusion rule — exactly one LlmFailedEvent and exactly zero LlmCompletionEvent on a failed call), 073-llm-failure-event-error-type-vendor-specific (3-case fixture covering the two acceptable error_type styles — vendor error code and upstream exception class name — plus the null companion when no impl-side type is available).

Notes

  • MINOR bump (pre-1.0). Purely additive at the spec level — the new event variant adds a second discriminator-on-type slot on the observer event union; LlmCompletionEvent's field set and dispatch contract from proposal 0049 / 0057 are unchanged. Observers consuming only LlmCompletionEvent continue to work unchanged; observers wanting to consume the failure event opt in via type discrimination on the new variant. Existing fixtures (001-068) continue to pass unchanged. With this proposal, the LLM-call observability surface has full type-discriminated coverage on both outcome sides — implementations adopting both LlmCompletionEvent and LlmFailedEvent consumption have a path to retiring the sentinel-namespace NodeEvent convention entirely.