Langfuse observer: fan-out / subgraph / detached parenting#81
Merged
Conversation
Extends LangfuseObserver with synthetic dispatch observations matching the OTel observer's structure. Spec §8.3 mandates Span observations for subgraph wrappers, fan-out nodes, and per-instance fan-out spans; §8.5 covers detached-trace mode where a configured subgraph or fan-out gets its own Langfuse Trace and the parent's dispatch observation surfaces metadata.detached_child_trace_ids. _InvState gains six new fields: - subgraph_observations: synthetic dispatch Span observations keyed by namespace prefix (lives in main Trace for non-detached, or in the detached Trace for detached subgraphs) - fan_out_instance_observations: per-instance dispatch Span observations keyed by prefix + (str(fan_out_index),) - detached_traces: prefix -> detached trace_id mapping that switches descendant observations onto the detached Trace - fan_out_instance_root_prefixes: tracks detached fan-out instance prefixes for the close path - fan_out_parent_node_name: cache populated from fan_out_config on the fan-out node's started event, bridging the lookup for the per-instance attribution metadata - detached_child_trace_ids: side-cache accumulator for the link-ids array on dispatch observations spawning detached children (the Protocol doesn't expose a metadata read accessor) LangfuseObserver gains two constructor kwargs (detached_subgraphs, detached_fan_outs) mirroring the OTel observer's surface. _resolve_parent_observation_id rewritten with full precedence: per-instance fan-out dispatch > subgraph dispatch (walked longest- first) > leaf-node ancestor walk > Trace root. Same precedence applied to _resolve_llm_parent_observation_id so an LLM call from inside a subgraph/fan-out parents correctly. _trace_id_for picks the right Trace (main or detached) for an observation by walking ancestor prefixes longest-first against the detached_traces map. Per-instance detached fan-out Traces are keyed by prefix + (str(fan_out_index),) and checked first. _sync_subgraph_observations is the synthesis driver: opens any ancestor dispatch observation the leaf event needs, closes any whose subtree we've left. Called before opening the leaf so the parent resolver sees them. Detached fan-out instance roots are exempted from cursor-move close — they close with the fan-out node's own completed event. Asymmetry documented in the helper docstrings: detached subgraphs synthesize a second "link" observation in the main Trace (subgraphs have no per-subgraph node event of their own, so there's no pre-existing observation to attach the link metadata to). Detached fan-out instances accumulate the link metadata on the fan-out node's pre-existing leaf observation instead. Four new unit tests cover the synthesis paths (no Langfuse spec fixtures exist for these in v0.23.0; spec considering a follow-on proposal to add them before v0.10.0 release): - subgraph dispatch parents inner-node observations - non-detached fan-out per-instance dispatches parent worker nodes - detached subgraph splits into two Traces with the metadata link - detached fan-out per-instance Traces with accumulating detached_child_trace_ids array on the fan-out node All 3 existing conformance fixtures (022-024) still pass; no regression on linear/LLM/prompt cases.
There was a problem hiding this comment.
Pull request overview
Extends LangfuseObserver to synthesize “dispatch” span observations for subgraphs and fan-out instances (including detached-trace mode) so observation parenting and trace selection match the observability spec’s composition rules.
Changes:
- Add per-invocation bookkeeping for synthetic dispatch observations, per-instance fan-out dispatches, and detached trace routing.
- Update parent/trace resolution to prefer per-instance dispatch, then subgraph dispatch, then leaf ancestors.
- Add unit tests covering subgraph parenting, fan-out per-instance dispatch, and detached subgraph/fan-out trace behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
src/openarmature/observability/langfuse/observer.py |
Implements synthetic dispatch observations, detached-trace creation/routing, and updated parent resolution logic. |
tests/unit/test_observability_langfuse.py |
Adds unit tests validating dispatch synthesis and detached-trace behavior for subgraphs and fan-out instances. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Five real catches from the PR #81 review, all behavioral or correctness fixes — no spec-touching changes. 1. Detached-subgraph link observation in main Trace was opened but never ended. Capture the handle returned by client.span(...) and call .end() immediately — the link observation is intentionally zero-duration metadata-only, mirroring the OTel observer's checkpoint-event synthetic-span pattern. 2. Synthetic subgraph dispatch / fan-out per-instance dispatch observations only close on namespace-cursor moves. A subgraph at the tail of an invocation never gets its close trigger fired. Added close_invocation(invocation_id) and shutdown() drain methods mirroring the OTel observer's lifecycle. close_invocation walks per-invocation state in child→parent order (LLM observations → leaf nodes sorted deepest-first → per-instance fan-out dispatches → subgraph dispatches), ending each. shutdown() iterates every in-flight invocation_id and calls close_invocation. Idempotent. 3. detached_child_trace_ids side-cache was never cleared on fan-out node completion. Cyclic graphs re-entering the same fan-out would accumulate prior-iteration trace ids into the next iteration's list, overwriting the link metadata. Pop the entry in _handle_completed's fan-out-completion branch alongside the existing fan_out_parent_node_name pop. 4. _resolve_llm_parent_observation_id docstring claimed a leaf-ancestor walk fallback that the impl doesn't have. The actual precedence (exact-leaf > per-instance fan-out dispatch > subgraph dispatch longest-prefix-first > None) is correct because the dispatch fallbacks cover the wrapped-call cases an ancestor walk would have caught. Docstring updated to describe the real chain. 5. cast("list[str]", link_ids) used the string-form unnecessarily. With from __future__ import annotations on, the type expression form cast(list[str], link_ids) is the right shape and survives strict pyright settings. New unit test verifies the close_invocation drain path: a graph whose last subtree is a subgraph leaves the dispatch observation in-flight after invoke + drain; shutdown() ends it cleanly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends
LangfuseObserverwith synthetic dispatch observations matching the OTel observer's structure. Spec §8.3 mandates Span observations for subgraph wrappers, fan-out nodes, and per-instance fan-out spans; §8.5 covers detached-trace mode (each detached subgraph or fan-out instance gets its own Langfuse Trace; parent's dispatch observation surfacesmetadata.detached_child_trace_ids).PR 3.5 of 6 in the v0.10.0 batch. PR 3 shipped the basic linear-graph + LLM + prompt-linkage observer; this PR completes spec §8 coverage on composition.
Data structure additions to
_InvState:subgraph_observations: dict[prefix, _OpenObservation]— synthetic dispatch Span observationsfan_out_instance_observations: dict[prefix + (str(idx),), _OpenObservation]— per-instance dispatchdetached_traces: dict[prefix, str]— prefix → detached trace_id mappingfan_out_instance_root_prefixes: set[prefix]— detached fan-out instance bookkeepingfan_out_parent_node_name: dict[prefix, str]— cache populated fromfan_out_configon the started eventdetached_child_trace_ids: dict[prefix, list[str]]— side-cache accumulator for the link-ids array (the Protocol doesn't expose a metadata read accessor)Constructor kwargs:
detached_subgraphs: frozenset[str]— subgraph wrapper names that mint their own Tracedetached_fan_outs: frozenset[str]— fan-out node names whose instances each mint their own TraceResolver rewrites:
_resolve_parent_observation_idprecedence: per-instance fan-out dispatch > subgraph dispatch (longest-prefix-first) > leaf-node ancestor walk > Trace root_resolve_llm_parent_observation_idextended with the same precedence so LLM calls from inside a subgraph or fan-out parent correctly_trace_id_forpicks the right Trace (main or detached) for any observationAsymmetry note (documented in helper docstrings):
prefix[-1]: one in the main Trace (link withmetadata.detached_child_trace_ids, empty subtree) and one in the detached Trace (real dispatch with the subgraph subtree under it). Subgraphs have no per-subgraph node event, so there's no pre-existing observation to attach the link metadata to.startedevent) — its metadata accumulatesdetached_child_trace_idsacross all instances. Each detached Trace then gets a per-instance dispatch + the inner-node subtree.Tests: Four new unit tests cover the synthesis paths. No Langfuse-side spec fixtures exist for fan-out / subgraph / detached today; spec considering a follow-on proposal to add them before v0.10.0 release (see coord thread). All 3 existing conformance fixtures (022-024) still pass; no regression on linear / LLM / prompt cases.
Test plan