Skip to content

[Bug]: Assistant_code trajectory step logs raw LLM content instead of extracted code #586

Description

@sami-marreed

What happened

When code is extracted from an assistant response, Assistant_code is written with the full raw LLM content instead of the extracted code. That makes Assistant_code a duplicate of Raw_Assistant_Response.

Fix requirement: keep both steps — do not collapse to one.

  • Raw_Assistant_Response → full raw LLM content (unchanged purpose)
  • Assistant_code → extracted code only (when code was extracted)
  • Assistant_nl → raw content when there is no code (existing branch)

Validated on origin/main @ 2538f1c2.

How to reproduce

  1. Run a CugaLite turn where the model returns prose + a python fence.
  2. Inspect trajectory steps.

Example model output:

Sure, here is the code:
```python
print(await find_tools("orders"))

Code path:

```python
self._tracker.collect_step(Step(name="Raw_Assistant_Response", data=content))
if code:
    self._tracker.collect_step(Step(name="Assistant_code", data=content))  # same content!

Expected for a code turn:

{"name": "Raw_Assistant_Response", "data": "Sure, here is the code:\n```python\nprint(await find_tools(\"orders\"))\n```"},
{"name": "Assistant_code", "data": "print(await find_tools(\"orders\"))"}

Actual: both steps have the same raw content string.

Non-goal: removing Raw_Assistant_Response or only logging extracted code.

Environment

  • OS: n/a (code-path bug)
  • CUGA: origin/main @ 2538f1c2
  • Component: CugaLite graph adapter / activity tracker

Logs, screenshots, or config

src/cuga/backend/cuga_graph/nodes/cuga_lite/adapter/graph_adapter.pyon_response_processed

Checklist

  • I searched existing issues and this doesn’t look like a duplicate

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingcomplexity: lowSmall, focused change — easy to reviewcomponent: agentCore agent loop, DynamicAgentGraph, LLM node, tool execution, CugaLitecomponent: evaluationBenchmarks and evaluationneeds-triageNewly filed, not yet reviewedpriority: lowNice to have, no urgency

Type

No type

Projects

Status
Backlog

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions