[GSoC 2026] Minimal Agentic Dora Prototype — Modular Dataflow for Real-Time Robot Control

[DEFAULTE-R]

Hi,

I’ve been exploring the "Agentic Dora" GSoC project idea and wanted to share a working prototype I built to validate the core concept of agent-driven robot control using Dora dataflows.

What I implemented

• A modular Dora dataflow with three nodes:
  input → agent_bridge → simulation
• A custom input node that streams commands into the pipeline
• An agent bridge that converts natural language commands into structured velocity commands
• A PyBullet-based simulation node executing commands in real time (with GUI support)
• End-to-end pipeline:
  command → agent → control signal → robot motion

Key aspects demonstrated

• Real-time message passing using Dora runtime
• Clear separation of concerns between nodes (input / reasoning / execution)
• Easy node replacement (e.g., agent can be swapped with an LLM without changing the pipeline)
• Working physics-based simulation

Example commands

• move forward
• turn left / right
• stop

Current focus

This prototype focuses on validating the dataflow architecture and modularity, rather than full agent intelligence.

Planned next steps

• Replace rule-based agent with an LLM (e.g., smolagents)
• Add structured command schema + validation
• Introduce safety mechanisms (timeouts, constraints)
• Support multi-step commands (e.g., "move in a square")

Questions

• Does this direction (starting from a minimal modular pipeline and incrementally adding intelligence and safety) align with expectations for this project?
• Are there specific Dora features (e.g., observability, node composition, deployment patterns) that would be valuable to demonstrate early?

Thanks!

[DEFAULTE-R]

Hi, sharing a quick update after further development on the prototype:

✅ Improvements made

• Added a dedicated safety node (agent → safety → simulation)

  • Enforces clamping and validation before execution

• Introduced structured messaging (JSON schema) across nodes:

  • velocity (linear, angular)
  • meta (source, timestamp, command_id)

• Initialized a Rust schema layer (ControlMessage) for future typed communication

• Refactored pipeline to be more aligned with Dora’s modular philosophy

🔁 Updated architecture

Input → Agent → Safety → Simulation
         ↓
   Structured messages

Observations

• Dora’s non-interactive execution required replacing input() with streaming input
• Node isolation makes it straightforward to evolve components independently
• Failure modes (e.g., simulation exit) highlight need for future fault tolerance

Next steps

• Preserve structured messages end-to-end (including metadata through safety node)
• Replace rule-based agent with LLM (e.g., smolagents)
• Add multi-step command execution
• Explore observability/debugging features in Dora

Would love feedback on whether this direction (introducing structure + safety early) aligns with expectations for the project.

🔗 Prototype Repository:
https://github.com/DEFAULTE-R/dora-agentic-demo

(Working demo with modular Dora pipeline, safety layer, and structured messaging)

Thanks!

[DEFAULTE-R]

Hi, sharing a quick update since my last message:

I have now extended the prototype into a full agentic pipeline with an LLM-based agent layer.

🔄 Updated architecture:
Input → LLM Agent → Action Mapping → Safety → Simulation

✅ Key additions:

• Integrated a local LLM (Mistral via Ollama) for natural language → action reasoning
• Switched from direct control outputs to a tool-based abstraction:
  { "action": "move_forward" } → mapped to velocity commands
• Added an explicit action-to-execution layer to decouple reasoning from control
• Maintained safety validation as an independent node

🧠 This now demonstrates:

• Agent reasoning (LLM in the loop)
• Tool-based execution pattern
• Modular design where components can be swapped independently

The system runs end-to-end in real time with the LLM integrated into the control loop.

🔗 Prototype repository:
https://github.com/DEFAULTE-R/dora-agentic-demo

Additionally, I’ve also started contributing directly to the Dora codebase:
🔗 PR: dora-rs/dora-rs.github.io#60

Would this direction (introducing LLM reasoning + tool abstraction early, along with separate safety validation) align with expectations for the project?

Thanks!

[Galaxylee319]

Great work building this minimal agentic Dora prototype so quickly — it’s really solid to see a complete, runnable end-to-end pipeline that validates the core agent-driven robot control pattern with Dora’s dataflow architecture!
To answer your questions directly:
Absolutely, this direction aligns perfectly with the project expectations.Starting from a minimal, modular pipeline to first validate the dataflow structure, real-time message passing, and clean node separation is exactly the right approach for GSoC. Incrementally layering in LLM integration, command schemas, safety guards, and complex multi-step behaviors on top of a working foundation lowers risk, keeps progress measurable, and stays true to the goal of agentic robot control with Dora. This iterative, architecture-first mindset is ideal.
These Dora features are absolutely worth highlighting early:
Observability: Dora’s built-in tracing, logging, and dataflow visualization will make it easy to show how natural-language commands flow through the agent bridge and into control signals — critical for debugging and demonstrating the pipeline clearly.
Node hot-swapping & composition: You already mentioned swapping the rule-based agent for an LLM; leaning into Dora’s native node composability will make this replacement clean and demonstratable, which is a core value of the framework.
Real-time latency & runtime guarantees: Since this is for robot control, showing Dora’s low-latency message-passing with the PyBullet simulation reinforces its fitness for real-world robotics.
Structured message schemas: Aligning with Dora’s standardized data formats early will simplify future extensions (e.g., adding perception nodes, collision checking, or multi-robot coordination).
Small extra suggestion: Keeping the rule-based agent as a baseline while integrating the LLM will make the improvement easy to compare, and adding basic safety constraints early will help frame the project as practical robotics rather than just a demo.
Excited to see how this evolves — excellent start!

[DEFAULTE-R]

Hi,
Thanks for the detailed feedback.

I’ll incorporate your suggestions by:
eeping the rule-based agent as a baseline alongside the LLM agent for comparison
adding clearer observability of message flow through the pipeline
continuing to align message formats with Dora’s structured schemas

I’m also working on extending this into a more complete example and contributing additional improvements directly to the Dora repository.

Appreciate the guidance! I’ll share further updates soon.

[DEFAULTE-R]

Hi,
sharing a quick update on my progress:

I have implemented a dora-smolagents node and opened a PR to dora-hub that integrates smolagents with Dora dataflows.

🔄 Updated architecture:
Input → smolagents Agent → Tool Execution → Action Output

✅ Key additions:

• Integrated smolagents for tool-based reasoning
• Added support for local LLMs via Ollama (Mistral) using LiteLLM
• Implemented structured action outputs (e.g., {"action": "move_forward"})
• Designed the node to be modular and LLM-agnostic

This now demonstrates:

• Agent-based reasoning inside Dora pipelines
• Tool abstraction for robot actions
• Easy swapping of LLM backends (local or API-based)

🔗 PR:
dora-rs/dora-hub#54

I’d appreciate any feedback on the node design, especially regarding:

• best practices for integrating agent-based nodes in Dora
• structuring tool interfaces for extensibility

Thanks!

[DEFAULTE-R]

Hi,

quick update based on the earlier feedback:

I’ve added initial observability to the smolagents-based Dora node to make the agent decision flow explicit.

Each command now logs:

• input command
• structured output
• latency measurement

Example:
[TRACE] agent.input: move forward
[TRACE] agent.output: {"action": "move_forward"}
[TRACE] agent.latency: 0.42s

PR: dora-rs/dora-hub#55

This is a first step towards integrating structured tracing (e.g., OTLP) and improving debuggability of agent-driven pipelines.

Next, I plan to extend this with:

• more structured tracing
• better safety handling
• multi-step execution support

Would appreciate any feedback on how best to align this with Dora’s observability patterns.

Thanks!