Improve single node execution performance for C# SDK

[heavy-d]

What the backend actually does today (why “tiny nodes” still cost ~80–200ms)

Even when you execute a single trivial node (concat/add), the worker goes through the full job infrastructure:

• WS receive + decode + Pydantic validation (small but non-zero)
• Create ProcessingContext
• Create persistent run state in the DB (multiple DB writes)
• Start a new job execution
• • In threaded mode, it still creates a new dedicated thread + event loop per job
• Run the workflow runner (your actual node compute; for concat/add this is basically negligible)
• Stream messages back to the client using a polling loop with a default 10ms sleep when no messages (adds jitter)

You can see the biggest recurring costs directly in code:

• Polling sleep jitter (up to ~10ms per “wait for next message”):

unified_websocket_runner.pyLines 216-237
async def process_workflow_messages(    job_execution: JobExecution,    sleep_interval: float = 0.01,    explicit_types: bool = False,) -> AsyncGenerator[dict[str, Any], None]:    ...    while job_execution.is_running():        if job_execution.context.has_messages():            async for msg in process_message(job_execution.context, explicit_types):                yield msg        else:            await asyncio.sleep(sleep_interval)

• DB writes on every run (RunState.create_run, then mark “running”):

job_execution_manager.pyLines 121-156
# 1. Create RunState record (DB source of truth)async with ResourceScope():    run_state = await RunState.create_run(        run_id=run_id,        execution_strategy=request.execution_strategy.value,        worker_id=worker_id,    )    run_state.execution_id = execution_id    await run_state.save()...# Update run state to runningasync with ResourceScope():    run_state = await RunState.get(run_id)    run_state.status = "running"    await run_state.save()

• Thread/event-loop creation per job (thread start cost + OS scheduling variance):

threaded_job_execution.pyLines 210-234
runner = WorkflowRunner(job_id=job_id)# Create persistent event loop for this jobevent_loop = ThreadedEventLoop()event_loop.start()# Create the job record in databasejob_model = Job(...)async with ResourceScope():    await job_model.save()

Why 0.08s sometimes, 0.2s other times?

That variance is exactly what you’d expect from:

• SQLite/file-backed DB commits + ResourceScope acquisition (can vary with disk/cache)
• Thread creation/scheduling (can vary with OS load)
• 10ms polling interval jitter in the message streamer
• Occasional GC / logging overhead / background tasks

Theoretical improvements (future ideas) + “ideal” latency floors
If you keep the same architecture (WS + job records + per-job threads), you won’t get “1ms” runs—most time isn’t the node, it’s the framework around it.

What would move the needle most:

• Fast path for ephemeral node runs (skip persistence)
  Avoid creating RunState/Job rows for “fire-and-forget local runs”. This is likely the single biggest latency win.
• Reuse execution infrastructure
  Instead of one thread+event-loop per job, use a shared event loop / pool for lightweight jobs.
• Event-driven message streaming
  Replace polling (sleep_interval=0.01) with “await next message” semantics → reduces the ~0–10ms jitter (and more importantly removes extra wakeups).

Rough “best case” numbers (order-of-magnitude)

• Ideal architecture (in-process direct call, no DB, no per-job thread, no polling): ~2–15ms overhead + node compute
• Still WS but optimized (no DB + shared runner + event-driven streaming): ~10–40ms typical
• Keep DB persistence and per-job thread creation: hard to get below ~50–150ms typical on older hardware, and you’ll still see spikes

[heavy-d]

What needs to change for build-time typing

• Workflows: stop returning output_schema.type="any"
  Backend (preferred) should generate output_schema by walking the graph: for each nodetool.output.*, follow its incoming edge and emit the upstream output’s type (not any).
  Alternatively (client-side), the SDK can infer output types from workflow.graph + /api/nodes/metadata, but the API should ideally just provide it.
• Expose type identity for enums (not just values)
  Ensure metadata includes a stable enum identifier (e.g. type_name) and clients preserve it so C# can bind to a generated enum type instead of string.
• Add a type-binding layer in the C# SDK
  Use TypeMetadata + schema at build time to map pins/results to generated Nodetool.Types.* CLR types, and fall back to NodeToolValue only when truly unknown/dynamic.
  That’s the minimum path from “types appear at runtime” → “types are known at build time.”

[heavy-d]

Concrete backend changes (without breaking existing clients)

Goal

Provide build-time type context for workflow inputs/outputs so SDKs can create strongly typed APIs/pins without waiting for runtime messages.

1) Add typed workflow I/O metadata to workflow responses (additive fields)

Keep existing input_schema / output_schema unchanged (backward compatibility), but add new fields:

• input_type_metadata: Dict[str, TypeMetadata] keyed by workflow input name
• output_type_metadata: Dict[str, TypeMetadata] keyed by workflow output name

Old clients ignore unknown fields; new clients can bind to Nodetool.Types.*.

2) Infer output types by walking the workflow graph

Today, workflow outputs are emitted as "type": "any" in get_output_schema (hardcoded).

Instead, for each nodetool.output.* node:

• Find its incoming edge(s) (the upstream producer)
• Determine the upstream output slot type
  • Prefer node.dynamic_outputs[sourceHandle] when present (already part of the graph model)
  • Otherwise resolve the node class and use its declared output slots to find the matching handle
• Emit that upstream TypeMetadata into output_type_metadata[output_name]
• If anything can’t be inferred safely, fall back to TypeMetadata(type="any")

This preserves today’s behavior while enabling typed clients when inference succeeds.

3) Preserve enum identity

For enums, TypeMetadata must include:

• type="enum"
• values=[...]
• type_name (stable identifier)

Without type_name, clients cannot bind to generated C# enum types at build time and must treat enums as strings.

4) Optional: add an opt-in query flag (extra safety)

If desired, gate new metadata behind a query param:

• GET /api/workflows/{id}?include_type_metadata=1

Default responses remain byte-for-byte compatible with existing clients.

• Problem: runtime updates can hint types, but SDKs need build-time types for pins/APIs
• Main fix: add input_type_metadata / output_type_metadata to workflow responses (keep schemas unchanged)
• Output typing: infer workflow output types by walking the graph (follow edges into nodetool.output.*)
• Enums: preserve TypeMetadata.type_name so C# can bind to generated enums (otherwise they stay strings)
• Handle contract: inference depends on stable sourceHandle/targetHandle ↔ output slot naming
• SDK fallback: if metadata is missing/any/dynamic, keep returning NodeToolValue (don’t break unknown cases)