Skip to content

Implement Pipeline IPC and Remote Execution Routing #9453

@tobiu

Description

@tobiu

Implement Pipeline IPC and Remote Execution Routing

Goal

Enable the App Worker's Pipeline instance to seamlessly execute its counterpart inside the Data Worker.

Context

In issue #9451, we created Neo.data.Pipeline which handles instantiating a mirror of itself in the Data Worker if workerExecution: 'data' is set. However, calling pipeline.read() currently does nothing remotely. We need to implement the IPC messaging that triggers the remote execution.

Acceptance Criteria

  • Implement read(), create(), update(), and destroy() methods on Neo.data.Pipeline.
  • When invoked in the App Worker with workerExecution: 'data', these methods must send an IPC message (e.g., { action: 'pipeline.execute', id: this.remoteId, operation: 'read', params: ... }) to the Data Worker.
  • Add an onPipelineExecute message handler in src/worker/Data.mjs.
  • This handler must look up the correct Pipeline instance from this.instances, invoke the requested operation (which will run the local Connection, Parser, and Normalizer in the Data Worker), and then send the finalized record array back to the App Worker's Promise resolver.

Metadata

Metadata

Assignees

Labels

aiarchitectureArchitecture related issuescoreCore framework functionalityenhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions