-
-
Notifications
You must be signed in to change notification settings - Fork 200
Open
Labels
aiarchitectureArchitecture related issuesArchitecture related issuescoreCore framework functionalityCore framework functionalityenhancementNew feature or requestNew feature or request
Description
Create Pipeline Cornerstone and Refactor Store Implementation
Goal
Establish Neo.data.Pipeline as the central orchestrator for data transformation and remote execution, and remove the brittle remote instantiation logic from Neo.data.Store.
Context
Currently, Neo.data.Store attempts to directly manage the cross-worker instantiation of its Normalizer (via afterSetNormalizer). This is an abstraction leak; a Store should not be hardcoded to Neo.worker.Data or manage remote IDs.
We need a dedicated Neo.data.Pipeline class. The Store will aggregate a Pipeline using ClassSystemUtil.beforeSetInstance(). The Pipeline takes over the responsibility of owning the Connection, Parser, and Normalizer, and importantly, orchestrating whether they run locally in the App Worker or remotely in the Data Worker.
Acceptance Criteria
- Create
src/data/Pipeline.mjsextendingNeo.core.Base. - Give
Pipelinethe following reactive configs:workerExecution(default'app'),connection_,parser_, andnormalizer_. - Use
ClassSystemUtil.beforeSetInstanceinside the Pipeline to instantiate these sub-components. - If
workerExecution: 'data', the Pipeline should useNeo.worker.Data.createInstanceto spawn the actual Connection, Parser, and Normalizer instances exclusively inside the Data Worker (meaning the App Worker Pipeline only holds the configs, not the instances), storing the remote ID. - Refactor
Neo.data.Store: RemoveafterSetNormalizerandafterSetParser. Introduce apipeline_config that usesClassSystemUtil.beforeSetInstanceto create the Pipeline. - The Store's
load()method should delegate tothis.pipeline.read().
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
aiarchitectureArchitecture related issuesArchitecture related issuescoreCore framework functionalityCore framework functionalityenhancementNew feature or requestNew feature or request