Is this a new feature, an improvement, or a change to existing functionality?
New Feature
How would you describe the priority of this feature request
Significant improvement
Please provide a clear description of problem this feature solves
Some users have custom ingestion jobs for pulling chunks from JIRA, remote document stores, and web services of many kinds.
Instead of making a custom pipeline for those, it'd be convenient for NRL to make it super easy to ingest lists of text chunks. It should support go through the default text splitting, embedding, and writing to VDB tasks.
Describe the feature, and optionally a solution or implementation and any alternatives
Something like:
def get_remote_data():
return ["some chunk", "another chunk"]
ingestor.chunks(get_remote_data())
.embed()
.vdb_upload()
Additional context
No response
Is this a new feature, an improvement, or a change to existing functionality?
New Feature
How would you describe the priority of this feature request
Significant improvement
Please provide a clear description of problem this feature solves
Some users have custom ingestion jobs for pulling chunks from JIRA, remote document stores, and web services of many kinds.
Instead of making a custom pipeline for those, it'd be convenient for NRL to make it super easy to ingest lists of text chunks. It should support go through the default text splitting, embedding, and writing to VDB tasks.
Describe the feature, and optionally a solution or implementation and any alternatives
Something like:
Additional context
No response