-
Notifications
You must be signed in to change notification settings - Fork 337
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge?
As discussed here: #1650 (comment)
We should have a TaskWriter that can write either partitioned or unpartitioned data.
Describe the solution you'd like
- TaskWriter should be able to write both partitioned and unpartitioned data
- TaskWriter should be generic, and the implementations should be engine-specific
- For partitioned data, TaskWriter should split the incoming record batch by partition, and assign the (partition_key, splitted batch) pairs to
PartitioningWriter
Willingness to contribute
I can contribute to this feature independently
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request