Motivations
Given that most AP databases already support ingesting data from S3 in standard formats, a generic S3 sink would be a valuable addition for TP-to-AP use cases.
Solution
such as:
[sinker]
db_type=s3
sink_type=write
format=parquet|jsonl|csv|orc
s3_bucket=...
s3_root_dir=...
Motivations
Given that most AP databases already support ingesting data from S3 in standard formats, a generic S3 sink would be a valuable addition for TP-to-AP use cases.
Solution
such as: