Skip to content

[Feature] supports a generic sinker for S3 #531

Description

@caiq1nyu

Motivations

Given that most AP databases already support ingesting data from S3 in standard formats, a generic S3 sink would be a valuable addition for TP-to-AP use cases.

Solution

such as:

  [sinker]
  db_type=s3
  sink_type=write
  format=parquet|jsonl|csv|orc
  s3_bucket=...
  s3_root_dir=...

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions