Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multiple topics consuming by Kafka connector #214

Closed
BorisTyshkevich opened this issue Feb 13, 2025 · 2 comments
Closed

multiple topics consuming by Kafka connector #214

BorisTyshkevich opened this issue Feb 13, 2025 · 2 comments

Comments

@BorisTyshkevich
Copy link

For all DB connectors (mongo, mysql, postgres) Transfer can pull data from many tables. However, for Kafka it's possible to specify only 1 topic:

      Topic: "my-kafka-topic"

It would be great, if Transfer can consume from multiple topics. The very same as Clickhouse's Kafka Engine.

src:
  type: kafka
  params:
    topics:
    - topic1
    - topic2
    - topic3

Boris.

@laskoviymishka
Copy link
Contributor

That's supported, but under different field.
It's either topic or groupTopics

See docs here https://doublecloud.github.io/transfer/docs/connectors/kafka.html

@laskoviymishka
Copy link
Contributor

Docs btw slightly outdated as usual :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants