Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Examples for Debezium 2.3 #328

Draft
wants to merge 13 commits into
base: main
Choose a base branch
from
Draft

Update Examples for Debezium 2.3 #328

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
8044aef
[Apache Pulsar] Update to Debezium 2.3.2.Final & Apache Pulsar 2.11.2
Naros Aug 4, 2023
4a2d1aa
[Camel Component] Update to Camel 4.0.0-RC2 and deps
Naros Aug 4, 2023
30d8d6a
[Camel Component] Switch to Debezium 2.3.2.Final
Naros Aug 4, 2023
fb39794
[Camel Kafka Connect] Switch to Debezium 2.3
Naros Aug 4, 2023
1029b80
[CloudEvents] Fix image to work with Quarkus 2.x+
Naros Aug 4, 2023
2021bcf
[CloudEvents] Switch to Debezium 2.3.2.Final, update deps
Naros Aug 4, 2023
467ce9f
[Debezium Server Mongo PubSub] Update to Debezium 2.3
Naros Aug 4, 2023
8901c49
[Debezium Server Name Mapper] Switch to Debezium 2.3.2.Final, update …
Naros Aug 4, 2023
cc121c1
[Debezium Server Name Mapper] Switch to Smallrye Jandex 3.1.2 plugin
Naros Aug 4, 2023
dd4c61a
[Debezium Server Postgres PubSub] Update to Debezium 2.3
Naros Aug 4, 2023
69741ae
[Distributed Caching] Fix build for Windows
Naros Aug 4, 2023
a424628
[Distributed Caching] Fix missing JDK timezone data
Naros Aug 4, 2023
b22fdf5
[Distributed Caching] Switch to Debezium 2.3.2.Final; update deps
Naros Aug 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions apache-pulsar/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,19 @@ which comes with [support for Debezium's connectors](https://pulsar.apache.org/d
* Start up a single Pulsar node via Docker:

```
docker run -it -p 6650:6650 -p 8080:8080 --rm --name pulsar apachepulsar/pulsar:2.11.0 bin/pulsar standalone
docker run -it -p 6650:6650 -p 8080:8080 --rm --name pulsar apachepulsar/pulsar:2.11.2 bin/pulsar standalone
```

* Start up Postgres via Docker, with the example database used in the Debezium tutorial:

```
docker run -it --rm --name postgres -p 5432:5432 -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres quay.io/debezium/example-postgres:2.1
docker run -it --rm --name postgres -p 5432:5432 -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres quay.io/debezium/example-postgres:2.3
```

* Launch psql to run some SQL queries:

```
docker run -it --rm -e PGOPTIONS="--search_path=inventory" -e PGPASSWORD=postgres --link postgres:postgres quay.io/debezium/example-postgres:2.1 psql -h postgres -U postgres
docker run -it --rm -e PGOPTIONS="--search_path=inventory" -e PGPASSWORD=postgres --link postgres:postgres quay.io/debezium/example-postgres:2.3 psql -h postgres -U postgres
```

## Building the Source Code and Running Debezium Embedded
Expand Down
4 changes: 2 additions & 2 deletions apache-pulsar/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<version.debezium>2.1.3.Final</version.debezium>
<version.debezium>2.3.2.Final</version.debezium>
<version.slf4j>1.7.36</version.slf4j>
<pulsar.version>2.11.0</pulsar.version>
<pulsar.version>2.11.2</pulsar.version>
</properties>

<build>
Expand Down
4 changes: 4 additions & 0 deletions camel-component/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
The integration framework [Apache Camel](https://camel.apache.org/) provides a set of components integrating Debezium ([Postgres](https://camel.apache.org/components/latest/debezium-postgres-component.html) and other connectors) with Apache Camel.
This example demonstrates a sample application (A Q&A site, inspired by StackOverflow) and an integration pipeline capturing changes in the database generated by the application.

## Requirements

* This project uses Camel 4, which requires Java 17+.

## Topology

The example consists of multiple components:
Expand Down
16 changes: 8 additions & 8 deletions camel-component/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,16 @@

<mysql.init.timeout>30000</mysql.init.timeout>

<version.camel>4.0.0-M3</version.camel>
<version.debezium>2.3.0.Final</version.debezium>
<version.camel>4.0.0-RC2</version.camel>
<version.debezium>2.3.2.Final</version.debezium>
<version.debezium.tag>2.3</version.debezium.tag>
<version.postgresql.driver>42.5.1</version.postgresql.driver>
<version.postgresql.driver>42.6.0</version.postgresql.driver>
<version.org.slf4j>1.7.36</version.org.slf4j>
<version.quarkus>3.0.0.Final</version.quarkus>
<version.jackson>2.13.4</version.jackson>
<version.kafka>3.4.0</version.kafka>
<version.plugin.docker>0.31.0</version.plugin.docker>
<version.plugin.shade>3.2.1</version.plugin.shade>
<version.quarkus>3.2.3.Final</version.quarkus>
<version.jackson>2.13.5</version.jackson>
<version.kafka>3.5.0</version.kafka>
<version.plugin.docker>0.40.2</version.plugin.docker>
<version.plugin.shade>3.5.0</version.plugin.shade>
</properties>

<modules>
Expand Down
2 changes: 1 addition & 1 deletion camel-kafka-connect/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM quay.io/debezium/connect:2.1
FROM quay.io/debezium/connect:2.3

ARG CAMEL_KAFKA_CONNECT_VERSION=0.1.0

Expand Down
2 changes: 1 addition & 1 deletion camel-kafka-connect/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The example consists of multiple components:
Build the CoAP server and start the example by running:

```
$ export DEBEZIUM_VERSION=2.1
$ export DEBEZIUM_VERSION=2.3
$ mvn clean package
$ docker-compose up --build
```
Expand Down
2 changes: 1 addition & 1 deletion cloudevents/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This demo automatically deploys the topology of services as defined in the [Debe
## Preparations

```shell
export DEBEZIUM_VERSION=1.9
export DEBEZIUM_VERSION=2.3
mvn clean install -f avro-data-extractor/pom.xml
docker-compose up --build
```
Expand Down
8 changes: 5 additions & 3 deletions cloudevents/avro-data-extractor/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
FROM fabric8/java-jboss-openjdk8-jdk
FROM fabric8/java-centos-openjdk11-jdk
ENV JAVA_OPTIONS='-Dquarkus.http.host=0.0.0.0 -Dquarkus.http.port=${PORT}'
COPY target/lib/* /deployments/lib/
COPY target/*-runner.jar /deployments/app.jar
COPY target/quarkus-app/lib/ /deployments/lib/
COPY target/quarkus-app/*.jar /deployments/
COPY target/quarkus-app/app/ /deployments/app/
COPY target/quarkus-app/quarkus/ /deployments/quarkus/
ENTRYPOINT [ "/deployments/run-java.sh" ]
253 changes: 123 additions & 130 deletions cloudevents/avro-data-extractor/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,136 +1,129 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>io.debezium.examples</groupId>
<artifactId>cloudevents-data-extractor</artifactId>
<version>0.1-SNAPSHOT</version>
<packaging>jar</packaging>

<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<failOnMissingWebXml>false</failOnMissingWebXml>

<mvn.compiler.version>3.8.1</mvn.compiler.version>
<mvn.dependency.version>3.1.1</mvn.dependency.version>
<version.surefire>3.0.0-M6</version.surefire>

<apache.kafka.version>3.2.0</apache.kafka.version>
<version.quarkus>2.7.2.Final</version.quarkus>
<version.debezium>1.9.5.Final</version.debezium>
</properties>

<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-bom</artifactId>
<version>${version.quarkus}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>io.debezium</groupId>
<artifactId>debezium-core</artifactId>
<version>${version.debezium}</version>
</dependency>
</dependencies>
</dependencyManagement>

<build>
<finalName>aggregator</finalName>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>io.debezium.examples</groupId>
<artifactId>cloudevents-data-extractor</artifactId>
<version>0.1-SNAPSHOT</version>
<properties>
<apache.kafka.version>3.5.0</apache.kafka.version>
<failOnMissingWebXml>false</failOnMissingWebXml>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<mvn.compiler.version>3.8.1</mvn.compiler.version>
<mvn.dependency.version>3.1.1</mvn.dependency.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<version.debezium>2.3.2.Final</version.debezium>
<version.quarkus>3.2.3.Final</version.quarkus>
<version.surefire>3.0.0-M6</version.surefire>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-bom</artifactId>
<version>${version.quarkus}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>io.debezium</groupId>
<artifactId>debezium-core</artifactId>
<version>${version.debezium}</version>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-health</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-websockets</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kafka-streams</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
</dependency>
<dependency>
<groupId>io.debezium</groupId>
<artifactId>debezium-core</artifactId>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-streams-avro-serde</artifactId>
<version>7.4.0</version>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-confluent-registry-avro</artifactId>
</dependency>
</dependencies>
<repositories>
<repository>
<id>confluent</id>
<url>https://packages.confluent.io/maven/</url>
</repository>
</repositories>
<build>
<finalName>aggregator</finalName>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${version.surefire}</version>
<configuration>
<systemProperties>
<java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
</systemProperties>
</configuration>
</plugin>
<plugin>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-maven-plugin</artifactId>
<version>${version.quarkus}</version>
<executions>
<execution>
<goals>
<goal>build</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>native</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${version.surefire}</version>
<plugin>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-maven-plugin</artifactId>
<version>${version.quarkus}</version>
<executions>
<execution>
<goals>
<goal>native-image</goal>
</goals>
<configuration>
<systemProperties>
<java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
</systemProperties>
<enableHttpUrlHandler>true</enableHttpUrlHandler>
<autoServiceLoaderRegistration>false</autoServiceLoaderRegistration>
</configuration>
</plugin>
<plugin>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-maven-plugin</artifactId>
<version>${version.quarkus}</version>
<executions>
<execution>
<goals>
<goal>build</goal>
</goals>
</execution>
</executions>
</plugin>
</execution>
</executions>
</plugin>
</plugins>
</build>

<dependencies>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-health</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-undertow-websockets</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kafka-streams</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
</dependency>
<dependency>
<groupId>io.debezium</groupId>
<artifactId>debezium-core</artifactId>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-streams-avro-serde</artifactId>
<version>5.3.2</version>
</dependency>
</dependencies>
<profiles>
<profile>
<id>native</id>
<build>
<plugins>
<plugin>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-maven-plugin</artifactId>
<version>${version.quarkus}</version>
<executions>
<execution>
<goals>
<goal>native-image</goal>
</goals>
<configuration>
<enableHttpUrlHandler>true</enableHttpUrlHandler>
<autoServiceLoaderRegistration>false</autoServiceLoaderRegistration>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

<repositories>
<repository>
<id>confluent</id>
<url>https://packages.confluent.io/maven/</url>
</repository>
</repositories>

</build>
</profile>
</profiles>
</project>
6 changes: 3 additions & 3 deletions .../src/main/java/io/debezium/examples/cloudevents/dataextractor/StreamsPipelineManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@
import java.util.Collections;
import java.util.Map;

import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.inject.Produces;
import javax.inject.Inject;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.inject.Produces;
import jakarta.inject.Inject;

import org.apache.avro.generic.GenericRecord;
import org.apache.kafka.common.serialization.Serde;
Expand Down
2 changes: 1 addition & 1 deletion cloudevents/register-postgres-avro-avro.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"database.user": "postgres",
"database.password": "postgres",
"database.dbname" : "postgres",
"database.server.name": "dbserver3",
"topic.prefix": "dbserver3",
"slot.name":"dbserver3",
"schema.include.list": "inventory",
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
Expand Down
2 changes: 1 addition & 1 deletion cloudevents/register-postgres-json-avro.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"database.user": "postgres",
"database.password": "postgres",
"database.dbname" : "postgres",
"database.server.name": "dbserver2",
"topic.prefix": "dbserver2",
"slot.name":"dbserver2",
"schema.include.list": "inventory",
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
Expand Down
Loading