[FLINK-34467] add lineage integration for jdbc connector #149
Conversation
HuangZhenQiu
force-pushed
the
FLINK-34467-jdbc-lineage
branch
7 times, most recently
from
f1f5587 to
d468795
Compare
HuangZhenQiu
force-pushed
the
FLINK-34467-jdbc-lineage
branch
from
d468795 to
231ad7d
Compare
HuangZhenQiu
force-pushed
the
FLINK-34467-jdbc-lineage
branch
from
231ad7d to
17c6ad8
Compare
|
@HuangZhenQiu I notice your comment "The PR is not ready for review.", I wonder if you could turn this PR into a draft until it is ready for review - thanks. |
HuangZhenQiu
force-pushed
the
FLINK-34467-jdbc-lineage
branch
from
17c6ad8 to
4719928
Compare
HuangZhenQiu
force-pushed
the
FLINK-34467-jdbc-lineage
branch
from
4719928 to
2d0ba42
Compare
|
|
||
| /** Default implementation of {@link TypeDatasetFacet}. */ | ||
| @PublicEvolving | ||
| public class DefaultTypeDatasetFacet implements TypeDatasetFacet { |
There was a problem hiding this comment.
At some point this should be moved to flink-core but I understand why this hasn't been done prior to this PR.
| public class Db2LocationExtractor implements JdbcLocationExtractor { | ||
|
|
||
| private JdbcLocationExtractor delegate() { | ||
| return new OverrideJdbcLocationExtractor("db2"); |
There was a problem hiding this comment.
wouldn't it be better to have delegate as class member instead of creating it each call?
| * limitations under the License. | ||
| */ | ||
|
|
||
| package org.apache.flink.connector.jdbc.cratedb.database.lineage; |
There was a problem hiding this comment.
Wouldn't it make more sense to implement those classes within openlineage-sql-java package?
It's kind of a namespace convention thing, and implementing this a generic way, which is reusable across the integrations, can be a better solution. what do you think?
| return nameOf(simpleJdbcQueryStatement.query()); | ||
| } | ||
|
|
||
| public static Optional<String> nameOf(String query) { |
There was a problem hiding this comment.
Please specify what name is this. Dataset name, job name?
HuangZhenQiu
force-pushed
the
FLINK-34467-jdbc-lineage
branch
from
2d0ba42 to
032eab5
Compare
@HuangZhenQiu It would be worth changing this text in the PR, as I think the PR is ready for review. |
Add native lineage support for JDBC connector. It is an ongoing effort. The PR is not ready for review.