Skip to content

Conversation

@HurSungYun
Copy link

@HurSungYun HurSungYun commented Oct 2, 2025

In b609aab (#12518), it is added to make a workaround for bug in AutoValueSchema. It seems fixed now (#12520), so I suggest to remove this function.

Feel free to close this PR if I got something wrong.

I found this code change by investigating my issue, which is #36356. When updating Dataflow job, it occurred NullPointerException because of topic is null. (Drain and re-deploy worked well, but it happens only in existing job update scenarioes)

I doubt this code change would resolve the original issue, but I believe this change might be needed in any way.

Related PRs -> #36295


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@HurSungYun HurSungYun marked this pull request as ready for review October 2, 2025 08:25
@github-actions
Copy link
Contributor

github-actions bot commented Oct 2, 2025

Assigning reviewers:

R: @kennknowles for label java.
R: @Dippatel98 for label kafka.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@sjvanrossum
Copy link
Contributor

Note for reviewers: I can't recall if these objects get serialized into the pipeline graph, but if that's the case then we can't remove this static method because the auto-generated serialVersionUID would change (see https://docs.oracle.com/javase/8/docs/platform/serialization/spec/class.html#a4100) and break update compatibility. 🙃

@kennknowles
Copy link
Member

I think they may be both serialized to the graph and shipped in Kafka SDF using SerializableCoder (haven't checked). So, yes, this could be an issue.

@HurSungYun
Copy link
Author

HurSungYun commented Oct 13, 2025

@kennknowles So, could you please tell me what should I do now?

If this change breaks something, it would be better to just comment not to change this due to backward-compatibility, and just keep use KafkaSourceDescriptor.create function.

This PR might mitigate potential issues in any way.

Please feel free to close this PR if needed.

@kennknowles
Copy link
Member

Yea it is pretty hard to test. You would have to start a pipeline with the released version of the SDK and then use this version of the SDK to update. Right now we don't have good automation around it in the Beam project.

It is OK to break update compatibility but we have to put in a flag like here: https://github.com/apache/beam/pull/28853/files#diff-b8cf6c3051a36c566f2f28f525449f456a88b05b3b4c17c814e6a55ba2ce36e9R77-R90

@github-actions
Copy link
Contributor

Reminder, please take a look at this pr: @kennknowles @Dippatel98

@github-actions
Copy link
Contributor

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java.
R: @johnjcasey for label kafka.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

@github-actions
Copy link
Contributor

github-actions bot commented Nov 1, 2025

Reminder, please take a look at this pr: @Abacn @johnjcasey

@github-actions
Copy link
Contributor

github-actions bot commented Nov 5, 2025

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @chamikaramj for label java.
R: @Abacn for label kafka.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

@github-actions
Copy link
Contributor

Reminder, please take a look at this pr: @chamikaramj @Abacn

@github-actions
Copy link
Contributor

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.
R: @johnjcasey for label kafka.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

@github-actions
Copy link
Contributor

Reminder, please take a look at this pr: @kennknowles @johnjcasey

@github-actions
Copy link
Contributor

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @ahmedabu98 for label java.
R: @chamikaramj for label kafka.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

@github-actions
Copy link
Contributor

github-actions bot commented Dec 3, 2025

Reminder, please take a look at this pr: @ahmedabu98 @chamikaramj

@github-actions
Copy link
Contributor

github-actions bot commented Dec 8, 2025

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java.
R: @johnjcasey for label kafka.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

@github-actions
Copy link
Contributor

Reminder, please take a look at this pr: @Abacn @johnjcasey

@github-actions
Copy link
Contributor

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.
R: @damccorm for label kafka.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

@damccorm
Copy link
Contributor

R: @kennknowles to silence the bot

I generally agree with the comments above though - I don't think this is update-safe in its current form; we should either do nothing (which is probably fine here) or follow Kenn's comments here - #36357 (comment) (this may add more complexity than it is worth though)

@github-actions
Copy link
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

@HurSungYun
Copy link
Author

I agree with all the comments above, so please feel free to close this PR if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants