Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Failed to reset visibility timeout due to message not existing #3861

Open
chenqi0805 opened this issue Dec 19, 2023 · 0 comments
Open
Assignees
Labels
bug Something isn't working

Comments

@chenqi0805
Copy link
Collaborator

chenqi0805 commented Dec 19, 2023

Describe the bug
Without any separate process deleting the sqs message, users found data prepper could still encounters the following error:

2023-12-15T15:42:44.320 [acknowledgement-callback-2] ERROR org.opensearch.dataprepper.plugins.source.s3.SqsWorker - Failed to set visibility timeout for message 5df184d5-a97e-4031-af20-ab91800adbcc to 30
software.amazon.awssdk.services.sqs.model.SqsException: Value xxx for parameter ReceiptHandle is invalid. Reason: Message does not exist or is not available for visibility timeout change. (Service: Sqs, Status Code: 400, Request ID: dd3f6a59-96f7-54d3-94fe-37afe0c3a4b6)
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125) ~[sdk-core-2.21.23.jar:?]
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:82) ~[sdk-core-2.21.23.jar:?]
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60) ~[sdk-core-2.21.23.jar:?]
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:41) ~[sdk-core-2.21.23.jar:?]
	at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40) ~[sdk-core-2.21.23.jar:?]
....

There could be race condition between acknowledgement and change message visibility

To Reproduce
Do not have the exact input files but the s3 source configuration prototype is as follows:

source:
    s3:
      notification_type: "sqs"
      buffer_timeout: "60s"
      codec:
        newline: null
      sqs:
        queue_url: ...
        visibility_timeout: "60s"
        visibility_duplication_protection: true
      acknowledgments: true
      compression: "gzip"

Expected behavior
The error message should not show up.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS: [e.g. Ubuntu 20.04 LTS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Development

No branches or pull requests

3 participants