Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Events are missing from S3 sink even with end-to-end acknowledgements #5413

Open
dlvenable opened this issue Feb 5, 2025 · 0 comments
Assignees
Labels
bug Something isn't working untriaged

Comments

@dlvenable
Copy link
Member

Describe the bug

We have a pipeline which appears to be losing events on the s3 sink. We see metrics that indicate a significant difference between s3.recordsIn.count and s3.s3SinkObjectsEventsSucceeded.count. However, I do not see any error logs.

To Reproduce

We do not yet have steps to reproduce.

Expected behavior

These metrics should align and all data should be in the S3 bucket.

Environment (please complete the following information):

Data Prepper 2.10.1

Additional context

This may be related to #5412. At the very least, that issue may be causing us to acknowledge the events even though they were not actually written.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working untriaged
Projects
Development

No branches or pull requests

1 participant