Skip to content

[BUG] Outputs bigger than 15G are not being saved #700

@JakubKC

Description

@JakubKC

Describe the bug
Outputs bigger than 15G are not being saved by the s3 plugin on the s3 bucket (they just don't show there after) despite that it says: "upload complete; uploaded xxx bytes of data" and ends with 0 exit code.

To Reproduce
Steps to reproduce the behavior:

  1. Create simple pipe in shell:
pv --stop-at-size -s 15G /dev/zero | \
/shield/plugins/s3 store -e "${SHIELD_STORE_ENDPOINT}"
  1. Run it and verify that it ends successfully, but there is no such file on s3 after.
  2. Change size to for example "-s 14990M" and run again. Verify that now there is a file.

Expected behavior
Size should be limited only by s3 limits:
https://docs.aws.amazon.com/AmazonS3/latest/dev/qfacts.html

Screenshots
If applicable, add screenshots to help explain your problem.

SHIELD versions (please complete the following information):

  • SHIELD 8.7.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    triageIssues that are pending triage / validation by a SHIELD contributor

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions