aws s3 sync traverses excluded directory - takes up a long time

[ChrisSLT]

This has been discussed elsewhere but I wasn't sure what the response was. I wanted to see if there would be an ability to stop SYNC from traversing an excluded directory because it takes up a lot of time.

When you do a sync from / with --exclude "STAR" and --include "/Documents/STAR" for example, the sync operation still takes many minutes to complete because it walks through every single file in every directory in the user's home folder. I wish I knew enough to get into the actual code, but I'm wondering if you could stop the SYNC operation from doing this?

Thank you

[jamesls]

I agree, we should stop traversing into a directory if it's excluded/included. This is also related to #1117 which will throw warnings even if a file is excluded, which has the same root cause: we're traversing into a directory when we could be skipping it.

I don't see any reason why we shouldn't change this behavior.

[ChrisSLT]

Terrific, thank you

[ChrisSLT]

Would you be able to let us know a timeline on this?

[kyleknap]

It is difficult to put a specific timeline/date for this. Since this would fix a couple of bugs that have been reported to us, it is has a relatively high priority for us. However, implementation-wise it may take a while because it requires a good amount of refactoring on the internals of the s3 commands. Ultimately, the feature is near the top of our list of features to implement, but time has yet to be allocated to work on it. We can let you know when we do start working on this feature.

[ChrisSLT]

Thank you. Really appreciate it.

[ChrisSLT]

Can you please give an indication as to when this issue will be fixed? It creates a lot of complexity to have to have multiple scripts to get around this bug. Thank you

[ChrisSLT]

Can you tell me if this is intended to be fixed?

[ChrisSLT]

Please update us on this issue. Due to the bug, our team is getting pressure to move away from S3 sync.

[ASayre]

Good Morning!

We're closing this issue here on GitHub, as part of our migration to UserVoice for feature requests involving the AWS CLI.

This will let us get the most important features to you, by making it easier to search for and show support for the features you care the most about, without diluting the conversation with bug reports.

As a quick UserVoice primer (if not already familiar): after an idea is posted, people can vote on the ideas, and the product team will be responding directly to the most popular suggestions.

We’ve imported existing feature requests from GitHub - Search for this issue there!

And don't worry, this issue will still exist on GitHub for posterity's sake. As it’s a text-only import of the original post into UserVoice, we’ll still be keeping in mind the comments and discussion that already exist here on the GitHub issue.

GitHub will remain the channel for reporting bugs.

Once again, this issue can now be found by searching for the title on: https://aws.uservoice.com/forums/598381-aws-command-line-interface

-The AWS SDKs & Tools Team