Skip to content

Commit ae3cab7

Browse files
authored
Fix NoSuchKey errors (#1037)
1 parent 574c1e7 commit ae3cab7

File tree

1 file changed

+3
-1
lines changed
  • quixstreams/sources/community/file

1 file changed

+3
-1
lines changed

quixstreams/sources/community/file/s3.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,9 @@ def get_file_list(self, filepath: Union[str, Path]) -> Iterable[Path]:
144144
yield from self.get_file_list(folder["Prefix"])
145145

146146
for file in resp.get("Contents", []):
147-
yield Path(file["Key"])
147+
# Skip folder-like objects (0-byte objects that represent S3 folder prefixes)
148+
if file["Size"] > 0:
149+
yield Path(file["Key"])
148150

149151
def read_file(self, filepath: Path) -> BytesIO:
150152
data = self._client.get_object(Bucket=self._bucket, Key=str(filepath))[

0 commit comments

Comments
 (0)