How to zip a folder which has child folders and files?

[heindrickdumdum0217]

I can't find how to zip a folder which has child folders and files in the documentation.

[michalc]

That's right it's not in the docs. But it's possible. Using a variant of https://stackoverflow.com/a/68817065/1319998:

import os
from datetime import datetime
from pathlib import Path
from stream_zip import ZIP_AUTO, stream_zip

import os
from datetime import datetime
from pathlib import Path
from stream_zip import ZIP_AUTO, stream_zip

def stream_zip_directory(directory_path):
    # Recursively zips all the files and directories in directory_path, exposing the filesystem
    # mode and modification time of each. If this is not acceptable for your use case, modify the
    # code below
    #
    # It uses ZIP_AUTO mode, and so only makes Zip64 archives if it has to based on the size of
    # the file and other properties like how many files are in the ZIP so far. This does mean
    # that the behaviour of this will be inconsistent on clients without Zip64 support - sometimes
    # it will work, and sometimes not. If this is not acceptable for your use case, modify the
    # code below.

    directory = Path(directory_path)
    if not directory.exists():
        raise Exception(f'{directory_path} does not exist')
    if not directory.is_dir():
        raise Exception(f'{directory_path} is not a directory')

    def contents(path):
        with open(path, 'rb') as f:
            while chunk := f.read(65536):
                yield chunk

    member_files = (
        (
            str(entry.relative_to(directory)) + ('/' if entry.is_dir() else ''),
            datetime.utcfromtimestamp(entry.stat().st_mtime),
            entry.stat().st_mode,
            ZIP_AUTO(entry.stat().st_size),
            () if entry.is_dir() else contents(entry)
        )
        for entry in directory.rglob("*")
    )
    return stream_zip(member_files)

So then for example to get an iterable of the bytes of a ZIP of all the files of the directory my-dir in the current working directory:

zipped_chunks = stream_zip_directory('./my-dir')

Which you can then for example save to a file:

with open('out.zip', 'wb') as f:
    for chunk in zipped_chunks:
        f.write(chunk)

[heindrickdumdum0217]

Perfect, thank you @michalc

This is another question.
When we open file and write chunks, how many memory does it consume?

[michalc]

If I’m understanding, it would be roughly proportional to the chunk_size parameter of stream_zip, that defaults to 64k. And more specifically I would expect memory usage to be a low multiple of that value. You might have to measure it if you need more precise values.

Briefly mentioned at https://stream-zip.docs.trade.gov.uk/advanced-usage/, albeit without mentioning memory directly.

[michalc]

Ah also: note that there is a small bit of memory allocated per member file that isn’t released until the end of the zip file has been output.