Add Android and iOS support

[hyuri]

With the recent promotion of Android and iOS to Tier 3 supported platforms in Python 3.13 [1, 2, 3], and BeeWare recently making it easier than ever to build wheels for these platforms [4], mobile platforms have become a viable avenue for Python developers and we should start to see an increasing number of Python apps targeting these platforms.

With that in mind, it would be invaluable to have stream-unzip available for these platforms, with built Android and iOS wheels.

[michalc]

Hi @hyuri,

Do you have a specific use case for this, e.g. an app you’re making in which it would be useful to stream open ZIP files? (I kinda lean to only adding things where there is reasonable evidence that they will be used at least once)

Thanks,

Michal

[hyuri]

Hi @michalc,

Yes, I'm developing a cross-platform (desktop and mobile) file conversion app that needs to access files inside a zip archive, and sometimes files inside a zip archive contained inside a zip archive — two layers of zip archives.

I can currently get a handle on regular files inside a zip archive and read them one by one since they are small, but I can't get a stream on the inner zip archive without loading the entire archive into memory — which can be gigabytes in size. I'm having to extract the inner zip archive to a temporary folder before getting a handle on the regular files inside it, but it's a messy extra step and it can get very slow, especially on mobile.

Using stream-unzip would help me avoid extracting the zip archive, and would also allow me to skip loading regular files fully into memory. Double improvement.

Thanks

[michalc]

Hi @hyuri,

Sounds pretty cool! Yeah would love to get wheels published for iOS and Android.

But… I fear it might be a project. I’ve been nosing around, and so far only found https://pypi.org/project/pyspamsum/1.0.7/#files that have wheels for iOS or Android. Have you found any more? Complications for stream-unzip are that it uses Rust and compiled Cython (via stream-inflate), and I guess I fear hitting a lot of so-far-unknown problems.

To check also - do you absolutely need wheels files for this? Can you compile things as needed as part of installing Python packages for each platform? The wheels files are essentially a convenience, and the lack of them for any particular platform doesn’t mean stream-unzip can’t be installed and used on it.

(Also - I was so close to closing this issue as “obvious” spam for BeeWare! Pleased I didn’t 😀)

Michal

[hyuri]

Hi @michalc,

(Also - I was so close to closing this issue as “obvious” spam for BeeWare! Pleased I didn’t 😀)

Oh, no I'm not part of BeeWare — thanks for not closing 😂 I've just been keeping up with their bleeding edge while developing my app.

But… I fear it might be a project. I’ve been nosing around, and so far only found https://pypi.org/project/pyspamsum/1.0.7/#files that have wheels for iOS or Android. Have you found any more?

pyspamsum might indeed be the only package with mobile wheels for now because support in PyPi has just been added this year without much fanfare; it was built by the @beeware team themselves, to demonstrate the integration. They told me they've been keeping it quiet for now and should start making noise about porting efforts within a month or two — if release cycles align.

I asked the BeeWare team and they said mobile support in cibuildwheel is ready for iOS, and in-progress for Android. Should make it significantly easier to build for these platforms.

@freakboy3742's comments, with some pointers:

For iOS, the code in cibuildwheel has landed; as have the docs for the new features. Those docs aren't published on the web yet, but you can read the text on Github https://github.com/pypa/cibuildwheel/blob/main/docs/platforms.md#ios. You'll also need to install cibuildwheel from github. The tl;dr is that for a simple project, you need to add a test-command that can invoke your test suite, and test-sources to declare the files that are part of your test suite; then run CIBW_PLATFORM=ios cibuildwheel. Beyond that - it depends on the project, and what other build requirements exist.

As for Android, cibuildwheel support is still in progress. The current best option is a Chaquopy recipe https://github.com/chaquo/chaquopy/tree/master/server/pypi; that recipe should be reasonably adaptable to what cibuildwheel will eventually need.

Perhaps it's better to wait until the work in cibuildwheel is fully released, to then assess how hard it would be? Maybe I can come back and give you a heads up.

Complications for stream-unzip are that it uses Rust and compiled Cython (via stream-inflate), and I guess I fear hitting a lot of so-far-unknown problems.

Yeah, that might cause some headaches. Cython might not be a problem, but I saw someone in the BeeWare discord chat hitting a few errors with Rust + cibuildwheel. Seems like they found fixes, but I'd have to ask.
@freakboy3742 also said:

if you want to expose rustc into the iOS build environment, you need to add xbuild-tools=['rustc'] to your pyproject.toml

---

...do you absolutely need wheels files for this? Can you compile things as needed as part of installing Python packages for each platform?

I would have to look into it; I was under the impression that they needed to be provided by the package. Ultimately, even if not mandatory, it would be very convenient indeed, especially as the user base grows, but I'll check if I can create the wheels on my end.

Thanks.

[freakboy3742]

...do you absolutely need wheels files for this? Can you compile things as needed as part of installing Python packages for each platform?

Hi - founder of BeeWare here.

A "compile sdist on install" experience might be possible in the long term - but it's not something we're targeting right now, to the extent that we actually prohibit sdist installs for iOS. sdist binary installs are already problematic in a lot of situations; there's plenty you can do to a local build environment that will render an sdist fragile - including, but not limited to, not having necessary dependencies available, or having incompatible version of build tools in the environment. The needs of a cross-platform build environment only make this situation more fragile.

However, it is possible to install from a local store of wheels, rather than PyPI. So - you could use cibuildwheel to build your own personal iOS wheel store (either a local directory, or an independent PyPI-compatible package server), and then point pip (or Briefcase) at that store.

[hyuri]

However, it is possible to install from a local store of wheels, rather than PyPI. So - you could use cibuildwheel to build your own personal iOS wheel store (either a local directory, or an independent PyPI-compatible package server), and then point pip (or Briefcase) at that store.

Ok, I'll try to figure out how to do that. Thanks.

[hyuri]

Checking cibuildwheel, it seems like it may be easier to build Rust wheels, after all,
as per https://github.com/pypa/cibuildwheel?tab=readme-ov-file#see-also:

If you are building Rust wheels, you can get by without some of the tricks required to make GLIBC work via manylinux; this is especially relevant for cross-compiling, which is easy with Rust. See maturin-action for a tool that is optimized for building Rust wheels and cross-compiling.

[michalc]

Perhaps it's better to wait until the work in cibuildwheel is fully released, to then assess how hard it would be? Maybe I can come back and give you a heads up.

So I've given this a bit of a go. stream-inflate now has ios wheels: https://pypi.org/project/stream-inflate/#files

But stream-unzip itself, not quite. #129 ends up with an error that I've not managed to figure out yet

💥 maturin failed
Caused by: Python interpreter should be a kind of interpreter (e.g. 'python3.8' or 'pypy3.9') when cross-compiling, got path to interpreter: /private/var/folders/hn/k7g0_sh57112t0xtjxcjcm5r0000gn/T/build-env-rv48otpj/bin/python

(Full error at https://github.com/uktrade/stream-unzip/actions/runs/14842743706/job/41669358994)

A "compile sdist on install" experience might be possible in the long term - but it's not something we're targeting right now, to the extent that we actually prohibit sdist installs for iOS

Ah I didn't realise it was forbidden in BeeWare

See maturin-action for a tool that is optimized for building Rust wheels and cross-compiling.

Nosing around the code of the action, nothing leaps out as being ios/iphone(/android)-y, so right now I think cibuildwheel is probably the better bet?

[freakboy3742]

Perhaps it's better to wait until the work in cibuildwheel is fully released, to then assess how hard it would be? Maybe I can come back and give you a heads up.

So I've given this a bit of a go. stream-inflate now has ios wheels: https://pypi.org/project/stream-inflate/#files

🎉 Nice work!

Glad the tools worked - if you've got any comments about the process, or any areas you found especially complex or confusing, let me know so I can put those on the list of things to document better.

But stream-unzip itself, not quite. #129 ends up with an error that I've not managed to figure out yet

💥 maturin failed
Caused by: Python interpreter should be a kind of interpreter (e.g. 'python3.8' or 'pypy3.9') when cross-compiling, got path to interpreter: /private/var/folders/hn/k7g0_sh57112t0xtjxcjcm5r0000gn/T/build-env-rv48otpj/bin/python

(Full error at https://github.com/uktrade/stream-unzip/actions/runs/14842743706/job/41669358994)

From the look of it, it's the Maturin integration causing a problem. Individual PEP 517 build systems usually need some patching to work (either in the tool itself, or in the cross-virtual environment that the tool is running in). stream-inflate is built with setuptools, which I've already adapted; I'm currently in the middle of getting meson-python ported. Maturin support is high on the priority list, but I haven't looked at it yet.

[hyuri]

So I've given this a bit of a go. stream-inflate now has ios wheels: https://pypi.org/project/stream-inflate/#files

Amazing! You're fast!

But stream-unzip itself, not quite. #129 ends up with an error that I've not managed to figure out yet

💥 maturin failed
Caused by: Python interpreter should be a kind of interpreter (e.g. 'python3.8' or 'pypy3.9') when cross-compiling, got path to interpreter: /private/var/folders/hn/k7g0_sh57112t0xtjxcjcm5r0000gn/T/build-env-rv48otpj/bin/python

(Full error at https://github.com/uktrade/stream-unzip/actions/runs/14842743706/job/41669358994)

If it helps, I was digging around and found the following answers, from messense (maturin maintainer), to that error message:

maturin pep517 build-wheel is not intended to be used outside of pip install, use maturin build instead.

:PyO3/maturin#1986 (comment)

Did you set the ARCHFLAGS env var by accident?

:PyO3/maturin#2101 (comment)

---

See maturin-action for a tool that is optimized for building Rust wheels and cross-compiling.

Nosing around the code of the action, nothing leaps out as being ios/iphone(/android)-y, so right now I think cibuildwheel is probably the better bet?

I have no experience with either, but it sounds like cibuildwheel might be the better bet.

[michalc]

Maturin support is high on the priority list, but I haven't looked at it yet.

If it helps, I was digging around and found the following answers, from messense (maturin maintainer), to that error message:

It does cross my mind a bit to strip out the Rust and go for Cython, since it would make things like this much more likely to be easily/already solved? Could even compile “all” of stream-unzip so it could give a bit of a speed boost for all uses as well (the Rust part is just used for ZipCryto decoding)

[hyuri]

@michalc I'm not experienced enough to give a recommendation on that, but my sense is that whatever causes less trouble and serves stream-unzip best.