Cannot use numpy 2.0 because old thinc version is used

[scott-8]

How to reproduce the behaviour

spaCy 3.7.6 was built against numpy 2.0, but still lists thinc>=8.2.2,<8.3.0 as a dependency. thinc does support numpy 2.0 until version 8.3.0, so spaCy is unable to use it.

Your Environment

• spaCy version: 3.7.6
• Platform: Windows-11-10.0.22621-SP0
• Python version: 3.12.0
• Pipelines: en_core_web_sm (3.7.1)

[dhar174]

Same problem here. Also still installs numpy 1.26.4.

[filbranden]

thinc released 8.3.0 with a dependency on numpy 2: https://github.com/explosion/thinc/releases/tag/release-v8.3.0

I launched a local build of spaCy updating the thinc>=8.2.2,<8.3.0 references to thinc>=8.3.0,<9.0.0 (in pyproject.toml, requirements.txt and setup.cfg) and it built fine for me, import spacy no longer fails in a venv that includes numpy 2.

Can we please get this bumped in spaCy please?

(If both numpy 1 and numpy 2 compatibility are desired, is it worth considering bumping the version, similar to how thinc did it?)

[honnibal]

Version 3.7.6 should be built against numpy v2. Is it working for you? https://github.com/explosion/spaCy/releases/tag/release-v3.7.6

[filbranden]

@honnibal No 3.7.6 is actually building against numpy 1.26

See build logs for 3.7.6

For example, the before_build of the cp312-manylinux_x86_64 wheel:

  Collecting numpy>=1.19.0 (from -r requirements.txt (line 16))
    Downloading numpy-2.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
  ...
  Collecting numpy>=1.19.0 (from -r requirements.txt (line 16))
    Downloading numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
  ...
  Downloading numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.0 MB)
  ...
  Installing collected packages: ..., numpy, ...
  Successfully installed ... numpy-1.26.4 ...

Looks like at first it matches 2.1.0 but later goes back to 1.26.4.

This is because thinc 8.2.x depends on numpy <2, we need to upgrade to thinc 8.3.x which updates to numpy 2.

Would you be able to help bump the dependency on a newer thinc and possibly release a 3.7.7 which will really build against numpy 2? Thanks!

[honnibal]

You're right, something went wrong with the dependencies on that version. I'd already published a v8.3.0 for Thinc that updated numpy, I'm building a v3.7.7 that should build against the correct dependencies now.

[filbranden]

Thanks @honnibal ! Looks like build for 3.7.8 succeeded and spot checking the logs dependencies look correct, looking forward to seeing that as an official releas. Much appreciated.

[honnibal]

Should be published now. I published as v3.8, as the change in dependencies might be disruptive for some folks.

[filbranden]

Very nice, thank you so much!

[filbranden]

Does this file need an update? Right now I see 3.8.dev0 but not 3.8 itself: https://raw.githubusercontent.com/explosion/spacy-models/master/compatibility.json

[filbranden]

Looks like the models themselves might have restrictive requirements too:

$ pip check
en-core-web-lg 3.7.1 has requirement spacy<3.8.0,>=3.7.2, but you have spacy 3.8.0.
en-core-web-md 3.7.1 has requirement spacy<3.8.0,>=3.7.2, but you have spacy 3.8.0.
en-core-web-sm 3.7.1 has requirement spacy<3.8.0,>=3.7.2, but you have spacy 3.8.0.

[filbranden]

And I'm also seeing a runtime dependency on thinc 8.2, probably from here: https://github.com/explosion/spaCy/blob/v3.8.x/requirements.txt#L6

$ pip check
spacy 3.8.0 has requirement thinc<8.3.0,>=8.2.2, but you have thinc 8.3.0.

Can you fix that one as well?

[filbranden]

Actually more likely this one is the source of the install requirement for thinc 8.2: https://github.com/explosion/spaCy/blob/v3.8.x/setup.cfg#L53