Skip to content

Breaking change in dependency joblib #25

@bbrewington

Description

@bbrewington

in requirements.txt files, joblib==0.13.2 needs to change to joblib==1.1.0, and need to add Cython==0.29.30

Before making the change, got this error:

(venv) ➜  full_scraper git:(master) ✗ python op.py --help
Traceback (most recent call last):
  File "/Users/brentbrewington/project-files-github/odds-portal-scraper/full_scraper/op.py", line 8, in <module>
    from joblib import delayed
  File "/Users/brentbrewington/project-files-github/odds-portal-scraper/full_scraper/venv/lib/python3.9/site-packages/joblib/__init__.py", line 119, in <module>
    from .parallel import Parallel
  File "/Users/brentbrewington/project-files-github/odds-portal-scraper/full_scraper/venv/lib/python3.9/site-packages/joblib/parallel.py", line 28, in <module>
    from ._parallel_backends import (FallbackToBackend, MultiprocessingBackend,
  File "/Users/brentbrewington/project-files-github/odds-portal-scraper/full_scraper/venv/lib/python3.9/site-packages/joblib/_parallel_backends.py", line 22, in <module>
    from .executor import get_memmapping_executor
  File "/Users/brentbrewington/project-files-github/odds-portal-scraper/full_scraper/venv/lib/python3.9/site-packages/joblib/executor.py", line 14, in <module>
    from .externals.loky.reusable_executor import get_reusable_executor
  File "/Users/brentbrewington/project-files-github/odds-portal-scraper/full_scraper/venv/lib/python3.9/site-packages/joblib/externals/loky/__init__.py", line 12, in <module>
    from .backend.reduction import set_loky_pickler
  File "/Users/brentbrewington/project-files-github/odds-portal-scraper/full_scraper/venv/lib/python3.9/site-packages/joblib/externals/loky/backend/reduction.py", line 125, in <module>
    from joblib.externals import cloudpickle  # noqa: F401
  File "/Users/brentbrewington/project-files-github/odds-portal-scraper/full_scraper/venv/lib/python3.9/site-packages/joblib/externals/cloudpickle/__init__.py", line 3, in <module>
    from .cloudpickle import *
  File "/Users/brentbrewington/project-files-github/odds-portal-scraper/full_scraper/venv/lib/python3.9/site-packages/joblib/externals/cloudpickle/cloudpickle.py", line 152, in <module>
    _cell_set_template_code = _make_cell_set_template_code()
  File "/Users/brentbrewington/project-files-github/odds-portal-scraper/full_scraper/venv/lib/python3.9/site-packages/joblib/externals/cloudpickle/cloudpickle.py", line 133, in _make_cell_set_template_code
    return types.CodeType(
TypeError: an integer is required (got type bytes)

After making the change the error went away:

(venv) ➜  full_scraper git:(master) ✗ python op.py --help            
usage: op.py [-h] [--number-of-cpus [NUMBER_OF_CPUS]]
             [--wait-time-on-page-load [WAIT_TIME_ON_PAGE_LOAD]]

oddsporter v1.0

optional arguments:
  -h, --help            show this help message and exit
  --number-of-cpus [NUMBER_OF_CPUS]
                        Number parallel CPUs for processing (default -1 for max available)
  --wait-time-on-page-load [WAIT_TIME_ON_PAGE_LOAD]
                        How many seconds to wait on page load (default 3)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions