Skip to content

Endless warning message when use imap #269

@BeiYaaa

Description

@BeiYaaa

Hi there,

I'm running below script on Google Vertex AI Jupyter Notebook.

import pathos as pa
with PPool(pa.helpers.cpu_count()) as p:
                    # make sure the pool is started
                    try:
                        p.restart()
                    except:
                        pass
                    
                    record_df_ls = []
                    record_df_ls = p.imap(self._fit_n_evaluate, param_grid)
                    
                    # close the process pool
                    p.close()
                    # wait for all tasks to complete
                    p.join()

I was able to run it successfully before. However, today I notice that there are warning messages printed on the screen and I couldn't kill the kernel. Below is part of the recurring and endless output.
File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 286, in loads return load(file, ignore, **kwds) Process ForkPoolWorker-9: Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap self.run() File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py", line 110, in worker task = get() File "/opt/conda/lib/python3.7/site-packages/multiprocess/queues.py", line 357, in get return _ForkingPickler.loads(res) File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 272, in load return Unpickler(file, ignore=ignore, **kwds).load() File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 419, in load obj = StockUnpickler.load(self) File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 942, in _create_namedtuple t = collections.namedtuple(name, fieldnames, defaults=defaults, module=modulename) Process ForkPoolWorker-10: File "/opt/conda/lib/python3.7/collections/__init__.py", line 370, in namedtuple raise ValueError('Field names cannot start with an underscore: ' Traceback (most recent call last): ValueError: Field names cannot start with an underscore: '_1' File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap self.run() File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run self._target(*self._args, **self._kwargs) Process ForkPoolWorker-11: File "/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py", line 110, in worker task = get() File "/opt/conda/lib/python3.7/site-packages/multiprocess/queues.py", line 357, in get return _ForkingPickler.loads(res)

I noticed that it says ValueError can't start with an underscore. But I don't have file starting with an underscore in the fit function. This confuses me a lot.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions