-
-
Notifications
You must be signed in to change notification settings - Fork 89
Description
Hi there,
I'm running below script on Google Vertex AI Jupyter Notebook.
import pathos as pa
with PPool(pa.helpers.cpu_count()) as p:
# make sure the pool is started
try:
p.restart()
except:
pass
record_df_ls = []
record_df_ls = p.imap(self._fit_n_evaluate, param_grid)
# close the process pool
p.close()
# wait for all tasks to complete
p.join()
I was able to run it successfully before. However, today I notice that there are warning messages printed on the screen and I couldn't kill the kernel. Below is part of the recurring and endless output.
File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 286, in loads return load(file, ignore, **kwds) Process ForkPoolWorker-9: Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap self.run() File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py", line 110, in worker task = get() File "/opt/conda/lib/python3.7/site-packages/multiprocess/queues.py", line 357, in get return _ForkingPickler.loads(res) File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 272, in load return Unpickler(file, ignore=ignore, **kwds).load() File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 419, in load obj = StockUnpickler.load(self) File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 942, in _create_namedtuple t = collections.namedtuple(name, fieldnames, defaults=defaults, module=modulename) Process ForkPoolWorker-10: File "/opt/conda/lib/python3.7/collections/__init__.py", line 370, in namedtuple raise ValueError('Field names cannot start with an underscore: ' Traceback (most recent call last): ValueError: Field names cannot start with an underscore: '_1' File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap self.run() File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run self._target(*self._args, **self._kwargs) Process ForkPoolWorker-11: File "/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py", line 110, in worker task = get() File "/opt/conda/lib/python3.7/site-packages/multiprocess/queues.py", line 357, in get return _ForkingPickler.loads(res)
I noticed that it says ValueError can't start with an underscore. But I don't have file starting with an underscore in the fit function. This confuses me a lot.