Can't get attribute 'collate_fn'

[ajr82]

When trying to perform the independent patient-wise stratified 10-Fold CV on my data, I get the following error:

I am running Win 11 and Python 3.10.

There is a GPU!
Finished loading dataset and creating dataloader
Initializing models
Epoch 0/99

0%| | 0/9 [00:00<?, ?it/s]Traceback (most recent call last):
File "", line 1, in
File "C:\Program Files\Python310\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Program Files\Python310\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
AttributeError: Can't get attribute 'collate_fn' on <module 'mp_main' from 'C:\GitHub\WSI-Pancreas-Classification-master\src\main.py'>
Traceback (most recent call last):
File "", line 1, in
File "C:\Program Files\Python310\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Program Files\Python310\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
AttributeError: Can't get attribute 'collate_fn' on <module 'mp_main' from 'C:\GitHub\WSI-Pancreas-Classification-master\src\main.py'>
Traceback (most recent call last):
File "", line 1, in
File "C:\Program Files\Python310\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Program Files\Python310\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
AttributeError: Can't get attribute 'collate_fn' on <module 'mp_main' from 'C:\GitHub\WSI-Pancreas-Classification-master\src\main.py'>
Traceback (most recent call last):
File "", line 1, in
File "C:\Program Files\Python310\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Program Files\Python310\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
AttributeError: Can't get attribute 'collate_fn' on <module 'mp_main' from 'C:\GitHub\WSI-Pancreas-Classification-master\src\main.py'>
0%| | 0/9 [00:05<?, ?it/s]
Traceback (most recent call last):
File "C:\Users\ajraj_t7g\AppData\Roaming\Python\Python310\site-packages\torch\utils\data\dataloader.py", line 1132, in _try_get_data
data = self._data_queue.get(timeout=timeout)
File "C:\Program Files\Python310\lib\queue.py", line 179, in get
raise Empty
_queue.Empty

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\GitHub\WSI-Pancreas-Classification-master\src\main.py", line 264, in
model, results = train(model, criterion, optimizer, dataloaders, transforms,
File "C:\GitHub\WSI-Pancreas-Classification-master\src\wsi_model.py", line 94, in train
for batch_idx, batch in enumerate(tqdm(dataloaders[phase])):
File "C:\Users\ajraj_t7g\AppData\Roaming\Python\Python310\site-packages\tqdm\std.py", line 1178, in iter
for obj in iterable:
File "C:\Users\ajraj_t7g\AppData\Roaming\Python\Python310\site-packages\torch\utils\data\dataloader.py", line 633, in next
data = self._next_data()
File "C:\Users\ajraj_t7g\AppData\Roaming\Python\Python310\site-packages\torch\utils\data\dataloader.py", line 1328, in _next_data
idx, data = self._get_data()
File "C:\Users\ajraj_t7g\AppData\Roaming\Python\Python310\site-packages\torch\utils\data\dataloader.py", line 1284, in _get_data
success, data = self._try_get_data()
File "C:\Users\ajraj_t7g\AppData\Roaming\Python\Python310\site-packages\torch\utils\data\dataloader.py", line 1145, in _try_get_data
raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e
RuntimeError: DataLoader worker (pid(s) 8168, 21536, 8668, 8244) exited unexpectedly

[pacocp]

Hi,

It looks like that is a specific Windows error, and unfortunately, I don't have a Windows machine so the code was only tested on Linux.

In the main.py file you can find the following function:

def collate_fn(batch):
    """Remove bad entries from the dataloader
    Args:
        batch (torch.Tensor): batch of tensors from the dataaset
    Returns:
        collate: Default collage for the dataloader
    """
    batch = list(filter(lambda x: x[0] is not None, batch))
    return torch.utils.data.dataloader.default_collate(batch)

that is used in the Pytorch DataLoaders to omit corrupted images. Unless you have deleted it, it should be able to find that function and use it. Can you please check out if the function is in the main.py file?