DataLoader worker exited unexpectedly (pid(s) 48817, 48818)

Viewed 2460

When running my code I recieved this error message "RuntimeError: DataLoader worker (pid(s) 48817, 48818) exited unexpectedly" I am completely unsure where to begin to solve this issue. Any guidance at all would be greatly appreciated. Code and traceback posted below

batch_size = 128
image_size = (64,64)
stats = (0.5, 0.5, 0.5), (0.5, 0.5, 0.5)

transform_ds = transforms.Compose([transforms.Resize(image_size),
#                                    transforms.RandomCrop(32, padding=2),
#                                    transforms.RandomHorizontalFlip(),
                                   transforms.ToTensor(),
                                   transforms.Normalize(*stats)
                                   ])

train_ds = facesDataset(csv_file = 'imagesdataset.csv', root_dir = 'images', 
                      transform = transform_ds)

train_dl = DataLoader(train_ds, batch_size, shuffle=True, num_workers=3, pin_memory=True)
print(len(train_ds))

def denorm(img_tensors):
    return img_tensors * stats[1][0] + stats[0][0]

def show_images(img, nmax=64):
    fig, ax = plt.subplots(figsize=(8, 8))
    ax.set_xticks([]); ax.set_yticks([])
    ax.imshow(make_grid(denorm(img.detach()[:nmax]), nrow=8).permute(1, 2, 0))

def show_batch(dl, nmax=64):
    for img, _ in dl:
        show_images(img, nmax)
        break

show_batch(train_dl)

traceback

Traceback (most recent call last):
  File "/Users/___/Desktop/stylegan/stylegan.py", line 52, in <module>
    show_batch(train_dl)
  File "/Users/___/Desktop/stylegan/stylegan.py", line 48, in show_batch
    for img, _ in dl:
  File "/Users/___/opt/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/Users/___/opt/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1186, in _next_data
    idx, data = self._get_data()
  File "/Users/___/opt/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1152, in _get_data
    success, data = self._try_get_data()
  File "/Users/___/opt/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1003, in _try_get_data
    raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e
RuntimeError: DataLoader worker (pid(s) 48817, 48818) exited unexpectedly
1 Answers

One of the reason might be data loading with multiprocessing. As far as I know, in Windows, if you don't set num_workers to 0 then there would be errors. So I recommend you to try without num_workers (because by default, it is 0) or just set it num_workers=0.

train_dl = DataLoader(train_ds, batch_size, shuffle=True, num_workers=0, pin_memory=True)
Related