Cpu/Runtime Error in fastai/Pytorch with learn.fit_one_cycle()

Viewed 795

Its my first time properly training a cnn model with fastai on a laptop with 16GB of ram and I was trying to follow a tutorial that had this code:

np.random.seed(42)
data = vision.ImageDataBunch.from_folder(path, valid_pct=0.2,
        ds_tfms=get_transforms(), size=224, num_workers=4, bs=32).normalize(imagenet_stats)
data.classes, data.c, len(data.train_ds), len(data.valid_ds)

learn = cnn_learner(data, models.resnet50, metrics=accuracy).to_fp16()

learn.fit_one_cycle(4)

when I tried to run learn.fit)one_cycle(4), it returns this error:

epoch     train_loss  valid_loss  accuracy  time    
c:\users\lu_41\fastai1\fastai\vision\transform.py:247: UserWarning: torch.solve is deprecated in favor of torch.linalg.solveand will be removed in a future PyTorch 
release.
torch.linalg.solve has its arguments reversed and does not return the LU factorization.
To get the LU factorization see torch.lu, which can be used with torch.lu_solve or torch.lu_unpack.
X = torch.solve(B, A).solution
should be replaced with
X = torch.linalg.solve(A, B) (Triggered internally at  ..\aten\src\ATen\native\BatchLinearAlgebra.cpp:859.)
  return _solve_func(B,A)[0][:,0]
c:\users\lu_41\fastai1\fastai\vision\transform.py:247: UserWarning: torch.solve is deprecated in favor of torch.linalg.solveand will be removed in a future PyTorch 
release.
torch.linalg.solve has its arguments reversed and does not return the LU factorization.
To get the LU factorization see torch.lu, which can be used with torch.lu_solve or torch.lu_unpack.
X = torch.solve(B, A).solution
should be replaced with
X = torch.linalg.solve(A, B) (Triggered internally at  ..\aten\src\ATen\native\BatchLinearAlgebra.cpp:859.)
  return _solve_func(B,A)[0][:,0]
c:\users\lu_41\fastai1\fastai\vision\transform.py:247: UserWarning: torch.solve is deprecated in favor of torch.linalg.solveand will be removed in a future PyTorch 
release.
torch.linalg.solve has its arguments reversed and does not return the LU factorization.
To get the LU factorization see torch.lu, which can be used with torch.lu_solve or torch.lu_unpack.
X = torch.solve(B, A).solution
should be replaced with
X = torch.linalg.solve(A, B) (Triggered internally at  ..\aten\src\ATen\native\BatchLinearAlgebra.cpp:859.)
  return _solve_func(B,A)[0][:,0]
c:\users\lu_41\fastai1\fastai\vision\transform.py:247: UserWarning: torch.solve is deprecated in favor of torch.linalg.solveand will be removed in a future PyTorch 
release.
torch.linalg.solve has its arguments reversed and does not return the LU factorization.
To get the LU factorization see torch.lu, which can be used with torch.lu_solve or torch.lu_unpack.
X = torch.solve(B, A).solution
should be replaced with
X = torch.linalg.solve(A, B) (Triggered internally at  ..\aten\src\ATen\native\BatchLinearAlgebra.cpp:859.)
  return _solve_func(B,A)[0][:,0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\users\lu_41\fastai1\fastai\train.py", line 23, in fit_one_cycle
    learn.fit(cyc_len, max_lr, wd=wd, callbacks=callbacks)
  File "c:\users\lu_41\fastai1\fastai\basic_train.py", line 200, in fit
    fit(epochs, self, metrics=self.metrics, callbacks=self.callbacks+callbacks)
  File "c:\users\lu_41\fastai1\fastai\basic_train.py", line 101, in fit
    loss = loss_batch(learn.model, xb, yb, learn.loss_func, learn.opt, cb_handler)
  File "c:\users\lu_41\fastai1\fastai\basic_train.py", line 26, in loss_batch
    out = model(*xb)
  File "C:\Users\lu_41\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\lu_41\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\container.py", line 141, in forward
    input = module(input)
  File "C:\Users\lu_41\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\lu_41\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\container.py", line 141, in forward
    input = module(input)
  File "C:\Users\lu_41\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\lu_41\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\conv.py", line 447, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "C:\Users\lu_41\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\conv.py", line 443, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'

does anyone know what I could try to do to fix this? Is it something to do with my CPU?

2 Answers

try to change to_fp16() -> to_fp32()

I had the same error. If you really want to use fp16 there are two options.

  1. use bfloat16 instead of float16.
  2. move your data and model to gpu and try again

If you choose to do 2, you can use following commands

device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

Above should return cuda:0, which means you have gpu. Then you can move model and data to gpu using following commands.

model.to(device)
inputs, labels = data[0].type(torch.float16).to(device), data[1].to(device)
Related