How do I apply a learning rate scheduler lr_scheduler_cls to a DARTS RNNmodel?

user_18021961 0
Viewed 676

I am trying to create an RNN with learning rate scheduler using DARTS and start fitting:

rnn_model2_cov = RNNModel(model= 'GRU',     
    hidden_dim=30,
    input_chunk_length=200, 
    output_chunk_length=100,                                                           
    random_state=42,
    n_rnn_layers=4,
    batch_size=1024,
    dropout= 0.1,
    optimizer_kwargs={'lr': 1e-3},
    optimizer_cls = torch.optim.Adam ,                            
    lr_scheduler_cls = torch.optim.lr_scheduler.ReduceLROnPlateau ,                            
    log_tensorboard=False,                              
    nr_epochs_val_period = 5
)

rnn_model2_cov.fit(series=z[:15000]],
    future_covariates=[covary],
    num_loader_workers=0,
    epochs = 100,                      
    verbose=True
)

Fitting starts up and crashes with an error:


  File "C:\Users\micha\anaconda3\envs\darts_env\lib\site-packages\darts\utils\torch.py", line 65, in decorator
    return decorated(self, *args, **kwargs)

  File "C:\Users\micha\anaconda3\envs\darts_env\lib\site-packages\darts\models\forecasting\torch_forecasting_model.py", line 436, in fit
    self.fit_from_dataset(train_dataset, val_dataset, verbose, epochs, num_loader_workers)

  File "C:\Users\micha\anaconda3\envs\darts_env\lib\site-packages\darts\utils\torch.py", line 65, in decorator
    return decorated(self, *args, **kwargs)

  File "C:\Users\micha\anaconda3\envs\darts_env\lib\site-packages\darts\models\forecasting\torch_forecasting_model.py", line 530, in fit_from_dataset
    self._train(train_loader, val_loader, tb_writer, verbose, train_num_epochs)

  File "C:\Users\micha\anaconda3\envs\darts_env\lib\site-packages\darts\models\forecasting\torch_forecasting_model.py", line 827, in _train
    self.lr_scheduler.step()

TypeError: step() missing 1 required positional argument: 'metrics'

How & where can I pass the missing metric and how do I formulate the other arguments with lr_scheduler_kwargs={ } ? It seems to need an optimizer class as argument?

The documentation is quiet good, but very brief on this particular topic.

u8darts pytorch machine-learning python
2 Answers

You are passing optimizer_kwargs and lr_scheduler_cls as you should. See a toy example with another model from darts which trains without any problem at all:

from darts.models.forecasting.nbeats import NBEATSModel

# Generic architecture
model_nbeats = NBEATSModel(
  input_chunk_length=32,
  output_chunk_length=1,
  generic_architecture=True,
  num_stacks=2,
  num_blocks=2,
  num_layers=2,
  layer_widths=256,
  n_epochs=50,
  nr_epochs_val_period=1,
  batch_size=128,
  random_state=0,
  optimizer_cls = torch.optim.Adam,
  optimizer_kwargs={'lr': 1e-3},
  lr_scheduler_cls = torch.optim.lr_scheduler.ReduceLROnPlateau,
  torch_device_str="cuda:0",
  model_name="nbeats_run",
)
model_nbeats.fit(series=train, val_series=val, verbose=True)

Notice that the lr_scheduler_cls is the Torch scheduler, i.e. torch.optim.lr_scheduler._LRScheduler. The problem is that your Darts library version was outdated and didn't support that functionality. Upgrading to version 0.16 fixes this.

user_24113200

Using ReduceLROnPlatuea in DARTS forecast models was not correctly handled in previous versions of DARTS, and was fixed only a few weeks ago: see this commit

Updating DARTS to the latest version should fix the issue.

user_87425050
Related