gradient descent using numpy

Question

I have a gradientDescent function that takes ini train_X, train_y, test_X, test_y, L, num_iter as its inputs and returns the optimal value of parameters and history of loss (which is just the RSME between predicted and actual y)on training data and testing data respectively.

I have previously wrote a function to calculate the loss:

def RMSEFunction(X, theta, y):

  loss = None
  predicted_y = np.dot(X, theta)
  diff = np.subtract(predicted_y,y)
  diff_sq = np.square(diff)
  MSE = np.mean(diff_sq)
  loss = math.sqrt(MSE)

  return loss

However, I am stuck at trying to write the algorithm for gradient descent. For now, I have:

def gradientDescent(train_X, train_y, test_X, test_y, L, num_iter):  

  N_train, D = train_X.shape # number of training samples, number of features
  theta = np.zeros((D,1)) # the parameters of the linear regression model
  opt_theta = None # the optimized parameters on the training data 

  # record loss of each iteration - both rmse between training data and target
  # data as well as between testing data and target data.
  loss_history = np.zeros((2,num_iter))
  test_loss = RMSEFunction(test_X, theta, test_y)
  
  # Gradient descent steps
  for i in range(num_iter): 
    predicted_y = np.dot(train_X, theta)
    loss = np.subtract(predicted_y,train_y)
    gradient = np.dot(train_X.T, loss)
    opt_theta -= (1/N_train) * L * gradient

    loss_history[0,i] = RMSEFunction(train_X, theta, train_y)
    loss_history[1,i] = RMSEFunction(test_X, theta, test_y)
  
    if loss_history[1,i] < test_loss: # store the optimal parameter
        test_loss = loss_history[1,i]
        opt_theta = np.copy(theta)
  
  return [opt_theta, loss_history]

The problem lies for when I start the gradient descent steps till opt_theta. When I run the code, I get:

TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

I would like to deal with my flawed logic and the errors in my code if possible. Could someone please provide some guidance?

Thank you.

gradient descent using numpy

0 Answers