I am trying to build a Linear Regression method from scratch by following a tutorial called Zero to Gans.
I am using the Medical Cost dataset https://www.kaggle.com/datasets/mirichoi0218/insurance
I first started off by changing all the non-numerical values to numerical ones.
df["region"].replace(["northeast", "southeast", "southwest", "northwest"], [0,1,2,3], inplace = True)
I then split the data into input and output
inputdf = df.iloc[:, 0:6]
outputdf = df.iloc[:, 6]
Converting all of these to Numpy Arrays and then to Tesnsors
inputvalues = inputdf.to_numpy()
outputvalues = outputdf.to_numpy()
outputvalues = np.reshape(outputvalues, (1338, 1))
inputvalues = torch.from_numpy(inputvalues)
outputvalues = torch.from_numpy(outputvalues)
Created the weights and bias matrix and vector.
weights = torch.randn(1,6, dtype=torch.float64, requires_grad = True)
bias = torch.randn(1, dtype=torch.float64, requires_grad=True)
Then created the actual model
def linearegression(x, bias):
return x @ weights.t() + bias
The the issue comes around here.
def lossfunc(y, y_):
delta = y - y_
return torch.sum(delta * delta) / delta.numel()
loss = lossfunc(predictions, outputvalues)
print(loss)
When calculating the loss, I get something to 10^8 which is stupidly high for the loss. I am trying to do this by scratch, as I am new to machine learning and I want to knuckle down the basics. I fully understand that there are functions built into the PyTorch library which can create this model within 10 lines of code but I am trying to follow the first part of this Jovian Notebook https://jovian.ai/aakashns/02-linear-regression (however he using Linear Regression to calculate crop yield).
Does anyone know why my loss is so stupidly large?