loss too erratic unless I put complete data in a batch on simple pytorch network

I am trying out a tutorial from https://analyticsindiamag.com/step-by-step-guide-to-build-a-simple-neural-network-in-pytorch-from-scratch/

The training data shape is (67,4). When I put the batch size >=67, the loss is plotting smoothly

But if the batch size is even 66 or anything less, the loss plot is too erratic

I tried it out with different strategies like weight decay dropout, training data normalisation but this case didnt change. Least to say, I am confused. Can someone help me with what is happening here?

X, Y  = sklearn.datasets.make_classification(n_features=4,n_redundant=0,n_informative=3,n_clusters_per_class=2,n_classes=3)

data = Data()
loader = DataLoader(data,batch_size=10,shuffle=True)

Network and optimizer:

class Net(nn.Module):
    def __init__(self, input_dim, hidden, output_dim):
        super(Net,self).__init__()
        self.linear1 = nn.Linear(input_dim,hidden)
        self.linear2=nn.Linear(hidden, output_dim)
        
    def forward(self,X_ip):
        op1 = torch.relu(self.linear1(X_ip))
        op2 = self.linear2(op1)
        return op2

lossfn = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(clf.parameters(),lr=0.01,momentum=0.09)

from tqdm import tqdm
train_loss = []

for gx in tqdm(range(4000)):
    
    for index, data_ in enumerate(loader):
        clf.zero_grad()
        
        x,y = data_
        op = clf(x)
        loss = lossfn(op,y)
        train_loss.append(loss.detach().numpy())
        loss.backward()
        optimizer.step()

step=np.linspace(0,len(train_loss),len(train_loss))
plt.plot(step,np.array(train_loss))