Multiclass SVM from Scratch

Viewed 121

Can we have implementation of multiclass svm from scratch

Below is the code tried:

X_data, y_data = DF.drop(['label'], axis=1).values, DF.label.values
X_train, y_train, X_test, y_test = data_split(X_data, y_data)

X_train.shape, y_train.shape, X_test.shape, y_test.shape

shape = X_train[0].shape[0]
num_classes = len(np.unique(y_train))
loss = 0

w = np.random.uniform(0, 1, [num_classes, shape]) # initializing weights
delta = 1
lr = 0.0001

for x, y in zip(X_train, y_train): # training

    # inference
    dw = np.zeros_like(w)       # weight gradient
    scores = x.dot(w.T)
    correct_class_score = scores[y]

    # loss
    for i in range(num_classes):

        if i == y:
            continue

        res = scores[i] - correct_class_score + delta
        loss += np.max([0, res])

    # gradient
    for j in range(num_classes):

        if j == y:
            if scores[i] - correct_class_score + delta > 0:
                dw[j,:] = -x
        else:
            if scores[i] - correct_class_score + delta > 0 > 0:
                dw[j,:] = x

    w -=  (lr * dw)

Can we have a sample optimized code with an example so as to validate with standard package for the same?

0 Answers
Related