How do i find derivative of softmax in python

Viewed 396

i have trouble implementing back propogation for multi class classification of CIFAR10 dataset

My neural network has 2 layers

forward propagation

X -> L1 -> L2

weights w2 is initialized as random

np.random.randn(L1.shape[0], X.shape[0]) * 0.01

X is input of size (no_features * number of examples)

Z1 = (w1 * x) + b1

A1 = relu(Z1)

L1 has ReLu activation

Z2 = (w2 * A1) + b2

A2 = softmax(Z1)

L2 has softmax activation

cost is caluclated using this equation

cost = -(1/m)*np.sum((Y * np.log(A2) ) + ((1 - Y)*np.log(1-A2)))

back propagation

derivative of cost is calculated

dA2 = -(1/m)*(np.divide(Y, A2) - np.divide(1 - Y, 1 - A2))

dA2 = derivative of A2

Y = one hot encoded True values

now how do i proceed from here

how do i find dZ2 (derivative of Z2) using dA2

0 Answers
Related