I'm having a lot of troubles understanding why we have to do np.sum(..., axis=0) when we are computing the gradients of bias w.r.t. loss in a Neural Net.
I'm studying with a book and there's this nice 2-layer NN diagram:
now, if I want to know by how much I have to update (for example) B_2, I just do:
gradient of bias wrt loss = ∂(L)/∂(b_2) = ∂(L)/∂(P) * ∂(P)/∂(b_2) = -(Y - P) * np.ones_like(b_2)
Then just bias -= LR * (gradient of bias wrt loss).
And that's it, no? Where is calculus saying that we have to sum along the axis 0?