I was trying to write a function called spatial_batchnorm_forward, used in Convolutional Neural Network. In this function, I wanted to reuse the batchnorm_foward function, which is implemented for a (N, D) shaped input in Fully Connected Network.
The following is a correct implementation.
def spatial_batchnorm_forward(x, gamma, beta, bn_param):
"""Computes the forward pass for spatial batch normalization.
"""
out, cache = None, None
N, C, H, W = x.shape
x_ = x.transpose(0,2,3,1).reshape(N*H*W, C)
out_, cache = batchnorm_forward(x_, gamma, beta, bn_param)
out = out_.reshape(N, H, W, C).transpose(0,3,1,2)
return out, cache
But at first, I wrote it as:
def spatial_batchnorm_forward(x, gamma, beta, bn_param):
"""Computes the forward pass for spatial batch normalization.
"""
out, cache = None, None
N, C, H, W = x.shape
x_ = x.reshape(-1, C)
out_, cache = batchnorm_forward(x_, gamma, beta, bn_param)
out = out_.reshape(N, C, H, W)
return out, cache
This code can run, which means those dimensions match. But the output is slightly different from the above one. I was wondering what's going on here. Really appreciate your patience and help!!!
I guess the problem occurs in the reshape function, so I read the document.
numpy.reshape(a, newshape, order='C')[source]
Gives a new shape to an array without changing its data.
Parameters
aarray_like
Array to be reshaped.
newshapeint or tuple of ints
The new shape should be compatible with the original shape. If an integer, then the result will be a 1-D array of that length. One shape dimension can be -1. In this case, the value is inferred from the length of the array and remaining dimensions.
order{‘C’, ‘F’, ‘A’}, optional
Read the elements of a using this index order, and place the elements into the reshaped array using this index order. ‘C’ means to read / write the elements using C-like index order, with the last axis index changing fastest, back to the first axis index changing slowest. ‘F’ means to read / write the elements using Fortran-like index order, with the first index changing fastest, and the last index changing slowest. Note that the ‘C’ and ‘F’ options take no account of the memory layout of the underlying array, and only refer to the order of indexing. ‘A’ means to read / write the elements in Fortran-like index order if a is Fortran contiguous in memory, C-like order otherwise.
Returns
reshaped_arrayndarray
This will be a new view object if possible; otherwise, it will be a copy. Note there is no guarantee of the memory layout (C- or Fortran- contiguous) of the returned array.
But I still can't figure out what's going on here.