Numpy array losing dimensions when applying mask of same shape

user_9170199 0
Viewed 2301

I have created a 2D mask appropriately named mask to have the same shape as the array data, the array I want to apply it upon. However when I do so, the data loses it's shape and becomes 1D.

I thought that as each level for axis 0 is identical (shown with creation of mask using loop comprehension) that the output would produce output with shape (837, 10)

I am wondering is there any numpy tricks to be used to achieve this goal without using reshape?

>>> data.shape
(837, 44)

>>> m = altitudes < 50000
>>> m.shape
(44,)

>>> np.sum(m) # calculates my expected dimension for axis 1
10

>>> mask = [m for i in range(data.shape[0])]
>>> mask.shape
(837, 44)

>>> new_data = data[mask]
>>> new_data.shape
(8370,) # same as 837 * 10 (dimension wanted)

If this can't be achieved, why would this be?

masking numpy-ndarray numpy arrays python
2 Answers

The 'correct' way of achieving your goal is to not expand the mask to 2D. Instead index with [:, mask] with the 1D mask. This indicates to numpy that you want axis 0 unchanged and mask applied along axis 1.

a = np.arange(12).reshape(3, 4)
b = np.array((1,0,1,0),'?')
a
# array([[ 0,  1,  2,  3],
#        [ 4,  5,  6,  7],
#        [ 8,  9, 10, 11]])
b
# array([ True, False,  True, False])
a[:, b]
# array([[ 0,  2],
#        [ 4,  6],
#        [ 8, 10]])

If your mask is already 2D, numpy won't check whether all its rows are the same because that would be inefficient. But obviously you can use [:, mask[0]] in that case.

If your mask is 2D and just happens to have the same number of Trues in each row then either use @tel's answer. Or create an index array:

B = b^b[:3, None]
B
# array([[False,  True, False,  True],
#        [ True, False,  True, False],
#        [False,  True, False,  True]])
J = np.where(B)[1].reshape(len(B), -1)

And now either

np.take_along_axis(a, J, 1)
# array([[ 1,  3],
#        [ 4,  6],
#        [ 9, 11]])

or

I = np.arange(len(J))[:, None]
IJ = I, J
a[IJ]
# #array([[ 1,  3],
#         [ 4,  6],
#         [ 9, 11]])
user_72073920

I believe what you want can be done by calling new_data.reshape(837, -1). Here's a brief example:

arr = np.arange(8*6).reshape(8,6)
maskpiece = np.array([True, False]*3)
mask = np.broadcast_to(maskpiece, (8,6))

print('the original array\n%s\n' % arr)
print('the flat masked array\n%s\n' % arr[mask])
print('the masked array reshaped into 2D\n%s\n' % arr[mask].reshape(8, -1))

Output:

the original array
[[ 0  1  2  3  4  5]
 [ 6  7  8  9 10 11]
 [12 13 14 15 16 17]
 [18 19 20 21 22 23]
 [24 25 26 27 28 29]
 [30 31 32 33 34 35]
 [36 37 38 39 40 41]
 [42 43 44 45 46 47]]

the flat masked array
[ 0  2  4  6  8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46]

the masked array reshaped into 2D
[[ 0  2  4]
 [ 6  8 10]
 [12 14 16]
 [18 20 22]
 [24 26 28]
 [30 32 34]
 [36 38 40]
 [42 44 46]]
user_4254580
Related