compare entire row and column in numpy array and delete selected rows and columns

user_14629406 0
Viewed 378

I have 2 square array with shape = (25, 25) and I want to check if an entire row is filled with zeros and if the corresponding column is filled with zeros. If this is the case I want to remove those columns and rows from the array.

For example:

array = np.array([[1, 0, 1, 1],
                  [0, 0, 0, 0],
                  [1, 0, 1, 1],
                  [1, 0, 1, 1]])

I want it manipulated to

array=np.array([[1, 1, 1],
                [1, 1, 1],
                [1, 1, 1]])

I hope you can understand what I am aiming at. In this example row and column two have been removed as they are zero rows/columns.

I could do that by iterating through all of those arrays, as I have 10 million of those arrays I would like to have a pythonic/efficient way to solve this issue.

The second array is a tensorflow array manipulating that should be no problem if I know the index of the rows columns I want removed.

Edit:

I have now found following solution, but it is using for-looping:

def removepadding(y_true, y_pred):
 shape = np.shape(y_true)
 y_true_cleaned=[]
 for i in range(shape[0]):
     x = y_true[i]
     for n in range(shape[1] - 1, -1, -1):
         if sum(x[n, :]) == 0 and sum(x[:, n]) == 0:
             x = np.delete(np.delete(x, n, 0), n, 1)
     y_true_cleaned.append(x)

 return y_true_cleaned
tensorflow numpy arrays python
4 Answers

You can do it in one line:

array[array.any(axis = 1)][:, array.any(axis = 0)]
#array([[1, 1, 1],
#       [1, 1, 1],
#       [1, 1, 1]])
user_137432330

if there is negative values in the arr, np.sum may fail.

for 2d array:

import numpy as np
a = np.array([[1,0,2,3,0,4],
              [0,0,0,0,0,0],
              [0,0,0,0,0,0],
              [2,0,3,4,0,5],
              [3,0,4,5,0,6],
              [4,0,5,6,0,7],
              [5,0,6,7,0,8]])
row = np.all(a==0, axis=1)
col = np.all(a==0, axis=0)
a[~row][:,~col]

output

array([[1, 2, 3, 4],
       [2, 3, 4, 5],
       [3, 4, 5, 6],
       [4, 5, 6, 7],
       [5, 6, 7, 8]])

for 3d array:

a = np.ones((3,3,3))
a[1,:,1] = 0
a[1,1,:] = 0
a[:,1,1] = 0
z = np.all(a==0, axis=2)
y = np.all(a==0, axis=1)
x = np.all(a==0, axis=0)
Z = ~np.array([z]*a.shape[2])
Y = ~np.array([y]*a.shape[1])
X = ~np.array([x]*a.shape[0])
ZZ, YY, XX = (Z*Y*X).nonzero()
a[ZZ, YY, XX]
user_104577810

Edit: Thanks to @mad physicist, we can use np.flatnonzero. Here's the 2d case:

import numpy as np
a=np.array([[1,0,2,3,0,4],
            [0,0,0,0,0,0],
            [0,0,0,0,0,0],
            [2,0,3,4,0,5],
            [3,0,4,5,0,6],
            [4,0,5,6,0,7],
            [5,0,6,7,0,8]])
cols_to_keep = np.flatnonzero(a.sum(axis=0))
rows_to_keep = np.flatnonzero(a.sum(axis=1))
a = a[:, cols_to_keep]
a = a[rows_to_keep, :]
a

>>>
array([[1, 2, 3, 4],
       [2, 3, 4, 5],
       [3, 4, 5, 6],
       [4, 5, 6, 7],
       [5, 6, 7, 8]])

Here's the 3d case:

import numpy as np
a=np.array([
    [[1,0,2,3,0,4],
    [0,0,0,0,0,0],
    [0,0,0,0,0,0],
    [2,0,3,4,0,5],
    [3,0,4,5,0,6],
    [4,0,5,6,0,7],
    [5,0,6,7,0,8]],

    [[0,0,0,0,0,0],
    [0,0,0,0,0,0],
    [0,0,0,0,0,0],
    [0,0,0,0,0,0],
    [0,0,0,0,0,0],
    [0,0,0,0,0,0],
    [0,0,0,0,0,0]],

    [[5,0,5,5,0,5],
    [0,0,0,0,0,0],
    [0,0,0,0,0,0],
    [2,0,3,4,0,5],
    [3,0,4,5,0,6],
    [4,0,5,6,0,7],
    [5,0,6,7,0,8]],

])
ix_keep_axis_0 = np.flatnonzero(a.sum(axis=(1, 2)))
ix_keep_axis_1 = np.flatnonzero(a.sum(axis=(0, 2)))
ix_keep_axis_2 = np.flatnonzero(a.sum(axis=(0, 1)))
a = a[ix_keep_axis_0, :, :]
a = a[:, ix_keep_axis_1, :]
a = a[:, :, ix_keep_axis_2]
a

>>>
array([[[1, 2, 3, 4],
        [2, 3, 4, 5],
        [3, 4, 5, 6],
        [4, 5, 6, 7],
        [5, 6, 7, 8]],

       [[5, 5, 5, 5],
        [2, 3, 4, 5],
        [3, 4, 5, 6],
        [4, 5, 6, 7],
        [5, 6, 7, 8]]])
user_12878840

You can use np.count_nonzero to get the indices in one step per dimension:

nnz_row = np.count_nonzero(array, axis=1)
nnz_col = np.count_nonzero(array, axis=0)

Now you make a mask of where both are zero:

mask = (nnz_row == 0) & (nnz_col == 9)

You can turn the mask into indices and pass it to np.delete:

ind = np.flatnonzero(mask)
array = np.delete(np.delete(array, ind, axis=0), ind, axis=1)

Alternatively, you can compute the positive mask:

pmask = nnz_row.astype(bool) | nnz_col.astype(bool)

This mask can select directly, analogously to what delete did with the negative mask:

array = array[pmask, :][:, pmask]
user_29887300
Related