Is there an efficient way to get the position of the max element except for a specific column in a NumPy matrix?

Viewed 181

For example, There is a 2d Numpy matrix M:

[[1,10,3],
 [4,15,6]]

The max element except for those in M[:][1] is 6, and its position is (1,2). So the answer is (1,2).

Thank you very much for any help!

5 Answers

One way:

col = 1
skip_col = np.delete(x, col, axis=1)
row, column = np.unravel_index(skip_col.argmax(), skip_col.shape)
if column >= col:
    column += 1 

Translated:

  1. Remove the column
  2. find the maximum argument (argmax gives a flattened result, unravel_index gives back the placement in the 2d array)
  3. If the column is greater or equal to the skipped one, add one

Following Dunes comment, I like the suggestion. It's nearly identical in amount of lines, but does not require a copy (as in np.delete). So if you are memory bound (as in really big data):

col = 1
row, column = np.unravel_index(x[:, :col].argmax(), x[:, :col].shape)  # left max, saving a line assuming it's the global max, but less readable
right_max = np.unravel_index(x[:, col+1:].argmax(), x[:, col+1:].shape)
if x[right_max] > x[row, column]:
    row, column = right_max
    column += col

Here's a solution taking advantage of the set of nan functions:

In [180]: arr = np.array([[1,10,3],[4,15,6]])                                   
In [181]: arr1 = arr.astype(float)                                              
In [182]: arr1[:,1]=np.nan                                                      
In [183]: arr1                                                                  
Out[183]: 
array([[ 1., nan,  3.],
       [ 4., nan,  6.]])
In [184]: np.nanargmax(arr1)                                                    
Out[184]: 5
In [185]: np.unravel_index(np.nanargmax(arr1),arr.shape)                        
Out[185]: (1, 2)

It might not be optimal timewise, but is probably easier to debug that alternatives.

Looking at the np.nanargmax I see that it just replaces the np.nan with -np.inf. So we do something similar by just replacing the exclude column values with a small enough integer so they won't be the max.

In [188]: arr1=arr.copy()                                                       
In [189]: arr1[:,1] = np.min(arr1)-1                                            
In [190]: arr1                                                                  
Out[190]: 
array([[1, 0, 3],
       [4, 0, 6]])
In [191]: np.argmax(arr1)                                                       
Out[191]: 5
In [192]: np.unravel_index(np.argmax(arr1),arr.shape)                           
Out[192]: (1, 2)

I can also imagine a solution using np.ma.masked_array, but that tends to be more of a convenience than speed tool.

Here is what you can do:

m = [[1,10,3],
     [4,15,6]]

c = 1 # Choose the column to exclude 

a = max([[n,(k,b)] for k,i in enumerate(m) for b,n in enumerate(i) if b!=c])[1]

print(a)

Output:

(1, 2)

Agreeing with the comment by Dunes:

With small arrays, like your example, it's probably just quicker to make a copy of the matrix, without the given column, and then take the max. With a larger array it may be quicker to take the max either side of the column and take the max of the left and the right sides of the column.

Here is an implementation of each of these cases, and a dispatcher function. (A value for THRESHOLD_SIZE needs to be added based on experimentation.)

Small array case

Creates array with the specified column removed. Calculates the overall maximum and then the location where it occurs. Adds one to the column if it is on the right side.

Large array case

It creates temporary 1d arrays containing the column maxima. These will typically (although not in every case) be significantly smaller than the 2-dimensional array. First, it is identified which side of the excluded column contains the maximum, then it is identified which column it is, and finally which row it is. This avoids the need to examine every element twice. The code also avoids creating any 2-dimensional slice of the array at any point.

THRESHOLD_SIZE = .....


def get_max_position(m, exclude_column):
    return (get_max_position_largearray if m.size > THRESHOLD_SIZE 
            else get_max_position_smallarray)(m, exclude_column)


def get_max_position_smallarray(m, exclude_column):

    mnew = np.delete(m, exclude_column, axis=1)

    row, col = np.argwhere(mnew == np.max(mnew))[0]

    # uses: int(True)=1 and int(False)=0
    return (row, col + (col >= exclude_column))


def get_max_position_largearray(m, exclude_column):

    column_maxima = np.max(m, axis=0)

    l_col_maxima = column_maxima[:exclude_column]
    r_col_maxima = column_maxima[exclude_column + 1:]

    l_max = np.max(l_col_maxima) if l_col_maxima.size else None
    r_max = np.max(r_col_maxima) if r_col_maxima.size else None

    use_left = (True if r_max == None else
                False if l_max == None else
                (l_max > r_max))

    if use_left:
        themax = l_max
        col = np.argwhere(l_col_maxima == themax)[0][0]
    else:
        themax = r_max
        col = exclude_column + 1 + np.argwhere(r_col_maxima == themax)[0][0]

    row = np.argwhere(m[:,col] == themax)[0][0]

    return (row, col)

Here is the example in the question, by both methods:

m = np.array([[1,10,3],
              [4,15,6]])

exclude_column = 1

print(get_max_position_largearray(m, exclude_column))
print(get_max_position_smallarray(m, exclude_column))

Output:

(1, 2)
(1, 2)

Another way without a copy, indexing the columns with a list:

import numpy as np

m = np.array([[1, 10, 3], [4, 15, 6]])
exclude_col = 1

# assign nicer names to the shape
rows, cols = m.shape

# generate indices for slicing
inds = list(range(cols))
inds.remove(exclude_col)

# find the maximum in the sliced array
max_ind = np.unravel_index(np.argmax(m[:, inds]), (rows, cols - 1))
# fix the found column index if we exceeded exclude_col
max_ind = (max_ind[0], max_ind[1] if max_ind[1] < exclude_col else max_ind[1] + 1)

The last line is a good candidate for a Python3.8 assignment expression, so in Python3.8+ you could write:

max_ind = (max_ind[0], v if (v := max_ind[1]) < exclude_col else v + 1)

EDIT: Indexing like that probably also creates a copy, I have not tested it, but the elements are not contiguous in memory.

Related