Pandas drop rows lower then others in all colums

An more memory-efficient and faster solution than the one proposed so far is to use Numba. There is no need to create huge temporary array with Numba. Moreover, it is easy to write a parallel implementation that makes use of all CPU cores. Here is the implementation:

import numba as nb

@nb.njit
def is_dominated(arr, k):
    n, m = arr.shape
    for i in range(n):
        if i != k:
            dominated = True
            for j in range(m):
                if arr[i, j] < arr[k, j]:
                    dominated = False
            if dominated:
                return True
    return False

# Precompile the function to native code for the most common types
@nb.njit(['(i4[:,::1],)', '(i8[:,::1],)'], parallel=True, cache=True)
def dominated_rows(arr):
    n, m = arr.shape
    toRemove = np.empty(n, dtype=np.bool_)
    for i in nb.prange(n):
        toRemove[i] = is_dominated(arr, i)
    return toRemove

# Special case
df2 = df.drop_duplicates()

# Main computation
result = df2[~dominated_rows(np.ascontiguousarray(df.values))]

Benchmark

The input test is two random dataframes of shape 20000x5 and 5000x100 containing small integers (ie. [0;100[). Tests have been done on a (6-core) i5-9600KF processor with 16 GiB of RAM on Windows. The version of @BingWang is the updated one of the 2022-05-24. Here are performance results of the proposed approaches so far:

Dataframe with shape 5000x100
 - Initial code:   114_340 ms
 - BENY:             2_716 ms  (consume few GiB of RAM)
 - Bing Wang:        2_619 ms
 - Numba:              303 ms  <----

Dataframe with shape 20000x5
 - Initial code:    (too long)
 - BENY:             8.775 ms  (consume few GiB of RAM)
 - Bing Wang:          578 ms
 - Numba:               21 ms  <----

This solution is respectively about 9 to 28 times faster than the fastest one (of @BingWang). It also has the benefit of consuming far less memory. Indeed, the @BENY implementation consume few GiB of RAM while this one (and the one of @BingWang) only consumes no more than few MiB for this used-case. The speed gain over the @BingWang implementation is due to the early stop, parallelism and the native execution.

One can see that this Numba implementation and the one of @BingWang are quite efficient when the number of column is small. This makes sense for the @BingWang since the complexity should be O(N(logN)^(d-2)) where d is the number of columns. As for Numba, it is significantly faster because most rows are dominated on the second random dataset causing the early stop to be very effective in practice. I think the @BingWang algorithm might be faster when most rows are not dominated. However, this case should be very uncommon on dataframes with few columns and a lot of rows (at least, clearly on uniformly random ones).

We can do numpy board cast

s = df.values
out = df[np.sum(np.all(s>=s[:,None],-1),1)==1]
Out[44]: 
    A  B  C  D
0  12  7  1  0
1   7  1  2  0
2   1  1  1  1

Here is a try based on Kung et al 1975 http://www.eecs.harvard.edu/~htk/publication/1975-jacm-kung-luccio-preparata.pdf

Brutal force solution is from https://stackoverflow.com/a/68528943/11327160

I didn't robustly test it, but using these parameters it looks to be the same answer

There is no guarantee it is correct, or I am even following the paper. Please test thoroughly. In addition, there is very likely to be a commercial solution to calculate it.

D=5 #dimension, or number of columns
N=2000 #number of data rows
M=1000 #upper bound for random integers

Changing to D=20 and N=20000 you can see Kung75 completes in <1 minute but Brutal Force will use more than 10x the time. Even at Dimension=1000,Rows=20000,value range 0~999, it can still complete slightly over 1 minute

This can be revised similar to merge sort (compute small chunks by brutal force, then merge up with Filter), which is easier to switch to parallel computing. Another way of speeding up is to turn off array boundary check after you are comfortable with the code. This is due to heavy array indexing here. I would recommend C# if you want to try this path.

import pandas as pd
import numpy as np
import datetime

#generate fake data
D=1000 #dimension, or number of columns
N=20000 #number of data rows
M=1000 #upper bound for random integers

np.random.seed(12345) #set seed so this is reproducible
data=np.random.randint(0,M,(N,D))
for i in range(0,12):
    print(i,data[i])

#Compare w and v starting dimention d
def Compare(w,v,d=0):
    cmp=3 #0x11, low bit is GE, high bit is LE, together means EQ
    while d<D:
        if w[d]>v[d]:
            cmp&=1
        elif w[d]<v[d]:
            cmp&=2
        if cmp>0:
            d+=1
        else:
            break
    return cmp # 0=uncomparable, 1=GT, 2=LT, 3=EQ
#unit test:
#print(Compare(data[0],data[1]))
#print(Compare(data[0],data[1],4))
#print(Compare(data[1],data[11]))
#print(Compare(data[11],data[1]))
#print(Compare(data[1],data[1]))
def AuxSort(d,ndxArray): #stable sort desc by dimention d
    return [x[1] for x in sorted([(-data[n][d],n) for n in ndxArray])]
#unit test
#print(AuxSort(data,0,[0,4,3]))
#print(AuxSort(data,2,[0,1,2]))
#cumulatively find the pareto front. Time O(N^2), space O(N)
def N2BrutalForce(data,ndxArray=None,d=0):
    if len(data)==0:
        return []
    if not ndxArray: #by default check the entire data
        ndxArray=list(range(len(data)))
    #up to this point ndxArray is not empty
    result={ndxArray[0]:data[ndxArray[0]]}
    for i in range(1,len(ndxArray)):
        dominated=[]
        j=ndxArray[i]
        for k,v in result.items():
            c=Compare(data[j],v,d)
            if c>1:
                break
            elif c==1:
                dominated.append(k)
        else:
            for o in dominated:
                del result[o]
            result[j]=data[j]
    return [r for r in result]
def resultPrinter(res, ShowCountOnly=False):
    if not ShowCountOnly:
        for r in sorted(res):
            print(r,data[r])
    print(len(res),'results found',datetime.datetime.today())
#unit rest
#resultPrinter(N2BrutalForce(data),True)
#resultPrinter(N2BrutalForce(data,list(range(15))))
def FindT(R1,R2,S1,S2,d):
    S1R1=set(Filter(data,d,R1,S1))
    T1=[s for s in S1 if s in S1R1]
    S2R1=Filter(data,d+1,R1,S2)
    S2R2=set(Filter(data,d,R2,S2))
    T2=[s for s in S2R1 if s in S2R2]
    return T1+T2
def BreakAtPseudoMedian(sArray,d):
    sArray=AuxSort(d,sArray) #this could speed up by moving the sort to caller and avoid redo sorting
    if data[sArray[0]][d]==data[sArray[-1]][d]:
        return [],sArray
    L=len(sArray)
    mHigh=mLow=L//2
    while mLow>0 and data[sArray[mLow]][d]==data[sArray[mLow-1]][d]:
        mLow-=1
    if mLow>0:
        return sArray[:mLow],sArray[mLow:]
    while mHigh<L-1 and data[sArray[mHigh]][d]==data[sArray[mHigh+1]][d]:
        mHigh+=1
    return sArray[:mHigh],sArray[mHigh:]
def Filter(data,d,rArray,sArray):
    L=len(rArray)+len(sArray)
    if d==D-1 and rArray:
        R=max(data[r][d] for r in rArray)
        return [s for s in sArray if data[s][d]>R]
    elif len(rArray)*len(sArray)<=30 or len(rArray)<=2 or len(sArray)<=2:
        nonDominated=[]
        for s in sArray:
            for r in rArray:
                c=Compare(data[s],data[r],d)
                if c>1:
                    break
            else:
                nonDominated.append(s)
        return nonDominated
    S1,S2=BreakAtPseudoMedian(sArray,d)
    R1,R2=BreakAtRefValue(rArray,d,data[S2[0]][d])
    if not S1 and not R1:
        return Filter(data,d+1,rArray,sArray)
    return FindT(R1,R2,S1,S2,d)
#Filter(data,0,[0,1,2,3,4,5,6,7,8,9],[11])
def BreakAtRefValue(rArray,d,br):
    rArray=AuxSort(d,rArray)
    if data[rArray[0]][d]<=br:
        return [],rArray
    if data[rArray[-1]][d]>br:
        return rArray,[]
    mLow,mHigh=0,len(rArray)-1
    while mLow<mHigh-1 and data[rArray[mLow]][d]>br and data[rArray[mHigh]][d]<br:
        mid=(mLow+mHigh)//2
        if data[rArray[mid]][d]>br:
            mLow=mid
        elif data[rArray[mid]][d]<br:
            mHigh=mid
        else:
            mLow=mid
            break
    if data[rArray[mLow]][d]>br and data[rArray[mHigh]][d]<br:
        return rArray[:mHigh],rArray[mHigh:]
    if data[rArray[mLow]][d]==br:
        while data[rArray[mLow-1]][d]==br:
            mLow-=1
        return rArray[:mLow],rArray[mLow:]
    while data[rArray[mHigh-1]][d]==br:
        mHigh-=1
    return rArray[:mHigh],rArray[mHigh:]
def Kung75(data,d,ndxArray):
    L=len(ndxArray)
    if L<10:
        return N2BrutalForce(data,ndxArray,d)
    elif d==D-1:
        x,y=-1,-1
        for n in ndxArray:
            if y<0 or data[n][d]>x:
                x,y=data[n][d],n
        return [y]
    if data[ndxArray[0]][d]==data[ndxArray[-1]][d]:
        return Kung75(data,d+1,AuxSort(d+1,ndxArray))
    R,S=BreakAtPseudoMedian(ndxArray,d)
    R=Kung75(data,d,R)
    S=Kung75(data,d,S)
    T=Filter(data,d+1,R,S)
    return R+T
print('started at',datetime.datetime.today())
resultPrinter(Kung75(data,0,AuxSort(0,list(range(len(data))))),True)

We take the cumulative maximum value per column in the dataframe.

We want to keep all rows that have a single column value that is equal to the maximum. We then drop duplicates using pandas drop_duplicates

In [14]: df = pd.DataFrame(
    ...:     [[12, 7, 1, 0], [7, 1, 2, 0], [1, 1, 1, 1], [2, 2, 0, 0]],
    ...:     columns=["A", "B", "C", "D"],
    ...: )

In [15]: df[(df == df.cummax(axis=0)).any(axis=1)].drop_duplicates()
Out[15]: 
    A  B  C  D
0  12  7  1  0
1   7  1  2  0
2   1  1  1  1

df.sort_values(by=['A', 'B', 'C', 'D'], ascending=False, inplace=True)
df = df.iloc[:cutoff]

If this takes too long you could do it on subsets of the df until it is small enough.