Cartesian product of two numpy arrays, with condition

user_11188140 0
Viewed 429

Given 1d arrays w and x, below, I can form the Cartesian product using the following code.

import numpy as np

w = np.array([1, 2, 3, 4])
x = np.array([1, 2, 3, 4])

V1 = np.transpose([np.repeat(w, len(x)), np.tile(x, len(w))])
print(V1)

[[1 1]
 [1 2]
 [1 3]
 [1 4]
 [2 1]
 [2 2]
 [2 3]
 [2 4]
 [3 1]
 [3 2]
 [3 3]
 [3 4]
 [4 1]
 [4 2]
 [4 3]
 [4 4]]

But, I want the output, V1, to include ONLY array rows where w < x (as shown below). I can do this with loops, but I'm hoping for something a little faster.

[[1 2]
 [1 3]
 [1 4]
 [2 3]
 [2 4]
 [3 4]]
vectorization numpy arrays performance
3 Answers

Approach #1

Given ONLY array rows where w < x, which would be for pairwise combinations, here's one way to achieve the same -

In [81]: r,c = np.nonzero(w[:,None]<x) # or np.less.outer(w,x)

In [82]: np.c_[w[r], x[c]]
Out[82]: 
array([[1, 2],
       [1, 3],
       [1, 4],
       [2, 3],
       [2, 4],
       [3, 4]])

Approach #2

Using a purely masking based approach, it would be -

In [93]: mask = np.less.outer(w,x)

In [94]: s = (len(w), len(x))

In [95]: np.c_[np.broadcast_to(w[:,None], s)[mask], np.broadcast_to(x, s)[mask]]
Out[95]: 
array([[1, 2],
       [1, 3],
       [1, 4],
       [2, 3],
       [2, 4],
       [3, 4]])

Benchmarking

Using comparatively larger arrays :

In [8]: np.random.seed(0)
   ...: w = np.random.randint(0,1000,(1000))
   ...: x = np.random.randint(0,1000,(1000))

In [9]: %%timeit
   ...: r,c = np.nonzero(w[:,None]<x)
   ...: np.c_[w[r], x[c]]
11.3 ms ± 24.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [10]: %%timeit
    ...: mask = np.less.outer(w,x)
    ...: s = (len(w), len(x))
    ...: np.c_[np.broadcast_to(w[:,None], s)[mask], np.broadcast_to(x, s)[mask]]
10.5 ms ± 275 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [11]: import itertools

# @Akshay Sehgal's soln
In [12]: %timeit [i for i in itertools.product(w,x) if i[0]<i[1]]
105 ms ± 1.38 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
user_32938810

Try this one line method with itertools -

import itertools

w = np.array([1, 2, 3, 4])
x = np.array([1, 2, 3, 4])

[i for i in itertools.product(w,x) if i[0]<i[1]]

[(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)]

Itertools is extremely fast and memory-efficient. it should be significantly fast.

user_47559540

You can also filter your own solution by:

V1 = np.transpose([np.repeat(w, len(x)), np.tile(x, len(w))])
V1[V1[:,0]<V1[:,1]]

Of course proposed solution by @Divakar is faster since it does not do redundant calculations.

Benchmark using @Divakar's benchit package:

#@Proposed solution here
def m1(x):
  V1 = np.transpose([np.repeat(w, len(x)), np.tile(x, len(w))])
  return V1[V1[:,0]<V1[:,1]]

#@Divakar's approach 1
def m2(x):
  r,c = np.nonzero(w[:,None]<x) # or np.less.outer(w,x)
  return np.c_[w[r], x[c]]

#Divakar's approach 2
def m3(x):
  mask = np.less.outer(w,x)
  s = (len(w), len(x))
  return np.c_[np.broadcast_to(w[:,None], s)[mask], np.broadcast_to(x, s)[mask]]

in_ = [np.arange(n) for n in [10,100,1000]]
w = x.copy()

enter image description here

user_49759810
Related