Create a list of random numbers and filter the list to only have numbers larger than 50

Viewed 3207

I am using list comprehension to create a list of random numbers with numpy. Is there a way to check if random number generated is larger than 50 and only then append it to the list.

I know I can simply use:

numbers = [np.random.randint(50,100) for x in range(100)]

and that would solve the issue, but I just want to know if its possible to somehow check if np.random.randint(1,100) generated number greater than 50

Something like

numbers = [np.random.randint(1,100) for x in range(100) if {statement}]

Because comparing np.random.randit generates another number which is not the same as the first one.

I just want to know if there is a possibility to filter generated numbers before adding them to a list.

9 Answers

You use numpy, so we can leverage indexing method.

my_array = np.random.randint(1, 100, size=100)
mask = my_array > 50
print(my_array[mask]) # Contain only value greater than 50

But of course, the best way to do what you want is that.

results = np.random.randint(51,100, size=100)
# If you really need a list
results_list = results.tolist()

Please, don't loop over a numpy array in general.

Edit : replace my_list by my_array based on @norok2 comment.

Edit2 : Speed considerations

Numpy

With mask

%%timeit
my_array = np.random.randint(1, 100, size=100)
mask = my_array > 50
my_array[mask]

5.31 µs ± 127 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

10,000,000 elements:

1 loop, best of 3: 198 ms per loop

With numpy where (@Severin Pappadeux anwser)

%%timeit
q = np.random.randint(1, 100, 1000)
m = np.where(q > 50)
q[m]

20.9 µs ± 663 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

10,000,000 elements:

1 loop, best of 3: 196 ms per loop

Pure python

Part of @Alexander Cécile answer

%%timeit
rand_nums = (random.randint(0, 99) for _ in range(10))
arr = [val for val in rand_nums if val > 50]

19.4 µs ± 1.99 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

10,000,000 elements:

1 loop, best of 3: 11.4 s per loop

Mix numpy and list

@DrBwts answer

%%timeit
number = [x for x in np.random.randint(1, high=100, size=100) if x > 50]

28.9 µs ± 1.52 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

10,000,000 elements:

1 loop, best of 3: 2.76 s per loop

@makis answer

%%timeit 
numbers = [x for x in (np.random.randint(1,100) for iter in range(100)) if x > 50]

164 µs ± 19.4 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

10,000,000 elements:

1 loop, best of 3: 12.2 s per loop

@Romero Valentine answer

rand = filter(lambda x: x>50, np.random.randint(1,100,100))
rand_list = list(rand)

35.9 µs ± 1.97 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

10,000,000 elements:

1 loop, best of 3: 3.41 s per loop

Conclusions

  • On simple tasks, these methods are all fine.
  • On large arrays, numpy crushes the competition.

Answers that provide more information

  • @norok2
  • @Alexander Cécile

In your question you have a some code:

numbers = [np.random.randint(50, 100) for x in range(100)]

which should probably be

numbers = np.random.randint(50, 100, 100)

Then you are asking if there is a way of replicate this using list-comprehension combined with filtering, starting from something that reads like:

numbers = [np.random.randint(1, 100) for x in range(100) if ...]

The answer to this is in general NO.

The reason for this is that the condition-filtering syntax will act as a filter to the generated numbers, after the numbers are generated and tentatively yielded.

Therefore, while the code [np.random.randint(50,100) for x in range(100)] will generate 100 items, any method based on comprehension filtering or even explicit use of filter() will have an unknown number of elements (typically less than 100, depending on how many items do meet the specified condition).

To get to that level of control, you may use a while-based generator (which cannot be included in a comprehension), e.g.:

import numpy as np


def my_brand_new_generator(n, a=1, b=100):
    i = 0
    while i < n:
        x = np.random.randint(a, b)
        if x > 50:
            yield x
            i += 1


numbers = list(my_brand_new_generator(100))
print(numbers[10])
# [94 97 50 53 53 89 59 69 71 86]

print(len(numbers))
# 100

By contrast:

numbers = [x for x in (np.random.randint(1,100) for iter in range(100)) if x > 50]
print(len(numbers))
# 54

x = np.random.randint(1, 100, 100)
numbers = x[x > 50]
print(len(numbers))
# 43

As side notes:

  • the my_brand_new_generator() is to be considered only a toy example to illustrate the aforementioned idea.
  • in general NumPy offers better alternatives to explicit iteration, so, when available, please use that.
  • for generating a single random integer, you can use random.randint() from the Python standard library.

Yes its possible, first generate your 100 numbers (or how many you need) between 1 & 100 here but again that's upto you, then iterate through them checking each one & only keeping the ones over 50...

y = np.random.randint(1, high=100, size=100)
number = [x for x in y if x > 50]

Alternatively in one line...

number = [x for x in np.random.randint(1, high=100, size=100) if x > 50]

@FlorianBernard is right, do NOT mix Python lists and NumPy arrays. Stay in NumPy world, things would be faster anyway. THere is no equivalent in NumPy for python lists comprehension, because it operates on arrays, not on a single items one by one.

Anyway, there is a bit more complicated but more powerful alternative to @FlorianBernard answer, using numpy.where function

F.e., it allows fusion of filtered and baseline arrays into new one. Simple code to start with

import numpy as np
q = np.random.randint(1, 100, 1000)
m = np.where(q > 50)
print(q[m])

Be careful not to mix numpy and plain python when you have no reason to! You write that you could use numbers = [np.random.randint(50,100) for x in range(100)], but it would be better to just do np.random.randint(low=50, high=100, size=100) to create an array of 100 random numbers.

Here is a pure python solution:

import random

rand_nums = (random.randint(0, 99) for _ in range(10))
arr = [val for val in rand_nums if val > 50]

Which can obviously be written as one line:

arr = [val for val in (random.randint(0, 99) for _ in range(10)) if val > 50]

And here is a numpy solution:

import numpy as np

arr = np.random.randint(low=0, high=100, size=10)
arr = arr[arr > 50]

Use

numbers = [x for x in (np.random.randint(1,100) for iter in range(100)) if x > 50]

Step 1: (np.random.randint(1,100) for iter in range(100)): this generates 100 times, a random int between 1 and 100.

Step 2: if x > 50: The generated number is compared. If it's >50 it passes and is appended in the list.

Just use the filter function:

rand = filter(lambda x: x>50, np.random.randint(1,100,100))
rand_list = list(rand)

No list comprehension necessary.

import random
ml=random.sample(range(100),100)


# Get list of items larger than 50 using list comprehension approach
filtered = [x for x in ml if x > 50]
print(filtered)


# Get list of items larger than 50 using filter approach
filtered = list(filter(lambda x: x > 50, ml))
print(filtered)

Cheat it by creating a recursive function that will only unwind if the number is greater than 50.

import numpy as np

def genrandx():
    # Using a function allows you do to all sorts of crazy stuff and checks.
    x = np.random.randint(1,100)
    x = int((x*.5)**2)
    if (
        (x > 50) and
        (x < 1000) and 
        ( (x%3 == 0) or (x%3 == 0) )
       ): 
        return x 
    else: 
        return genrandx() 

numbers = [ genrandx() for x in range(100)]
print("({}){}".format(len(numbers), numbers))

OUTPUT:

(100)[930, 729, 306, 576, 600, 225, 324, 324, 462, 324, 462, 756, 324, 225, 702, 552, 576, 72, 81, 900, 600, 225, 930, 900, 576, 240, 552, 702, 441, 72, 462, 132, 144, 324, 72, 702, 462, 930, 72, 306, 240, 225, 870, 210, 729, 600, 420, 132, 240, 420, 441, 420, 72, 756, 225, 900, 72, 90, 72, 600, 72, 420, 210, 702, 240, 462, 600, 156, 81, 900, 144, 72, 225, 324, 144, 420, 600, 576, 729, 156, 900, 81, 756, 729, 702, 90, 462, 306, 600, 930, 729, 240, 552, 144, 90, 900, 420, 225, 600, 156]
Related