Is it possible to initialize a random arrays of float32 or float16 without casting?

user_3259896 0
Viewed 282

I see that there's np.random.uniform, but from the documentation https://numpy.org/doc/stable/reference/random/generated/numpy.random.uniform.html

There is no argument for the data type, and by default it uses float.64 values.

I can not just cast the values to float32 or float16 because I would need to initialize the arrays at float64 and for the number of embeddings I am working with, this would use up all the ram in my enviroment.

numpy python
2 Answers

You can achieve this for float32 using the random generator api, which does have a dtype parameter for the random method. Sadly this won't accept a float16.

import numpy as np
size = [1, 100]
high = 50
low = 20
rand_array = np.random.default_rng().random(size, dtype=np.float32) * (high - low) + low

This code will make rand_array a float32 array of shape [1,100] with uniformly random numbers between 20 and 50.

user_32636760

One option is to accept the need to cast, but do it in chunks that will not cause you to exceed the available memory. Adjust the chunk_size to keep the time spent on explicit looping to a reasonably small amount while avoiding excessive memory usage.

import numpy as np

def uniform(minval, maxval, length,
              dtype=None, chunk_size=100000):
    
    answer = np.empty((length), dtype=dtype)
    for start in range(0, length, chunk_size):
        end = min(start + chunk_size, length)
        answer[start:end] = np.random.uniform(minval, maxval, end - start)

    return answer


x = uniform(2, 4, 1234567, dtype=np.float16)
print(x, x.dtype)
user_135960370
Related