I was wondering what the fastest way of converting a bytestring to an array of unit8 would be in the following code? I use hashlib for SHA-256
x = hashlib.sha256(str(word).encode("ascii")).digest()
Now I need to convert x to an array of uint8. I currently do this by list comprehension, but this seems slow.
h_new = [int(y) for y in x]
num_hashes[idx_perm, :] = h_new
Does anyone have suggestions for a faster way of conversion?
You can use the function frombuffer of Numpy. Since the size of a sha256 is always 32 bytes and the output type is well known, the function can convert the input buffer x very quickly.
num_hashes[idx_perm, :] = np.frombuffer(x, np.uint8, 32)
This takes about 0.7 us per call on my machine while the initial code takes about 3.35 us. Thus, this version is about 4.8 faster. Note that this is also faster than converting the result to a list (due to the many int objects to be allocated and reference-counted) and the from fromiter Numpy function (since the iterable interface introduces an additional overhead).