Behavior of Python numpy tofile and fromfile when saving int16

user_16417474 0
Viewed 248

After appending data to an empty array and verifying it's shape, I use tofile to save it. When I read it back with fromfile, the shape is much larger (4x).

# Create empty array
rx_data = np.empty((0), dtype='int16')

# File name to write data
file_name = "Test_file.bin"

# Generate two sinewaves and add to rx_data array in interleaved fashion
for i in range(100):
    I = np.sin(2*3.14*(i/100))
    rx_data = np.append(rx_data, I)
    Q = np.cos(2*3.14*(i/100))
    rx_data = np.append(rx_data, Q)
print(rx_data.shape)

# Write array to .bin file
rx_data.tofile(file_name)


# Read back data from the .file
s_interleaved = np.fromfile(file_name, dtype=np.int16)
print(s_interleaved.shape)

The above code returns (200) before the array is saved, but return 800 when the array is read back in. Why does this happen?

numpy python
1 Answers

The problem is that rx_data has a float64 data type before you save it. This is because I and Q are float64 arrays, so when you use np.append, the type will be promoted to be compatible with the float64 values.

Also, populating an array using np.append is an anti-pattern in numpy. It's common to do in standard python, but with numpy, it is often better to create an array of the shape and data type you need, then to fill in the values in the for loop. This is better because you only need to create the array once. With np.append, you create a new copy every time you call it.

rx_data = np.empty(200, dtype="float64")
for i in range(200):
    rx_data[i] = np.sin(...)

Here is code that works, because it uses float64. But this pattern should be avoided in general. Prefer pre-allocating an array and replacing the values in the for loop.

# Create empty array
rx_data = np.empty((0), dtype='int16')

# File name to write data
file_name = "Test_file.bin"

# Generate two sinewaves and add to rx_data array in interleaved fashion
for i in range(100):
    I = np.sin(2*3.14*(i/100))
    rx_data = np.append(rx_data, I)
    Q = np.cos(2*3.14*(i/100))
    rx_data = np.append(rx_data, Q)
print(rx_data.shape)

# Write array to .bin file
print("rx_data data type:", rx_data.dtype)
rx_data.tofile(file_name)


# Read back data from the .file
s_interleaved = np.fromfile(file_name, dtype=np.float64)
print(s_interleaved.shape)

# Test that the arrays are equal.
np.allclose(rx_data, s_interleaved)  # True
user_56660870
Related