Understanding NumPy dtype "c" with strings

user_9568847 0
Viewed 1884

Goal: Convert str to np.ndarray of bytes of size 1:

import numpy as np
np.array("abc", dtype=[whatever])

Actual result without dtype: array('abc', dtype='<U3')

Desired result: array([b'a', b'b', b'c'], dtype=[whatever] This lets me use slicing to get

Workaround I found but don't understand:

np.array("abc", dtype='c')
# array([b'a', b'b', b'c'], dtype='|S1')

I found this by trial and error, thinking that 'c' could mean 'char'

What I don't understand: Why is dtype='c' working the way it is? According to arrays.dtypes reference the 'c' is short for "complex-floating point", while '|S1' is a "zero-terminated bytes (not recommended)" of length 1.

Also directly using '|S1' as dtype ignores every character but the first, which is not what I would expect, but I guess it just takes the "abc" as one argument and b'a' is what comes out if just a single byte is specified as dtype:

np.array("abc", dtype='|S1')
# array(b'a', dtype='|S1')

Question(s):

  1. Why is dtype='c' working the way it is?
  2. (If dtype='c' is just working "by accident", what would be a "right way" to do this?)

PS: Yes, there is a np.chararray, but according to the linked documentation:

The chararray class exists for backwards compatibility with Numarray, it is not recommended for new development. Starting from numpy 1.4, if one needs arrays of strings, it is recommended to use arrays of dtype object_, string_ or unicode_, and use the free functions in the numpy.char module for fast vectorized string operations.

However recommended dtypes object_, string_ and unicode_ don't split the string into chars but return a ndarray with one element.

char numpy arrays python
1 Answers

It seems like a bug to me. Note that if you don't specify the number of bytes after the character code 'c', then the dtype is actually 'S1', not the complex floating-point. Have a look at these attributes for dtypes:

>>> dt_S1 = np.dtype('S1')
>>> dt_S1, dt_S1.kind, dt_S1.name, dt_S1.char
(dtype('S1'), 'S', 'bytes8', 'S')

>>> dt_c = np.dtype('c')
>>> dt_c, dt_c.kind, dt_c.name, dt_c.char))
(dtype('S1'), 'S', 'bytes8', 'c')

>>> dt_c8 = np.dtype('c8')
>>> dt_c8, dt_c8.kind, dt_c8.name, dt_c8.char
(dtype('complex64'), 'c', 'complex64', 'F')

So one would expect for np.array('abc', dtype='c') and np.array('abc', dtype='S1') to return the same result array(b'a', dtype='S1'), or for the former to give an error as would np.array('abc', dtype='c8').

Imho, the correct way to achieve your task would be:

np.array(list('abc'), dtype='S1')
user_29453570
Related