How to select codes with different sizes in pandas?

user_8321427 0
Viewed 54

In Python 3, with pandas, I have this dataframe with several codes in the columns "CPF_CNPJ_doador" and "CPF_CNPJ_doador_originario"

cand_doacoes = pd.read_csv("doacoes_csv.csv",sep=';',encoding = 'latin_1',  decimal = ",")

cand_doacoes.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 427489 entries, 0 to 427488
Data columns (total 12 columns):
UF                                427489 non-null object
Partido                           427489 non-null object
Cargo                             427489 non-null object
Nome_candidato                    427489 non-null object
CPF_candidato                     427489 non-null int64
CPF_CNPJ_doador                   426681 non-null float64
Nome_doador                       427489 non-null object
Nome_doador_Receita               427489 non-null object
Valor                             427489 non-null float64
CPF_CNPJ_doador_originario        427489 non-null object
Nome_doador_originario            427489 non-null object
Nome_doador_originario_Receita    427489 non-null object
dtypes: float64(2), int64(1), object(9)
memory usage: 39.1+ MB

The codes in the columns "CPF_CNPJ_doador" and "CPF_CNPJ_doador_originario" are always integers and of different sizes: 14 digits, 13 digits, 11 digits or 10 digits

Examples of codes in "CPF_CNPJ_doador": 1256007000131, 20566882000164, 11459125000109, 5746038000121...

I need to create a dataframe with only 14- and 13-digit codes. Please, does anyone know how I can select only the 14- and 13-digit codes in the "CPF_CNPJ_doador" column in the dataframe "cand_doacoes"? Do I need to convert to string first?

pandas python
3 Answers

If you really have integers for the codes, one mathematical approach that should be quick is to take the base 10 log of the codes as integer type and add 1 to count the number of digits. This can be your mask. 

np.isin(np.log10(df.code_column.values).astype(int) + 1, [13, 14])

For example (borrowing Bharath's data), 

>>> df[np.isin(np.log10(df.num.values).astype(int) + 1, [13, 14])]
              num
0   1256007000131
1  20566882000164
2  11459125000109
user_46866250

One possible way using astype(str).str.len() and isin to select rows with only 13 and 14 digit values i.e 

df = pd.DataFrame({'num':[1256007000131, 20566882000164, 11459125000109, 57,5746038]})

df[df['num'].astype(str).str.len().isin([13,14])]
# Output 
              num
0   1256007000131
1  20566882000164
2  11459125000109
user_48006520

borowing @Bharath's sample DF:

In [114]: df['num'].astype(str).str.match(r'\d{13,14}')
Out[114]:
0     True
1     True
2     True
3    False
4    False
Name: num, dtype: bool

In [115]: df[df['num'].astype(str).str.match(r'\d{13,14}')]
Out[115]:
              num
0   1256007000131
1  20566882000164
2  11459125000109
user_57412050
Related