Numpy loadtxt: ValueError: Wrong number of columns

user_2132157 0
Viewed 37805

Having the file TEST.txt structured as following:

a   45
b   45  55
c   66

When I try to open it:

import numpy as np
a= np.loadtxt(r'TEST.txt',delimiter='\t',dtype=str)

I have got the following error:

ValueError: Wrong number of columns at line 2

It's clearly due to the fact that the second line has three columns instead of two, but I can't find an answer to my problem using the documentation.

Is there anyway I can fix it keeping all the data into an array?

In Matlab I can do something like:

a=textscan(fopen('TEST.txt'),'%s%s%s');

Something similar in Python would be apreciated.

pandas python-3.x numpy python
4 Answers

If you want all rows to have the same number of columns but some have missing values you can do it easily with pandas. But you have to know the total number of columns.

import pandas as pd
pd.read_csv('foo.txt', sep='\t', names=['col_a','col_b'])
user_69488590

I struggled a bit to use the above solutions, so for someone needing an alternative, here is one:

My test.dat file has:

0   1   2
3   4   5
6
7   8   9
10  11  12
13

Code to read above file, and flatten it out in an array:

import numpy as np

filename = 'test.dat'

i = 1
data=[]

with open(filename) as ff:
    for line in ff: #To read a file line by line
        p = []
        p = line.split( ) #To split the elements being read in above line
        if i%3 == 0: #This is to read only one element when column number is different
            data.append(p[0])
        else:
            data.append(p[0])
            data.append(p[1])
            data.append(p[2])
            
        i = i + 1

print(data)

The output is:

['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13']
user_53492620
Related