Ignore missing columns in 'usecol' parameter

user_3169714 0
Viewed 1946

I'm reading a table from csv and only want a subset of the columns. The list I'm using to subset contains field names that may not exist in the table I'm reading. For example:

# contents of sample.csv:
#a,b,c
#1,2,3
#4,5,6

subset = ['a', 'c', 'd']

I'd like to return the following, using pandas.read_csv and the subset, but this raises an error:

pd.read_csv(sample.csv, usecols=subset)
a  c
1  3
4  6

ValueError: Usecols do not match columns, columns expected but not found: ['d']

I think I might be able to do with a callable value for usecols, but am not sure how to implement.

pandas python
2 Answers

Use a callable checking if the column is in the subset

subset = ['a', 'c', 'd']
df = pd.read_csv('sample.csv', usecols=lambda x: x in subset)

   a  c
0  1  3
1  4  6
user_43333590

It is throwing error because as per your CSV file, columns are a,b,c. But in your subset you are trying to fetch a,b,d. But column d is not present in your sample.

user_101801670
Related