Getting ValueError: Columns must be same length as key

user_16467848 0
Viewed 2275

I get this error

ValueError: Columns must be same length as key

My code is

all_columns=[ col  for col, dt in y.dtypes.items() if dt == 'object']
df[all_columns] = df[all_columns].astype(str)

I have some columns which are in dict format. So I want to convert all those columns into proper string data type. Please tell me why I am getting this error, I am using the same column list on both sides, right?

> Answer : Duplicates columns in the dataframe resulted in this type of error. There are two columns with the same name.

dataframe python-3.x list python
2 Answers

Try this simpler iteration, since it looks like something is going wrong with your list comprehension. Given the issue you describe in your comments, do something like print the name of the column as you iterate, to see which column is crashing:

df = pd.DataFrame({'TEXT': ['hello there', 'python fun', np.nan, '']})

print(df.dtypes)
# TEXT    object
# dtype: object

for col in df.columns:
    print(col)
    if df[col].dtype == 'object':
        df[col] = df[col].astype(str)
        print('converted', col)

print(df.dtypes)
# TEXT    object
# dtype: object

Alternatively, try this list comprehension to get all object columns at once, maybe your list comprehension is the issue:

df = pd.DataFrame({'TEXT1': ['hello there', 'python fun', np.nan, ''],
                   'TEXT2': ['hi', 'bye', np.nan, ''],
                   'not text': [1, 2, 3, 4]})

obj_cols = [col for col in df.columns if df[col].dtype == 'object']

df[obj_cols] = df[obj_cols].astype(str)
user_163178920

@Hummarabi I started going step by step by taking 10 columns at a time and print the column names which are of type object

for col in y.columns[1:10]:
    if y[col].dtypes == 'object':
#          y[col] = y[col].astype(str)
        print(col)

So I got this error

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

during the y.columns[50:60] columns So I printed the columns by y[y.columns[60:70]].head(80) and to my surprise I saw that there are two columns with the same name. ('error','error') So I dropped one of the column and ran and now everything works fine and I get no errors now. But this type of manual checking is bit cumbersome for extremely large data though. But one observation is ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

occurs when there are duplicates in the data

user_164678480
Related