Pandas concat fail and appear NaN

user_14452873 0
Viewed 40

I want vertical concat two dataframe but I have no idea why it concact by ['data_label'] and generate NaN?

print(leakage_data1)

    0    1    2   data_label
0  0.2  0.1  0.7      0
1  0.5  0.6  0.4      0


print(noise_data1)

    0    1    2   data_label
0  0.9  0.8  1.0      1
1  0.7  0.5  0.1      1

I use model1_TestData = pd.concat([leakage_data1, noise_data1], axis=0, ignore_index=True)
and got

print(model1_TestData)

    0    1    2   data_label   0    1    2
0  0.2  0.1  0.7      0       NaN  NaN  NaN
1  0.5  0.6  0.4      0       NaN  NaN  NaN
2  NaN  NaN  NaN      1       0.9  0.8  1.0
3  NaN  NaN  NaN      1       0.7  0.5  0.1

How can i fix this problem?

pandas dataframe nan concatenation python
3 Answers

Check the data types on your column names. If one df has a string column name '1' and another is 1, it will print out on the screen the same but will give you the error you see there because to concat you have to have exactly the same column names and 1!='1'

model1_TestData.columns

should reveal something like Index(['0', '1',..., 0, 1], dtype='object')

user_32206490

You need to just use

df.merge(df1, right_index = True, left_index = True)

when you are using axis =0 you are essentially telling it to do it via rows instead of columns...unless you are trying to do that, so you would want your data to simply be more vertical instead of horizontal?

user_177504310

This should work:

df1 = pd.DataFrame({0:[.2, .5], 1:[.1, .6], 2:[.7, .4], 'data_label':0})
df2 = pd.DataFrame({0:[.9, .7], 1:[.8, .5], 2:[1., .1], 'data_label':1})
print(pd.concat([df1, df2]))

Result:

     0    1    2  data_label
0  0.2  0.1  0.7           0
1  0.5  0.6  0.4           0
0  0.9  0.8  1.0           1
1  0.7  0.5  0.1           1
user_74758380
Related