I have two data frames df1 and df2 with the same data types and the same column names.
now when I extract df1 from the merged frame and compare it with the original data frame I have an error!! I mean it shows they are not equal??
I don't understand why is there an error after concatenating two data frames even if they have same datatypes and data formats.
I checked the same using rbind in Rstudio and I still get errors like
please let me know if you ever faced a problem like this or any suggestions for a solution
Dummy Data Example (Since the data I have is confidential):
df1 = pd.DataFrame({"age_grp": ["<30", "30-40", "50-60", ">70"],
"year": [2015, 2015, 2015, 2015],
"sex": ["Male", "Female", "Female", "Male"],
"country": ["UK","France","Portugal","USA"]})
df2 = pd.DataFrame({"age_grp": ["30-40", "50-60","<30", ">70"],
"year": [2016, 2016, 2016, 2016],
"sex": ["Male", "Female", "Female", "Male"],
"country": ["France","France","USA","USA"]})
#concat two dfs
new_df = pd.concat([df1,df2], ignore_index=True)
#extract year 2016 from the merged dataframe(new_df)
extract = new_df.loc[new_df['year'] == 2016]
#compare extract with the original df2
extract.equals(df2) # should return TRUE but I am getting "False" with the data I have
Code used in R:
all.equal(extract,df2) #for comparison
The error I got:
[1] "Component “age_grp”: Attributes: < Component “levels”: Lengths (7, 6) differ (string compare on first 6) >"
[2] "Component “name”: Attributes: < Component “levels”: Lengths (120, 103) differ (string compare on first 103) >"
[3] "Component “setting”: Attributes: < Component “levels”: Lengths (17, 13) differ (string compare on first 13) >"
[4] "Component “sex”: Attributes: < Component “levels”: Lengths (8, 7) differ (string compare on first 7) >"