I'm currently obliged to use the 'boolean' variable type, which is like 'bool' except it allows missing values. My problem is that when I try to use the compare() function with 'boolean' data, it crashes. An example:
import pandas as pd
df1 = pd.DataFrame({'Key':['a','b','c','d'],
'BoolVar': pd.Series([True, False, None, True], dtype='boolean')})
df2 = pd.DataFrame({'Key':['a','b','c','d'],
'BoolVar': pd.Series([True, False, True, True], dtype='boolean')})
df1.compare(df2)
The last line crashes with "Assertion Error: object". If you replace 'boolean' with 'bool' in the code above, it works (though 'None' becomes 'False', of course). I can't figure out why. Can anyone explain it?
The variable type 'boolean' is officially experimental right now, which could explain things. Its values can be compared to each other (e.g., df1.iloc[0]['BoolVar'] == df2.iloc[0]['BoolVar'] works), so I could compare the dataframes in other ways, but I'd like to know what's going on here, especially if there are known ways in which this data type cannot be used.
The same problem exists with 'Int64', a version of 'int64' (also experimental) that can take missing values.