Pandas: why can you do arithmetic on series with non-identical indexes, but not comparisons?

user_6107981 0
Viewed 70

In Pandas, operations on Series are done with respect to the index. For example:

import pandas as pd

foo = pd.Series(data=[1, 2, 3, 4, 5], index=[0, 1, 2, 3, 4])
bar = pd.Series(data=[2, 3, 6, 5, -3], index=[1, 2, 3, 4, 5])

These two Series have indexes that partially overlap. When I add them, the values in the overlapping parts of the indexes are added, and the parts that don't overlap get NaN:

foo + bar

0     NaN
1     4.0
2     6.0
3    10.0
4    10.0
5     NaN
dtype: float64

foo - bar

0    NaN
1    0.0
2    0.0
3   -2.0
4    0.0
5    NaN
dtype: float64

However, the same logic does not seem to apply when using comparison operators, like ==, !=, <, <=, >, and >=. For example:

foo >= bar

ValueError: Can only compare identically-labeled Series objects

I would have expected this to match on the indices in the same way, giving something like

0    NA
1    True
2    True
3    False
4    True
5    NA
dtype: bool

Why doesn't this happen? And is there a way to workaround this limitation and get the above result in a straightforward way?

pandas python
1 Answers

not sure why it is not possible nor what you're trying to accomplish specifically. But this is an idea:

import numpy as np
import pandas as pd


foo = pd.Series(data=[1, 2, 3, 4, 5], index=[0, 1, 2, 3, 4])
bar = pd.Series(data=[2, 3, 6, 5, -3], index=[1, 2, 3, 4, 5])

ratio = (foo+bar)/bar
foo_greater = ratio.apply(lambda x: x if np.isnan(x) else True if x>=2 else False)

print(foo_greater) 

Returns:

0      NaN
1     True
2     True
3    False
4     True
5      NaN
dtype: object

Leveraging the fact you can execute addition and division, if the value in "ratio" is not NaN and greater or equal to 2, then foo >= bar, hence True.

Similar logic can be applied for all the other comparison operations.

user_136708670
Related