We know in pandas, we can set
pd.set_option('use_inf_as_na', True) to make sure inf is automatically casted to nan
Is there a similar option in polars?
1 Answers
Not automatically, that would be a very expensive setting as well.
But you can convert infinite values to NaN if you wish.
pl.DataFrame({
"a": [float("inf"), 1.0, 2.0]
}).with_column(
pl.when(pl.col("a").is_infinite()).then(float("nan")).otherwise(pl.col("a")).keep_name()
)
shape: (3, 1)
┌─────┐
│ a │
│ --- │
│ f64 │
╞═════╡
│ NaN │
├╌╌╌╌╌┤
│ 1.0 │
├╌╌╌╌╌┤
│ 2.0 │
└─────┘
Note that the floating point NaN does not mean missing data in polars. So it might make more sense to replace with None if you want other algorithms to treat it as missing.
Related