Since i have NaN in my df (they cant be replaced for any other value), the findmax() function returns NaN as the largest values found, which is not useful at all... I've been reading several blogs and posts and found nothing. To be more specific, is there anyway that i can make this function like python's np.nanmax()?
2 Answers
Perhaps a simpler solution (and faster, too, it seems) than going through skipmissing is:
foo(itr) = maximum(x for x in itr if !isnan(x))
findmax is not the same as np.nanmax. I assume you want the functionality of np.nanmax, then do:
julia> your_vector = [1.0, NaN, 3.0, NaN, 2.0]
5-element Vector{Float64}:
1.0
NaN
3.0
NaN
2.0
julia> maximum(skipmissing(isnan(x) ? missing : x for x in your_vector))
3.0
julia> your_vector = [NaN, NaN]
2-element Vector{Float64}:
NaN
NaN
julia> maximum(skipmissing(isnan(x) ? missing : x for x in your_vector))
ERROR: ArgumentError: reducing over an empty collection is not allowed
The trick is to replace NaN with missing (but without materializing the result to save memory) and then skip missing. Note that I have proposed a solution that errors when only NaN are present. In Python you would get a warning then, but in Julia we usually avoid printing warnings as they can be very easily lost unnoticed and instead require and explicit handling or error situation.
Now you might ask why there is no simple solution for NaN, but there is a solution for missing (as with missing you would just write maximum(skipmissing(your_vector)))? The answer is that NaN values are not typically encountered in Julia codes (unless indeed you have some computation that produces NaN) while in Python NaN is often used as a value that signals that we have a missing value in data (which is common).
Related