Why isless(0 , missing) returns true but 0<missing returns missing in Julia?

user_11747148 0
Viewed 77

I encountered a strange behavior from Julia today, and that's when I tried to compare missing with 0! If you try this 0<missing in REPL:

julia> 0<missing
missing

But if you try to use pre-built isless function like this:

julia> isless(0 , missing)
true

It's strange because 0<missing means 0 is less than missing same as isless(0 , missing). According to isless documentation:

isless(t1::Tuple, t2::Tuple)
Returns true when t1 is less than t2 in lexicographic order.

Shouldn't 0<missing return true as well?

julia
2 Answers

missing < 0 produces missing because < is a "standard" comparison operator.

Now consider a question: is missing less than 0? The answer is: we do not know. missing stands for some unknown value - it can be greater or less than 0. Therefore missing < 0 produces missing.

As you can see in the documentation of <:

this operator implements a partial order.

This means that it does not guarantee that all values are comparable. A similar (but not identical) situation is with NaN:

julia> NaN < 0.0
false

julia> NaN > 0.0
false

julia> NaN == 0.0
false

Also notice that although -0.0 and 0.0 are different floating point values we have:

julia> -0.0 < 0.0
false

Now let us discuss isless. It is needed because sometimes you want an operator that defines total order, as you can see in its documentation:

Test whether x is less than y, according to a fixed total order (defined together with isequal).

This is useful in cases you want to make sure that you can safely assume that every value is comparable. For example sort uses isless.

Now in order to make sure isless defines total order we need to define how missing is positioned in this total order. It was decided that missing is larger than any other value. Therefore isless(missing, 0) is false. Of course this is an arbitrary decision. It was made so that e.g. when you sort a vector missing is put at the end:

julia> sort([1, missing, 2])
3-element Vector{Union{Missing, Int64}}:
 1
 2
  missing

which is what people usually want.

Now notice the behavior of NaN and -0.0:

julia> isless(NaN, missing)
true

julia> isless(Inf, NaN)
true

julia> isless(-0.0, 0.0)
true

So as you can see NaN is greater than Inf but less than missing and -0.0 is less than 0.0. Again - since isless is a total order so distinct values have to be put in a distinct place in this total order.

user_12695670

From the doc:

Standard equality and comparison operators follow the propagation rule presented above: if any of the operands is missing, the result is missing.

julia> missing < 1
missing

julia> 2 >= missing
missing

And then, a bit below:

The isless operator is another exception: missing is considered as greater than any other value. This operator is used by sort, which therefore places missing values after all other values.

julia> isless(1, missing)
true

julia> isless(missing, Inf)
false

julia> isless(missing, missing)
false

So this seems to be by design:
https://docs.julialang.org/en/v1/manual/missing/#Propagation-of-Missing-Values

Now if your question is why this has been designed like this, then perhaps Julia's designers will stop by to enlighten us :)

user_29835680
Related