thanks very much - could you briefly explain why it works inside {} ? – jalapic 42 mins ago
The pipe operator modifies how the right-hand-side (RHS) function call is performed in a complex but consistent way. It's important to understand what the magrittr package's pipe operator actually is.
The pipe is an infix function. The expressions to the left hand side (LHS) and RHS of a pipe are both arguments to the %>% function. Other infix functions in base R are the normal arithmetic operators +, -, /, *, %in%, etc. The R interpreter expects that any expression with % as the first and last character of its symbol is an infix function. As a function call, the pipe is free to evaluate its arguments however the author decides.
my_vector %>% rank()
# is like
`%>%`(my_vector, rank())
Although rank() "looks" like a bare function call in the preceding example, R's standard evaluation rules for infix functions treat it as an argument to the infix function %>% when the interpreter reaches it. It does not get immediately evaluated.
The pipe is a function that performs nonstandard evaluation of its arguments. R permits functions to perform standard evaluation or nonstandard evaluation (NSE) of their function arguments. Many popular base functions like with() and subset() use NSE. Functions using only standard evaluation are pretty much referentially transparent - you can replace any argument with its value as evaluated as a bare expression by the interpreter and expect to get the same result, or kind of result if there is some nondeterministic operation in the function. Functions that use NSE are not referentially transparent - if you replace the arguments with their values the function call does not necessarily need to resolve the same way. This allows unusual syntax gymnastics, but can also make functions harder to reason about.
The pipe is unusual in that it is both an infix function and it uses NSE, which is a rare combination. The RHS argument to the %>% function "looks like" it's a bare function call, but it's not. It's the second argument to an infix function and evaluated with NSE- that's why, e.g., you can "pipe into" a "naked" call to rank() and it works, rather than throwing an exception (which is what happens if the interpreter evaluates a call to rank() with no arguments supplied.) The pipe uses its RHS argument to compose a new function call based on it, but never evaluates the RHS with standard rules.
The pipe imposes its own internal rules on how the new function call should be structured. It makes two opinionated design choices:
- The RHS argument to the pipe function must be, or must resolve to, a function.
- The object resolved by evaluating the LHS argument must be supplied as a whole, unmodified argument to one or more of the formal arguments of a new function call that gets composed by modifying the RHS argument.
The pipe has to supply the whole, unmodified LHS object as an argument to this new function call. In non-standardly evaluating the RHS argument, the pipe looks for the placeholder symbol . among the arguments to the RHS function. . is considered to be a reference by the user to the LHS object. If . is not present as a whole argument to the function call on the RHS, the LHS object is implicitly supplied as the positionally first unnamed argument to the function. At this point . has not been matched to a formal argument yet. Completely standard R argument matching rules apply from this point onward - I'll spell them out:
If no other arguments are supplied, . is provided as the only argument. R's default positional argument matching causes the value of . to get matched to the function's first formal argument:
# rank() has three formal arguments.
# In order, these are x, na.last, and ties.method.
my_vector %>% rank()
# is like
rank(my_vector)
# which resolves as if it were
rank(x = my_vector)
# because x is the first formal argument to rank().
**If there are other user supplied arguments to the RHS function call, but no occurrence of ., the pipe composes a new function call where the value of . is supplied as the first of the unnamed arguments to the function. This and any other unnamed arguments you supply to the function are matched to the function's formal arguments using normal positional matching. As usual, named arguments have priority over unnamed arguments when matching:
my_vector %>% rank(TRUE)
# is like
rank(my_vector, TRUE)
# which resolves as if it were
rank(x = my_vector, na.last = TRUE)
# na.last is the first formal argument that does not already
# have an argument matched to it. An invisible `.` is "first" and gets
# matched to the first formal argument. `TRUE` goes to `na.last`, the
# second.
my_vector %>% rank(x = TRUE)
# is like
rank(my_vector, x = TRUE)
# which resolves as if it were
rank(x = TRUE, na.last = my_vector)
# na.last is the first formal argument that does not already
# have an argument matched to it. Named arguments get matched
# before unnamed arguments. `.` is always an unnamed argument when
# not explicitly supplied by the user.
If you do explicitly supply . as one or more arguments to the right hand side function, it will not be used invisibly as the first unnamed argument. You can supply . as a named or unnamed argument and you can control its position. Again, standard argument matching rules are still in force- named arguments are matched to formal arguments before positional matches are determined.
. as an unnamed argument: If you explicitly supply one or more cases of . as an unnamed argument, it will be matched to formal arguments using normal positional matching.
my_vector %>% rank(., TRUE)
# is like
rank(my_vector, TRUE)
# which resolves as if it were
rank(x = my_vector, na.last = TRUE)
my_vector %>% rank(TRUE, .)
# is like
rank(TRUE, my_vector)
# which resolves as if it were
rank(x = TRUE, na.last = my_vector)
my_vector %>% rank(., x = TRUE)
# is like
rank(my_vector, x = TRUE)
# which resolves as if it were
rank(x = TRUE, na.last = my_vector)
# because named arguments get matched before unnamed arguments.
. as a named argument: If you explicitly supply one or more occurrences of . as named arguments, it will be matched to formal arguments with normal named argument match.
my_vector %>% rank(x = .)
# is like, and resolves as if it were
rank(x = my_vector)
my_vector %>% rank(na.last = ., TRUE)
# is like
rank(na.last = my_vector, TRUE)
# which resolves as if it were
rank(x = TRUE, na.last = my_vector)
# . is matched to a specific named formal argument.
# TRUE gets the first unsupplied formal argument, which is this case is
# also the first formal argument)
Arguments that merely contain . are not . for the purposes of determining whether . is implicitly supplied as the first unnamed argument. If you do not explicitly supply . as one or more whole arguments, it gets supplied implicitly as the first unnamed argument. However, arguments that contain . still get *evaluated* using the value of the LHS object as .`. Again, this is the fundamental design assumption of the pipe- the LHS object has to get passed unmodified to the RHS function call. If you don't do it explicitly, the pipe still attempts it, implicitly.
my_vector %>% rank(-.)
# is like
rank(my_vector, na.last = -my_vector)
# which resolves as if it were
rank(x = my_vector, na.last = -my_vector)
# THIS IS CRITICAL TO UNDERSTANDING THE PIPE!!!
my_vector %>% rank(x = -.)
# is like
rank(my_vector, x = -my_vector)
# which resolves as if it were
rank(x = 0.5 * my_vector, na.last = my_vector)
OP's specific case: To sum it up, %>% in an infix function call that uses nonstandard evaluation. It tries to supply the LHS object to the RHS function. However, it only looks for an argument to the RHS that has the bare value . when determining whether it will invisibly supply the LHS object as the first unnamed argument. It will not recognize the expression -. as an occurrence of the LHS object for the purposes of the determining whether to supply the LHS object invisibly. It does recognize it as an additional occurrence of the object for the purposes of composing the evaluated expression. That's why OP gets such surprising results - no whole instance of . is supplied as an argument, so . is matched to the first formal argument, and the unnamed argument -. is matched to the second formal argument:
my_vector %>% rank(-.)
# is like
rank(x = my_vector, na.last = -my_vector)
Now, why does this even "work" at all, and give an unexpected answer? Why doesn't it throw an exception? From reading ?rank it looks like a vector should not be a supported argument to to formal argument na.last. However, the formal argument na.last gets resolved in a pretty weird way. The argument accepts a lot of possible legal arguments - booleans, NA, and the character string "keep" are all cited. In the function definition for rank(), na.last is handled using the clause:
NAkeep <- (na.last == "keep")
if (NAkeep || na.last) {
yy[!nas] <- y
if (!NAkeep)
yy[nas] <- (length(y) + 1L):length(yy)
}
else {
len <- sum(nas)
yy[!nas] <- y + len
yy[nas] <- seq_len(len)
}
^ (The above is very sloppy and I urge you not to pull this kind of thing in code you write.)
What does a vector of negative numbers evaluate to? c(-5, -7, -14, -3) is TRUE (well actually -5 is TRUE, it only evaluates the first value) for the purposes of the if operator. Any nonzero, non-NA, non-NaN numeric is evaluated as TRUE by if.
Now, why does my_vector %>% {rank(-.)} seem to work "as expected"?
{ is not an abstract decoration. Like almost every other symbol in R, it's a function call. It has no named formal arguments - instead, it takes ..., which indicates that it can take any number of arguments. When you pipe into { you are supplying your object as the only argument to the function call {. Any expressions evaluated inside {} that has been piped to are evaluated as expressions in an environment where an object named . is available, and not with the NSE evaluation logic that the pipe performs.
Edit - I do not have enough reputation to comment
From @bcarlsen's answer, also c(5,7,14,3) %>% rank(x = -.), which is far more readable. – r2evans 16 mins ago
I shall, without hesitation, parrot YOLO's comment above: this is sorcery. Very good explanation, bcarlsen, it suggests the much simpler c(5,7,14,3) %>% rank(x = -.), which should likely be the punch-line in your detailed answer. – r2evans 15 mins ago
Do not do this. This practice is unsafe. . is still supplied to the second formal argument of rank(). It is only through good luck that this does not have negative effects (since there are no NAs in the vector).
my_vector %>% rank(x = -.)
# is like
rank(x = -my_vector, na.last = my_vector)
