Is is possible that I use separators other than comma in Rust's MacroMatch?

user_3316009 0
Viewed 48

In the rust book I see the definition of MacroMatch is like the following

MacroMatch :
      Token except $ and delimiters
   | MacroMatcher
   | $ ( IDENTIFIER_OR_KEYWORD except crate | RAW_IDENTIFIER | _ ) : MacroFragSpec
   | $ ( MacroMatch+ ) MacroRepSep? MacroRepOp

MacroFragSpec :
      block | expr | ident | item | lifetime | literal
   | meta | pat | pat_param | path | stmt | tt | ty | vis

MacroRepSep :
   Token except delimiters and MacroRepOp

MacroRepOp :
   * | + | ?

According to the definition of tokens, I found >> is a token. So, in my understading, we can use all tokens as MacroRepSep except from {}/[]/{}/*/+/?. However, the following codes can't compile, with a error "$a:expr is followed by >>, which is not allowed for expr fragments"

macro_rules! add_list2 {
    ($($a:expr)>>*) => {
        0
        $(+$a)*
    }
}

pub fn main() {
    println!("{}", add_list!(1>>2>>3));
}

Playground.

I wonder why, and if I can use a separator other than ,?

rust
1 Answers

Not sure if you're using a different Rust version, but with your code on the current compiler (1.62) it outputs an error that includes what separators are available:

error: `$a:expr` is followed by `>>`, which is not allowed for `expr` fragments
 --> src/main.rs:2:16
  |
2 |     ($($a:expr)>>*) => {
  |                ^^ not allowed after `expr` fragments
  |
  = note: allowed there are: `=>`, `,` or `;`

The problem with repetitions on exprs is that, since they are so varied, they can easily be ambiguous or could become ambiguous. I'll quote the section on Follow-set Ambiguity Restrictions:

The parser used by the macro system is reasonably powerful, but it is limited in order to prevent ambiguity in current or future versions of the language. In particular, in addition to the rule about ambiguous expansions, a nonterminal matched by a metavariable must be followed by a token which has been decided can be safely used after that kind of match.

As an example, a macro matcher like $i:expr [ , ] could in theory be accepted in Rust today, since [,] cannot be part of a legal expression and therefore the parse would always be unambiguous. However, because [ can start trailing expressions, [ is not a character which can safely be ruled out as coming after an expression. If [,] were accepted in a later version of Rust, this matcher would become ambiguous or would misparse, breaking working code. Matchers like $i:expr, or $i:expr; would be legal, however, because , and ; are legal expression separators.

And it includes the separators available for various fragment specifiers:

  • expr and stmt may only be followed by one of: =>, ,, or ;.
  • pat_param may only be followed by one of: =>, ,, =, |, if, or in.
  • pat may only be followed by one of: =>, ,, =, if, or in.
  • path and ty may only be followed by one of: =>, ,, =, |, ;, :, >, >>, [, {, as, where, or a macro variable of block fragment specifier.
  • vis may only be followed by one of: ,, an identifier other than a non-raw priv, any token that can begin a type, or a metavariable with a ident, ty, or path fragment specifier.
  • All other fragment specifiers have no restrictions.
user_21891300
Related