Regexp::Grammars Succeeds with Debug and Fails Without

user_4651077 0
Viewed 77

I have a Regexp::Grammars grammar that fails to match, but succeeds when I turn on debug:

#!/usr/bin/env perl

use feature 'say';

my $parser = do {
  use Regexp::Grammars;
  qr{
#    <debug:step>

    \A <command> \Z

    <token: command>
       <.identifier>+ %% [:]{2} 

    <token: identifier>
       [a-zA-Z0-9_]+
  };
};

my $input = 'foo::bar';

if ($input =~ $parser) {
  use Data::Dumper; say Data::Dumper::Dumper(\%/);
}
else {
  say 'Parse failed!';
}

$ ./grammar_problem.pl
Parse failed!

Turning on debug gets me:

#!/usr/bin/env perl

use feature 'say';

my $parser = do {
  use Regexp::Grammars;
  qr{
    <debug:step>

    \A <command> \Z

    <token: command>
       <.identifier>+ %% [:]{2} 

    <token: identifier>
       [a-zA-Z0-9_]+
  };
};

my $input = 'foo::bar';

if ($input =~ $parser) {
  use Data::Dumper; say Data::Dumper::Dumper(\%/);
}
else {
  say 'Parse failed!';
}

$ ./grammar_problem.pl
=======> Trying <grammar> from position 0
foo::bar |...Trying subpattern /\A/ 
         |    \_____subpattern /\A/ matched ''  
         |...Trying <command>   
         |   |...Trying <.identifier>   
         |   |   |...Trying subpattern /[a-zA-Z0-9_]+/  
::bar    |   |   |    \_____subpattern /[a-zA-Z0-9_]+/ matched 'foo'    
         |   |    \_____<.identifier> matched 'foo' 
         |   |...Trying subpattern /[:]{2}/ 
bar      |   |    \_____subpattern /[:]{2}/ matched '::'    
         |   |...Trying <.identifier>   
         |   |   |...Trying subpattern /[a-zA-Z0-9_]+/  
[eos]    |   |   |    \_____subpattern /[a-zA-Z0-9_]+/ matched 'bar'    
         |   |    \_____<.identifier> matched 'bar' 
         |   |...Trying subpattern /[:]{2}/ 
         |   |    \FAIL subpattern /[:]{2}/
         |   |...Trying subpattern /[:]{2}/ 
         |   |    \FAIL subpattern /[:]{2}/
         |    \_____<command> matched 'foo::bar'    
         |...Trying subpattern /\Z/ 
         |    \_____subpattern /\Z/ matched ''  
$VAR1 = {
          '' => 'foo::bar',
          'command' => 'foo::bar'
        };

Any idea what's going on here? Thanks! Stack Overflow says I need more details to submit this, but I don't know what else to say. Typing stuff to get past this. My apologies to the reader!

perl grammar parsing regex
1 Answers

I think this is related to how the %% operator works (and is a brilliant little tool). You need to group the sub-rule after it:

  qr{
    (?: \A <command> \Z )

    <rule: command>
       <.identifier>+ %% (::)

    <rule: identifier>
       [a-zA-Z0-9_]+
  }x;

The Regexp::Grammars module is very powerful, but you also need to get things exactly as they are specified (which isn't always clear).

Look for this phrase in the docs:

Note that, if a pattern is used to specify the separator, it must be specified in some kind of matched parentheses.

Now, the issue with turning on debug is interesting. That inserts code all over your pattern, so my guess is that however that happens, the subpattern you were using got some parens around it (probably with debugging code in them too). That satisfied the grammar's, well, grammar and it worked.

brian_d_foy_27661760
Related