eliminating embedded actions from antlr4 grammar

Viewed 1490

I have an antlr grammar in which embedded actions are used to collect data bottom up and build aggregated data structures. A short version is given below, where the aggregated data structures are only printed (ie no classes are created for them in this short sample code).

grammar Sample;
top returns [ArrayList l]
    @init { $l = new ArrayList<String>(); }
    : (mid { $l.add($mid.s); } )* ;
mid returns [String s]
    : i1=identifier 'hello' i2=identifier
        { $s = $i1.s + " bye " + $i2.s; }
    ;
identifier returns [String s]
    : ID { $s = $ID.getText(); } ;
ID  : [a-z]+ ;
WS  : [ \t\r\n]+ -> skip ;

Its corresponding Main program is:

public class Main {
  public static void main( String[] args) throws Exception 
  {
    SampleLexer lexer = new SampleLexer( new ANTLRFileStream(args[0]));
    CommonTokenStream tokens = new CommonTokenStream( lexer );
    SampleParser parser = new SampleParser( tokens );
    ArrayList<String> top = parser.top().l;
    System.out.println(top);
  }
}

And a sample test is:

aaa hello bbb
xyz hello pqr

Since one of the objectives of antlr is to keep the grammar file reusable and action-independent, I am trying to delete the actions from this file and move it to a tree walker. I took a first stab at it with the following code:

public class Main {
  public static void main( String[] args) throws Exception 
  {
    SampleLexer lexer = new SampleLexer( new ANTLRFileStream(args[0]));
    CommonTokenStream tokens = new CommonTokenStream( lexer );
    SampleParser parser = new SampleParser( tokens );
    ParseTree tree = parser.top();
    ParseTreeWalker walker = new ParseTreeWalker();
    walker.walk( new Walker(), tree );
  }
}
public class Walker extends SampleBaseListener {
  public void exitTop(SampleParser.TopContext ctx ) {
    System.out.println( "Exit Top : " + ctx.mid() );
  }
  public String exitMid(SampleParser.MidContext ctx ) {
    return ctx.identifier() + " bye "; // ignoring the 2nd instance here
  }
  public String exitIdentifier(SampleParser.IdentifierContext ctx ) {
    return ctx.ID().getText() ;
  }
}

But obviously this is wrong, because at the least, the return types of the Walker methods should be void, so they dont have a way to return aggregated values upstream. Secondly, I dont see a way how to access the "i1" and "i2" from the walker code, so I am not able to differentiate between the two instances of "identifier" in that rule.

Any suggestions on how to separate the actions from the grammar for this purpose?

Should I use a visitor instead of a listener here, since the visitor has the capability of returning values? If I use a visitor, how do I solve the problem of differentiating between "i1" and "i2" (as mentioned above)?

Does a visitor perform its action only at the exit of a rule (unlike the listeners, which exist for both entry and exit)? For example, if I have to initialize the list at the entry of rule "top", how can I do it with a visitor, which executes only at the conclusion of a rule? Do I need a enterTop listener for that purpose?

EDIT: After the initial post, I have modified the rule "top" to create and return a list, and pass this list back to the main program for printing. This is to illustrate why I need an initialization mechanism for the code.

2 Answers
Related