Antlr 4 Lexer with multiple modes failing to tokenise correctly

Viewed 454

I'm trying to create a lexer with multiple modes using Antlr 4.7. My lexer currently is:

ACTIONONLY  : 'AO'; 

BELIEFS :   ':Initial Beliefs:' -> mode(INITIAL_BELIEFS);
NAME    :   ':name:';
WORD:   ('a'..'z'|'A'..'Z'|'0'..'9'|'_')+;

COMMENT : '/*' .*? '*/' -> skip ;
LINE_COMMENT : '//' ~[\n]* -> skip ;
NEWLINE:'\r'? '\n' -> skip  ;
WS  :   (' '|'\t') -> skip ;

mode INITIAL_BELIEFS;
GOAL_IB :   ':Initial Goal:' -> mode(GOALS);
IB_COMMENT : '/*' .*? '*/' -> skip ;
IB_LINE_COMMENT : '//' ~[\n]* -> skip ;
IB_NEWLINE:'\r'? '\n' -> skip  ;
IB_WS  :   (' '|'\t') -> skip ;
BELIEF_BLOCK: ('a'..'z'|'A'..'Z'|'0'..'9'|'_'|'('|')'|','|'.')+;

mode REASONING_RULES;
R1: 'a';
R2: 'b';

mode GOALS;
GL_COMMENT : '/*' .*? '*/' -> skip ;
GL_LINE_COMMENT : '//' ~[\n]* -> skip ;
GL_NEWLINE:'\r'? '\n' -> skip  ;
GL_WS  :   (' '|'\t') -> skip ;
GOAL_BLOCK: ('a'..'z'|'A'..'Z'|'0'..'9'|'_'|'('|')'|','|'.')+;

Note that there is no way, at present, to get into the REASONING_RULES mode (so this should not, as I understand it have any effect on the operation of the lexer). Obviously I do want to use this mode, but this is the minimal version of the lexer that seems to display the problem I'm having.

My parser is:

grammar ActionOnly;

options { tokenVocab = ActionOnlyLexer; }

// Mas involving ActionOnly Agents
mas  :  aoagents;

aoagents: ACTIONONLY (aoagent)+;

// Agent stuff
aoagent  : 
    (ACTIONONLY?) 
    NAME w=WORD  
    BELIEFS (bs=BELIEF_BLOCK )?
    GOAL_IB gs=GOAL_BLOCK;

and I'm trying to parse:

AO

:name: robot

:Initial Beliefs:

abelief

:Initial Goal:

at(4, 2)

This fails with the error

line 35:0 mismatched input 'at(4,' expecting GOAL_BLOCK

which I'm assuming is because it isn't tokenising correctly.

If I omit rule R2 in the REASONING_RULES mode then it parses correctly (in general I seem to be able to have one rule in REASONING_RULES and it will work, but more than one rule and it fails to match GOAL_BLOCK)

I'm really struggling to see what I'm doing wrong here, but this is the first time I've tried to use lexer modes with Antlr.

1 Answers
Related