ANTLR4 Lexer Not Recognizing Comment Token

Viewed 29

So I have been trying to write an example grammar for myself from scratch based on a mix between (mainly) the Lua grammar and C Grammar in the official ANTLR repository. I'm trying to better understand how everything works, but I can't seem to figure out my first issue with running my grammar. The issue is that I have a comment Lexer rule, but it doesn't seem to get recognized at all. When I run my test file that starts with comments, it instantly gives me the parser error below:

Parser error (2, 1): mismatched input '//' expecting {<EOF>, 'break', 'while', 'if', 'fields', 'keys', 'vals', 'func', 'funcdoc', 'return', NAME},<unknown>,2,0,true

Tokens:[@0,81:82='//',<52>,2:0], [@1,159:160='//',<52>,4:0], [@2,381:382='//',<52>,8:0]

The beginning of my test file is displayed below:

///////////////////////////////////////////////////////////////////////////////
//
// group tests
//
// All of these should be very high level
//
///////////////////////////////////////////////////////////////////////////////

// Common Master funcs
master_funcs

I've tried putting comments in my stat rule and tried moving my COMMENT Lexer rule in front of all the other rules, apart from also trying to change little things about the rules. But my thought is, since my Lexer rules don't seem to be doing a whole lot, there's not much I could see going wrong. I'm thinking maybe it's some sort of unrelated problem with how I created my parser rules. However, I'm new to ANTLR and all, so clearly I don't really know what I'm talking about. Any help would be appreciated on this.

My grammar is displayed below:

grammar example; 

///////////////////////////
//      PARSER RULES     //
///////////////////////////

//starting rule
chunk 
    : block EOF
    ;

block 
    : stat* retstat?
    ;

//statements
stat
    : 'break'
    | funcdef
    | decblock 
    | varlist '=' explist
    | 'while' '(' exp ')' block 'endwhile'
    | 'if' ('(' exp ')'|exp) block ('elseif' ('(' exp ')'|exp) block)* ('else' block)? 'endif'
    ;

precompstat
    : '@define' NAME number
    | '@if' ('(' exp ')'|exp) block ('@elseif' ('(' exp ')'|exp) block)* ('@else' block)? '@endif'
    ;

//declaration blocks
decblock
    : 'fields' fieldlist? 'endfields'
    | 'keys' keylist? 'endkeys'
    | 'vals' NAME vallist? 'endvals'
    ;

exp
    : 'NULL' | 'TRUE' | 'FALSE'
    | tabledec
    | number
    | STRING
    | funccall
    | <assoc=right> exp opPower exp
    | opUnary exp
    | exp opMulDivMod exp
    | exp opAddSub exp
    | exp opComparison exp
    | exp opAnd exp
    | exp opOr exp
    | exp opBitwise exp
    ;

funcdef
    : 'func' '(' args? ')' funcbody
    | funcdoc 'func' '(' args? ')' funcbody
    ;

args
    : (explist | namelist | fieldlist | funccall | tableconstruct)+
    ;

funcbody
    : ('locals' namelist)? 'does' block 'endfunc'
    ;

funccall
    : NAME '(' args? ')' NAME
    ;

funcdoc
    : 'funcdoc' NAME STRING
    ;

explist
    : exp (',' exp)*
    ;

fieldlist
    : NAME ':' ('[]')? NAME (number)? (',' fieldlist)?
    ;

funclist
    : funccall ('(' args? ')')? (',' funclist)?
    ;

keylist
    : NAME '(' (explist|namelist|varlist)(',' (explist|namelist|varlist))* ')' 'comment' STRING (',' keylist)?
    ;

namelist
    : NAME (',' NAME)*
    ;

vallist
    : NAME (number|NAME|exp) (',' vallist)?
    ;

varlist
    : NAME ('[' (NAME|number) ']')? ('[' number (',' number)* ']') (',' varlist)?
    ;

//return statement
retstat
    : 'return' explist?
    ;

number
    : INT | HEX | FLOAT
    ;

tabledec
    : NAME ':' '[]' NAME 'new' '[]' NAME tableconstruct
    ;

tableconstruct
    : '{' (explist|namelist|funclist)* '}' | '(' (explist|namelist|funclist)* ')'
    ;

opOr
    : 'or'
    ;

opAnd
    : 'and'
    ;

opComparison
    : '<' | '>' | '<=' | '>=' | '=='
    ;

opAddSub
    : '+' | '-'
    ;

opMulDivMod
    : '*' | '/' | '%' | '//'
    ;

opBitwise
    : '&' | '|' | '~' | '<<' | '>>'
    ;

opUnary
    : 'not' | '#' | '-' | '~'
    ;

opPower
    : '^'
    ;

///////////////////////////
//      LEXER RULES      //
///////////////////////////

INT
    : Digit+
    ;

HEX
    : '0' [xX] HexDigit+
    ;

FLOAT
    : Digit+ '.' Digit*
    | '.' Digit+
    ;

NAME
    : [a-zA-Z_][a-zA-Z_0-9]*
    ;

STRING
    : '"' (EscapeSequence|~('\\'|'"'))* '"'
    ;

fragment
Digit
    : [0-9]
    ;

fragment
EscapeSequence
    : '\\' [abfnrtvz"'\\]
    | '\\' '\r'? '\n'
    ;

fragment
HexDigit
    : [0-9a-fA-F]
    ;

WS
    : [ \t\r\n]+ -> skip
    ;

COMMENT
    :   '//' ~[\r\n]*
        -> skip
    ;

Thank you!

0 Answers
Related