difficulty getting c-style comments in flex/lex

Viewed 29474

I want to make a rule in flex to consume a c-style comment like /* */

i have the following

c_comment "/*"[\n.]*"*/"

But it doesn't ever get matched. Any idea why? if you need more of my code please let me know and I'll submit the whole thing. Thanks to anyone who replies.

9 Answers

I suggest you use start conditions instead.

%x C_COMMENT

"/*"            { BEGIN(C_COMMENT); }
<C_COMMENT>"*/" { BEGIN(INITIAL); }
<C_COMMENT>\n   { }
<C_COMMENT>.    { }

Do note that there must not be any whitespace between the <condition> and the rule.

%x C_COMMENT defines the C_COMMENT state, and the rule /* has it start. Once it's started, */ will have it go back to the initial state (INITIAL is predefined), and every other characters will just be consumed without any particular action. When two rules match, Flex disambiguates by taking the one that has the longest match, so the dot rule does not prevent */ from matching. The \n rule is necessary because a dot matches everything except a newline.

The %x definition makes C_COMMENT an exclusive state, which means the lexer will only match rules that are "tagged" <C_COMMENT> once it enters the state.

Here is a tiny example lexer that implements this answer by printing everything except what's inside /* comments */.

Not sure why it's not being picked up but I do know that a pattern of that sort can produce large lexical elements. It's more efficient to detect just the start comment marker and toss everything in the bitbucket until you find the end marker.

This site has code which will do that:

"/*" {
    for (;;) {
        while ((c = input()) != '*' && c != EOF)
            ; /* eat up text of comment */
        if (c == '*') {
            while ((c = input()) == '*')
                ;
            if (c == '/')
                break; /* found the end */
        }
        if (c == EOF) {
            error ("EOF in comment");
            break;
        }
    }
}

There's a worked example in the Flex manual, which gets the gnarly edge cases right:

<INITIAL>"/*"         BEGIN(IN_COMMENT);
<IN_COMMENT>"*/"      BEGIN(INITIAL);
<IN_COMMENT>[^*\n]+   // eat comment in chunks
<IN_COMMENT>"*"       // eat the lone star
<IN_COMMENT>\n        yylineno++;

Another example:

"/*"([^*]*|(\*+[^/]))*"*/"
Related