Terminating blocks of multiline text in Xtext/ANTLR

user_2108965 0
Viewed 84

I am starting to get a handle on Xtext, but I am still having a bit of trouble seperating semantic sections by multiple newlines.

grammar org.example.dsl.MyDsl hidden(WS)

import "http://www.eclipse.org/emf/2002/Ecore" as ecore
generate words "http://www.example.org/dsl

Document:
    sections+=Paragraph+
;

Paragraph:
    lines+=Text+
    ( -> NL)
;

Text: 
    value=WordGroup
    NL
;

WordGroup: SIMPLE_WORD+;

terminal SIMPLE_WORD: 
    ('0'..'9' | 'a'..'z' | 'A'..'Z') 
    ('0'..'9' | 'a'..'z' | 'A'..'Z' | '-' | '_' | '.')*
;
terminal NL: ('\r'? '\n');
terminal WS: (' ' | '\t');

This is ok...

    @Test
    def void happyPath() {
        val model = parseHelper.parse('''
            The quick brown fox
            Jumps over the lazy dog

        ''')

        assertThat(model, notNullValue())
        assertThat(model.eResource.errors, equalTo(#[]))
        assertThat(model.sections.size(), equalTo(1))
        assertThat(model.sections.get(0).lines.size(), equalTo(2))
        // It works!
    }

ANTLRWorks Parser Happy

But this is not...

    @Test
    def void noTrailingNewlines() {
        val model = parseHelper.parse('''
            The quick brown fox
            Jumps over the lazy dog
        ''')

        assertThat(model, notNullValue())
        assertThat(model.eResource.errors, equalTo(#[]))
        // Fail ^^^ XtextSyntaxDiagnostic: null:2 mismatched input '<EOF>' expecting RULE_NL
        assertThat(model.sections.size(), equalTo(1))
        assertThat(model.sections.get(0).lines.size(), equalTo(2))
    }

ANTLRWorks Parser Fail

Both should be valid parsable text, but I can't get it to accept a single NL, if that is the last char found.

I tried the obvious ( -> NL?)...

Paragraph:
    lines+=Text+
    ( -> NL?)
;

...And This actually does cause the tests to pass, Only now I have ANTLR warnings.

ANTLRWorks Parser Alternatives

And as I suspected, this just kicks the problem down the road...

    @Test
    def void multipleParagraphs() {
        val model = parseHelper.parse('''
            The quick brown fox
            Jumps over the lazy dog

            But only on days that end in Y
        ''')

        assertThat(model, notNullValue())
        assertThat(model.eResource.errors, equalTo(#[]))
        assertThat(model.sections.size(), equalTo(2)) //Expected: <2> but: was <1>
    }
xtext antlr
1 Answers

And we have a winner!

Many thanks to @christian-dietrich and @redknite who where incredibly patient with me as they helped me work through the problem.

Document:
    sections+=Paragraph
    (NL sections+=Paragraph)*
;

Paragraph:
    lines+=Text+
;

Text: 
    value=WordGroup
    NL
;

@Test
def void singleParagraph() {
    val model = parseHelper.parse('''
        The quick brown fox
        Jumps over the lazy dog
    ''')

    assertThat(model, notNullValue())
    assertThat(model.eResource.errors, equalTo(#[]))

    val doc = model.document
    assertThat(doc, notNullValue())
    assertThat(doc.sections.size(), equalTo(1))
    assertThat(doc.sections.get(0).lines.size(), equalTo(2))
}

@Test
def void multiParagraph() {
    val model = parseHelper.parse('''
        The quick brown fox
        Jumps over the lazy dog

        But only on days that end in Y
    ''')

    assertThat(model, notNullValue())
    assertThat(model.eResource.errors, equalTo(#[]))

    val doc = model.document
    assertThat(doc, notNullValue())
    assertThat(doc.sections.size(), equalTo(2))
    assertThat(doc.sections.get(0).lines.size(), equalTo(2))
}

Both pass!

user_21089650
Related