I'm trying to parse boolean expressions into an Expr tree using Scala's Packrat parsers from scala-parser-combinators.
sealed trait Expr
case class Term(term: String) extends Expr
case class And(x: Expr, y: Expr) extends Expr
case class Or(x: Expr, y: Expr) extends Expr
aaa and bbb --> And(Term(aaa),Term(bbb))
aaa and bbb or ccc --> Or(And(Term(aaa),Term(bbb)),Term(ccc))
aaa and (bbb or ccc) --> And(Term(aaa),Or(Term(bbb),Term(ccc)))
This grammar seems to work perfectly fine:
object Parsing extends RegexParsers with PackratParsers {
override val skipWhitespace = false
val validIdentifiers = List("aaa", "bbb", "ccc", "ddd")
lazy val term: PackratParser[Term] = """\s*""".r ~> """\w+""".r flatMap { identifier =>
if (validIdentifiers.contains(identifier))
success(Term(identifier))
else
err(s"expected one of: $validIdentifiers")
}
lazy val and: PackratParser[And] =
expr ~ """\s+and\s+""".r ~ (term | parensExpr) ^^ { case e1 ~ _ ~ e2 => And(e1, e2) }
lazy val or: PackratParser[Or] =
expr ~ """\s+or\s+""".r ~ (term | parensExpr) ^^ { case e1 ~ _ ~ e2 => Or(e1, e2) }
lazy val parensExpr: PackratParser[Expr] = """\s*\(""".r ~> expr <~ """\s*\)""".r
lazy val expr: PackratParser[Expr] =
term ||| and ||| or ||| parensExpr
lazy val root: PackratParser[Expr] =
phrase(expr)
def parseExpr(input: String): ParseResult[Expr] =
parse(root, new PackratReader(new CharSequenceReader(input)))
}
But error messages are sometimes.... bad.
If the parser finds an invalid identifier on the left-hand side of an and, it'll correctly tell us so.
println(parseExpr("invalidIdentifier and aaa"))
[1.18] error: expected one of: List(aaa, bbb, ccc, ddd)
invalidIdentifier and aaa
^
But if it finds an invalid identifier on the right-hand side of an and, it'll give us this misleading error message.
println(parseExpr("aaa and invalidIdentifier"))
[1.4] failure: end of input expected
aaa and invalidIdentifier
^
I'm pretty sure this happens because expr will try all 4 options: and/or/parensExpr will fail, but term will succeed with just Term("aaa").
Then, root's phrase will kick in and check whether there's any input left to consume, and fail because there is: " and invalidIdentifier".
So I thought, I'll push phrase one level down. In other words, I changed this:
lazy val expr: PackratParser[Expr] =
term ||| and ||| or ||| parensExpr
lazy val root: PackratParser[Expr] =
phrase(expr)
Into this:
lazy val expr: PackratParser[Expr] =
term ||| and ||| or ||| parensExpr
lazy val root: PackratParser[Expr] =
phrase(term) ||| phrase(and) ||| phrase(or) ||| phrase(parensExpr)
Now, all 4 options should fail, but we should see and's error message because and consumed more input than the other 3 options
I'm now getting better error messages BUT, to my surprise, some previously valid inputs are now invalid!
println(parseExpr("aaa or bbb"))
[1.4] failure: string matching regex '\s+and\s+' expected but ' ' found
aaa or bbb
^
println(parseExpr("aaa and bbb or ccc"))
[1.12] failure: end of input expected
aaa and bbb or ccc
^
I don't understand why.
In fact, even just a simpler, trivial change such as this:
// before
lazy val root: PackratParser[Expr] =
phrase(expr)
// after
lazy val root: PackratParser[Expr] =
phrase(term ||| and ||| or ||| parensExpr)
... breaks previously valid inputs.
How come? Shouldn't these two definitions of root be equivalent? Are these parsers not referentially transparent?
More importantly, how should I go about fixing this?