I want to parse a document treating each line as a distinct token. A toy example might look something like this:
August Pizza Sales
2021-08-21
22
19
53
P.S. I <3
Lines might have arbitrary whitespace before and after the text. (My real case is more complicated, and might have different types of data sections, headers, etc. of arbitrary length.)
Here's a basic attempt:
import java.time.LocalDate
import scala.util.parsing.combinator.RegexParsers
case class Document(
title: String,
date: LocalDate,
data: Seq[Integer],
)
class LineParser extends RegexParsers {
private val eol = sys.props("line.separator")
def text: Parser[String] = ".*".r ^^ { _.toString }
def date: Parser[LocalDate] = ".*".r ^^ { LocalDate.parse(_) }
def blank: Parser[String] = "^\\s*$".r ^^ { _.toString }
def datum: Parser[Integer] = "[0-9]+".r ^^ { _.toInt }
def data: Parser[Seq[Integer]] = repsep(datum, eol)
def document: Parser[Document] = text ~ date ~ blank ~ data ~ blank ~ text ^^ {
case title ~ date ~ _ ~ data ~ _ ~ footer =>
Document(title=title, date=date, data=data, footer=footer)
}
}
When I run this, I get Document(Pizza Sales,2021-08-21,List(23),19).
There are many things I don't understand here ...
Tactical Questions
- What changes do I need to make to correctly parse the document?
- If I change my definition of
blankto"^\\s*$".r(which is what should actually match a blank line, I think) then everything breaks. Why?
Strategic Questions
- What's really going on with newlines here? The
textparser does not match across newlines, so it seems they're being treated specially (i.e., even if I try to matchtextto the entire document, it only getsPizza Sales.) - What is the impact of setting
skipWhitespace? - What is the impact of setting
val whiteSpace = ...(in combination withskipWhitespace)? - [SUPER BONUS QUESTION] What the heck does
~>mean?
Philosophical Questions
Since I already know how to tokenize my input (i.e., split it on newlines), is RegexParsers (or Scala parser combinators in general) the right tool for this job?