I am working on a language parser to allow for translation (automated) and message selection based on a desired language. So I am parsing my raw script files looking for a function named prep.msg and then I decompose the elements that need to be translated.
For example, let's say a file had the following code:
if( ( TYPE == "EXP" || TYPE == "POW") && any(df$y<=0) )
{
msg = prep.msg(
"\n\n\t\t\t",
"For \"TRENDLINE\" type [",type,'] you can\'t have any data',
"\n\t\t\t\t",
"(currently [",n,"] rows) in [y] that is <= 0 (less than or equal to zero) ...",
"\n\t\t\t\t",
"the \\\"*SYSTEM*\\\" is REMOVING ROWS and trying to COMPUTE",
"\n\n");
You can verify that it is valid parsed code by creating a function:
prep.msg = function(...) { unlist(list(...)) }
You will note I have some edge cases: DQ (double quote) inside DQ, SQ (single quote) inside SQ, and some BACKSLASH elements around the *SYSTEM* element.
Here is what readLines does to the file content when I call the function:
[1] ""
[2] "if( ( TYPE == \"EXP\" || TYPE == \"POW\") && any(df$y<=0) )"
[3] "\t\t{"
[4] "\t\tmsg = prep.msg("
[5] "\t\t\t\"\\n\\n\\t\\t\\t\", "
[6] "\t\t\t\"For \\\"TRENDLINE\\\" type [\",type,'] you can\\'t have any data', "
[7] "\t\t\t\"\\n\\t\\t\\t\\t\", "
[8] "\t\t\t\"(currently [\",n,\"] rows) in [y] that is <= 0 (less than or equal to zero) ...\", "
[9] "\t\t\t\"\\n\\t\\t\\t\\t\", "
[10] "\t\t\t\"the \\\\\\\"*SYSTEM*\\\\\\\" is REMOVING ROWS and trying to COMPUTE\", "
[11] "\t\t\t\"\\n\\n\");"
[12] "\t\t\t"
I have been able to recover the \\n and \\t with a simple string replace function (e.g., gsub fixed).
"\\n", "\n"
"\\t", "\t"
and the resulting output behaves except for the *SYSTEM* edge case:
> parse.lang(lines)
idx line.no char.s char.e quote what content
1 1 5 14 18 DQ -.EMPTY.- \n\n\t\t\t
2 2 6 14 37 DQ -.TRANSLATE.- For "TRENDLINE" type [
3 3 6 40 43 DQ -.OBJECT.- type
4 4 6 46 71 SQ -.TRANSLATE.- ] you can't have any data
5 5 7 14 18 DQ -.EMPTY.- \n\t\t\t\t
6 6 8 14 25 DQ -.TRANSLATE.- (currently [
7 7 8 28 28 DQ -.OBJECT.- n
8 8 8 31 90 DQ -.TRANSLATE.- ] rows) in [y] that is <= 0 (less than or equal to zero) ...
9 9 9 14 18 DQ -.EMPTY.- \n\t\t\t\t
10 10 10 14 72 DQ -.TRANSLATE.- the \\\\"*SYSTEM*\\\\" is REMOVING ROWS and trying to COMPUTE
11 11 11 14 15 DQ -.EMPTY.- \n\n
That is, I can recognize a SQ or DQ is within a string. I am trying to figure out how to get the BACKSLASH case to work. If I use one of the BACKSLASH for the DQ in *SYSTEM*, it seems like the escaped DQ is being counted as a standard BACKSLASH.
I am trying to figure out a string replace technique before it enters the parser or a string replace technique after it enters the parser so the BACKSLASH count will work as expected.
Question: how to manipulate lines from readLines to account for the *SYSTEM* edgecase?
Minimal Case
str = "the \\\"*SYSTEM*\\\" is REMOVING ROWS and trying to COMPUTE";
# lines = readLines("parse/parse-lang-simple.txt");
# line = lines[10];
# dput(line);
line = " \"the \\\\\\\"*SYSTEM*\\\\\\\" is REMOVING ROWS and trying to COMPUTE\", ";
where str is from the RAW file (inside double quotes) and line is what readLines grabs (including the whitespace).
How to manipulate line so it is identical to str?
GOAL
nline = FUN(line);
identical(nline,str);
where FUN represents a function to manipulate the line.