I am getting a code generator to produce source maps that tracks line/column based on Unicode code-points with lines broken by (LF, CR, CRLF). I worry about other line-breaks embedded in comments and supplementary characters might cause source-map consumers to disagree on which part of a source text a (line, column) pair references.
Specifically, I am confused by the source-map v3 specification's use of terms like
the zero-based starting line in the original source
the zero-based starting column of the line in the source represented
I imagine that these numbers are used by programmer tools to navigate to/highlight chunks of code in the original source.
Problems due to different line break definitions
- JavaScript recognizes CR, LF, CRLF, U+2028, U+2029 as line breaks
- Rust recognizes those and U+85 though or only recognizes LF and CRLF as breaks depending on how you slice the grammar.
- Java only treats CR, LF, CRLF as line breaks.
- Languages that follow Unicode TR #13 might additionally treat VT and FF as line breaks.
So if a source-map generator treats U+0085 (for example) as a newline and a source-map consumer does not, might they disagree about where a (source-line, source-column) pair points?
Problems due to differing column definitions.
Older versions of JavaScript defined a source text as UTF-16 code-units, suggesting that a column count is the number of UTF-16 code-units since the end of the last line-break.
ECMAScript source text is represented as a sequence of characters in the Unicode character encoding, version 2.1 or later, using the UTF-16 transformation format.
But the current spec does not describe source texts in terms of UTF-16:
SourceCharacter::
any Unicode code point
Are column counts likely to be thrown off if source-map consumers differently treat supplementary characters as occupying one code-point column or two UTF-16 columns?
For example, since '' is U+1d12C, a supplementary code-point encoded using two UTF-16 code-units, might column counts disagree for a line like
let = "" /* */ + ""
Is the + symbol at column 20 (zero-indexed by code-point) or column 23 (zero-indexed by UTF-16 code-unit)?
Am I missing something in the specification that clarifies this, or is there a de facto rule used by most source-map producers/consumers?
If these are problems, are there known workarounds or best practices when tracking line/column counts for source languages that translate to JS?
I will probably have to reverse-engineer what implementations like Mozilla's source-map.js or Chrome's dev console do, but thought I'd try and find a spec reference so I know whom to file bugs against and who is correct.