Is it possible to preserve anchors, comments, and references when unmarshaling data structures back into YAML in Go?

user_7157204 0
Viewed 973

Is it possible to preserve anchors and so forth when unmarshaling data structures back into YAML in Go?

These related questions below pertained to the same concept using Python, which seems do-able using the ruyamel.yaml package and according to the doc https://yaml.readthedocs.io/en/latest/example.html it seems possible to preserve comments, anchors, and references. This is referred to as a "round-trip".

Using the example code in the README.md in https://github.com/go-yaml/yaml, I'm able to read yaml file into data structures. Then I derived a further example using anchors and references for a condensed document. But the anchoring and so forth is then lost when dumping back to YAML. Is it possible to do the same in Golang?

go yaml
1 Answers

go-yaml has a rather minimal interface, as it is typical for Go packages. Since the lowest possible structure you can access (via Marshaler / Unmarshaler interface) is yaml.Node, it is not possible to implement round-tripping with it, without changing go-yaml itself (just like ruamel is a fork of PyYAML).

For proper round tripping, you need:

  • Preservation of comments and whitespace. This needs to happen at the lexer level, a level very few YAML implementations cover externally (libyaml is the only one I know that does it, but you can't feed lexer tokens back into it so it's basically useless for that purpose)
  • Preservation of scalar presentation. This is the hardest to get right since quoted scalars allow you to escape any unicode character and if you only have the resulting string, you cannot know the original presentation. I think not even ruamel is able to reproduce scalar representation. Scalar representation also includes where line breaks are in a folded scalar, chomping and indentation indicators, none of which are exposed in go-yaml.
  • Preservation of anchors & aliases. go-yaml actually lets you access this with Node.Anchor however you need to figure out which was the first node that has the anchors, and which are the others that were originally aliases. This is difficult because…
  • Preservation of key order. YAML defines that the order of keys in a mapping must not convey content information. Typically, mappings are loaded in a hashmap which loses this information (as is the case with Go's map). However, Node does provide the original order of child items, so this is doable in go-yaml.
  • Preservation of document header & footer. YAML allows directives before a document (%YAML and %TAG) and also allows documents to end with multiple ... lines (and comments in between). This information is not available with go-yaml.

Since round-tripping goes against the YAML spec if interpreted strictly (because it disallows presentation details to have any impact on the processing), it is usually not part of a YAML implementation and afaik ruamel is the only implementation that attempts to do it.

So the answer is: Of course it's possible to do it in Go, but you need to implement it yourself, possibly by forking go-yaml. Since parsing YAML is very complicated by itself (see here, not one implementation gets it completely right), a proper YAML round-tripping implementation the covers all edge cases will be very complex to write and I strongly advise against it.

user_3479640
Related