comment structure of yaml in ruamel.yaml

user_5150385 0
Viewed 112

I'm trying to understand the comment structure in ruamel.yaml library so that I can manipulate the comments correctly. What I don't get is why the comments in a .ca are inside a 4 items list? What are those other items and why they are always None?

For example, a comment attached to a sequence is like this with the first being CommentToken and the rest are Nones:

Comment(
  start=None,
  items={
    2: [CommentToken('\n\n########### Foo ###########\n', line: 294, col: 0), None, None, None]
  })

For a comment attached to a map, it seems to always be placed at the 3rd index?

Comment(
  start=None,
  items={
    bar: [None, None, CommentToken('\n\n########## Bar ###########\n', line: 87, col: 0), None]
  })

What's the difference between these and what's the significance of the order of their placements?

ruamel.yaml
1 Answers

It only seems to you that for a YAML mapping the comments are placed in the same position, because you tend put the comments in the same place in your YAML file (i.e. at the end of the line after a value). I did the same when starting ruamel.yaml back in 2014, but over the years issues reported towards package pointed out that some users (of YAML) tend to put their comments in different places (e.g. between a key and a value on the next line, something not originally handled at all).

The significance is where the representer, on dumping the data structure, tries to insert the comments back into the YAML output stream. The code to do that has evolved over the last years to deal with more comment placements that were originally not handled (i.e. comments were lost or replaced on round-trip).

There is no documented API for this, the code that creates the Comment instances is changed in tandem with the code that processes them, so in principle the meaning of any position might change, and those meanings actually has changed in the past. That list structure might also be replaced by a dict with keys that are more indicative of where the comment (stored int he corresponding value) came from/has to be inserted in, than indices. A dict could also do away with the None values indicating empty slots.

I can't guarantee that the source code documentation is up to date but this is what it says:

# map key (mapping/omap/dict) or index (sequence/list) to a  list of
# dict: post_key, pre_key, post_value, pre_value
# list: pre item, post item

IIRC some of these positions are no longer in use.

You should pin the version number of ruamel.yaml you work with. ruamel.yaml follows semantic versioning, but it there is no API for handling comment and it is pre version 1.0, so anything can change at any time. However the minor number tends to be bumped on major (internal) changes or on dropping support for no longer maintained Python versions. So stick with ruamel.yaml<0.18 if you get your code to work with 0.17.21, and test extensively if 0.18 and later will still do what you want.

An API for handling comments will at some point be forthcoming, and apart from dealing with more (exotic) comment placements, also have a way to specify how multi-line comments needs to be handled so they don't necessarily are attached to the preceding node like they are now, but can be assigned to the following node or split between those nodes according to some rule (e.g. first empty line).

user_13079050
Related