ruamel.yaml weird behavior with datetime like values

user_7421250 0
Viewed 175

check.py

import os
from pathlib import Path
import sys
import json

from ruamel.yaml import YAML
yaml = YAML(pure=True)
yaml.sort_base_mapping_type_on_output = None
yaml.indent(mapping=2, sequence=4, offset=2)


def yamlCheck(file: os.PathLike):
    with open(file, 'r', encoding='utf8') as yr:
        data = yaml.load(yr)

    print(data)

    check = json.dumps(data, indent=4, ensure_ascii=False)
    print(check)


if __name__ == "__main__":
    yamlCheck('test.yaml')

test1.yaml, with error.

a:
  - b: 2
  - 123
  - 2000-12-26

test2.yaml, runs ok

a:
  - b: 2
  - 123
  - 111-12-26

test2.yaml, runs ok

a:
  - b: 2
  - 123
  - 2000-12-26111

runs ok means the code runs without error, json.dumps load the OrderedDict data well.

with the output of print(data) I think ruamel.yaml has parse that xxxx-xx-xx formatted data as datetime and construct a datetime object with it, which causes the json.dumps function's error.

So what is the standard of the ruamel.yaml to decide to parse the value as datetime?

ruamel.yaml yaml python
1 Answers

I am not sure what you consider weird behaviour, ruamel.yaml uses a regular expression to implicitly tag a scalar as a timestamp as defined in resolver.py. That regex is adapted from the regular expression in Timestamp Language Independent Type for YAML.

The regex assumes 4 digits for the year and a maximum of two for the month and date. A valid datetime.date or datetime.datetime (e.g. no month 2000-12-44 matches the regex, but is not a valid date) is only checked pretty late in the compositing phase, and when it fails the scalar is assumed to be a string.

So it is normal that 2000-12-26 loads as a date, and that both 111-12-26 and 2000-12-26111 load as strings.

Since json.dumps() doesn't support dumping datetime.date instances, you'll have to explicitly convert the types to strings by specifying default=str, to prevent the error that you are getting.

What is weird, in my opinion, is that you assume json.dumps() loads something ("json.dumps load the OrderedDict data well"). That routine doesn't load, it dumps, as the name implies, and that it can dump OrderedDict is because that is a subclass of dict

It is also inefficiently to print() the result of json.dumps() instead of directly dumping to sys.stdout.

import sys
import json
import ruamel.yaml

yaml_strs = [
"""\
a:
  - b: 2
  - 123
  - 2000-12-26
""",
"""\
a:
  - b: 2
  - 123
  - 111-12-26
""",
"""\
a:
  - b: 2
  - 123
  - 2000-12-26111
"""
]

yaml = ruamel.yaml.YAML()
for yaml_str in yaml_strs:
    data = yaml.load(yaml_str)
    print(type(data['a'][2]))
    try:
        json.dump(data, sys.stdout, default=str)
        print() # since json.dump doesn't write a final newline
    except TypeError as e:
        print(e)

which gives:

<class 'datetime.date'>
{"a": [{"b": 2}, 123, "2000-12-26"]}
<class 'str'>
{"a": [{"b": 2}, 123, "111-12-26"]}
<class 'str'>
{"a": [{"b": 2}, 123, "2000-12-26111"]}
user_13079050
Related