How can I parse this list with pyparsing?

Richard Kelly 0
Viewed 52

I'm trying to use pyparsing to parse a list with "section headers" and "items. In this example, the sections will be days, and the items will be grocery items we need to buy.

from pyparsing import *

input = """Monday
- eggs
- milk
Tuesday
- bread
- flour
"""

day = Word(alphas)("day")
item = Suppress("- ") + rest_of_line
items = OneOrMore(item)("items")
daily_shopping_list = OneOrMore(day + items)

print(daily_shopping_list.parse_string(input).asDict())

This returns {'day': 'Tuesday', 'items': ['bread', 'flour']}

The desired output is {{'day': 'Monday', 'items': ['eggs', 'milk']}, {'day': 'Tuesday', 'items': ['bread', 'flour']}}

Why is this code skipping Monday?

Thank you.

Edit: As Tim Roberts mentioned, dropping .asDict() produces a valid output:

['Monday', ['eggs', 'milk'], 'Tuesday', ['bread', 'flour']]

pyparsing python
2 Answers

The desired output is {{'day': 'Monday', 'items': ['eggs', 'milk']}, {'day': 'Tuesday', 'items': ['bread', 'flour']}}

First of all, as mentioned by @TimRoberts in the comments, your desired output is invalid.

{{'day': 'Monday', 'items': ['eggs', 'milk']}, {'day': 'Tuesday', 'items': ['bread', 'flour']}}

This would be a set with dict as elements. No can do.

If you just type that in a Python console, you will get:

TypeError: unhashable type: 'dict'

But you probably meant a list of dict instead, and this is pretty much possible.

raw_list = daily_shopping_list.parse_string(input)
result = [dict(day=day, items=items) for day, items in zip(*[iter(z)]*2)]
print(result)

# [{'day': 'Monday', 'items': ['eggs', 'milk']}, 
#  {'day': 'Tuesday', 'items': ['bread', 'flour']}]
Rodrigo Rodrigues0

This is a great first project with pyparsing. There are some features that pyparsing offers that help in returning structured data like this.

First off, when you call parse_string(), it returns a pyparsing ParseResults https://pyparsing-docs.readthedocs.io/en/latest/pyparsing.html#pyparsing.ParseResults object. If you print this out, you get:

result = daily_shopping_list.parse_string(input)
print(result)
['Monday', 'eggs', 'milk', 'Tuesday', 'bread', 'flour']

It looks like a list of strings, but has a lot more features.

The first thing to look at is to call the dump() method. This will list out the parsed strings, followed by an indented listing of named items.

print(result.dump())
['Monday', 'eggs', 'milk', 'Tuesday', 'bread', 'flour']
- day: 'Tuesday'
- items: ['bread', 'flour']

As Tim Roberts points out, the default for naming is similar to that for Python dicts: the last value stored is the one you end up with.

You are actually pretty close to getting the structured results you want. Add a pyparsing Group expression, changing this line:

daily_shopping_list = OneOrMore(day + items)

to this line:

daily_shopping_list = OneOrMore(Group(day + items))

This will create day-items groups, and after this change, results.dump() prints:

[['Monday', 'eggs', 'milk'], ['Tuesday', 'bread', 'flour']]
[0]:
  ['Monday', 'eggs', 'milk']
  - day: 'Monday'
  - items: ['eggs', 'milk']
[1]:
  ['Tuesday', 'bread', 'flour']
  - day: 'Tuesday'
  - items: ['bread', 'flour']

This is actually a ParseResults containing 2 ParseResults, one for each parsed Group. dump() shows the names that are available for accessing the named values. For instance, you can get the first day's values as:

results[0]["day"]
results[0]["items"]

like they are dicts. You can also treat them like attributes in an object (if the names are valid Python identifiers):

results[0].day
results[0].items

If you want them as dicts, then call as_dict() on each contained ParseResults:

print([day_list.as_dict() for day_list in result])
[{'day': 'Monday', 'items': ['eggs', 'milk']}, {'day': 'Tuesday', 'items': ['bread', 'flour']}]

If you want this as a nested dict, where each day name is the dict key to the sub-dict, you can have pyparsing use the first element in each group as a key by wrapping the OneOrMore expression in a pyparsing Dict:

daily_shopping_list = Dict(OneOrMore(Group(day + items)))

Now the first part of results.dump() shows these keys:

[['Monday', 'eggs', 'milk'], ['Tuesday', 'bread', 'flour']]
- Monday: ['eggs', 'milk']
  - day: 'Monday'
  - items: ['eggs', 'milk']
- Tuesday: ['bread', 'flour']
  - day: 'Tuesday'
  - items: ['bread', 'flour']

And you can use the day names as keys:

result["Monday"]["items"]

Now calling result.as_dict() will pprint as:

from pprint import pprint
pprint(result.as_dict())
{'Monday': {'day': 'Monday', 'items': ['eggs', 'milk']},
 'Tuesday': {'day': 'Tuesday', 'items': ['bread', 'flour']}}
PaulMcG0
Related