Adding items to list in dict if entry already exists

user_8569411 0
Viewed 76

I'm receiving many CSV-files that contain orders for different products. Those CSV-files need to be "converted" into a specific JSON-structure.

Each row of the CSV-file represents the order of one product. This means that if I would order two products, the CSV would contain two rows.

A simplified version of the CSV-file may look like this (please note the orderId "111" in the first and third row):

orderId,itemNumber,itemName,name,street
111,123,testitem,john doe,samplestreet 1
222,345,anothertestitem,jane doe,samplestreet 1
111,345,anothertestitem,john doe,samplestreet 1

My current solution works but I think I'm overcomplicating things.

Currently, I'm iterating over each CSV-row and create the JSON-structure where I use a helper-function that will either add the order or append a list that contains ordered items like so:

def add_orderitem(orderitem, order, all_orders):
    """ Adds an ordered product to the order or "create" a new order if it doesn't exist """
    for row in all_orders:
        # Order already exists
        if any(order["orderNumber"] == value for field, value in row.items()):
            print(f"Order '{order['orderNumber']}' already exists, adding product #{orderitem['sku']}")
            row["orderItems"].append(orderitem)
            return all_orders

    # New order
    print(f"New Order found, creating order '{order['orderNumber']}' and adding product #{orderitem['sku']}")
    all_orders.append(order)
    order["orderItems"].append(orderitem)
    return all_orders


def parse_orders():
    """ Converts CSV-orders into JSON """
    results = []
    orders = read_csv("testorder.csv")  # helper-function returns CSV-dictreader (list of dicts)
    for order in orders:
        # Create basic structure
        orderdata = {
            "orderNumber": order["orderId"],
            "address": {
                "name": order["orderId"],
                "street": order["street"]
            },
            "orderItems": []  # <-- this will be filled later
        }

        # Extract product-information that will be inserted in above 'orderItems' list
        product = {
            "sku": order["itemNumber"],
            "name": order["itemName"]
        }

        # Add order to final list or add item if order already exists
        results = add_orderitem(product, orderdata, results)

    return results


def main():
    from pprint import pprint
    parsed_orders = parse_orders()

    pprint(parsed_orders)


if __name__ == "__main__":
    main()

The skript works fine, the output below is what I'm expecting:

New Order found, creating order '111' and adding product #123
New Order found, creating order '222' and adding product #345
Order '111' already exists, adding product #345
[{'address': {'name': '111', 'street': 'samplestreet 1'},
  'orderItems': [{'name': 'testitem', 'sku': '123'},
                 {'name': 'anothertestitem', 'sku': '345'}],
  'orderNumber': '111'},
 {'address': {'name': '222', 'street': 'samplestreet 1'},
  'orderItems': [{'name': 'anothertestitem', 'sku': '345'}],
  'orderNumber': '222'}]

Is there a way, to do this "smarter"?

csv list dictionary python
1 Answers

Imo a namedtuple and a groupby would make your code clearer:

from collections import namedtuple
from itertools import groupby

# csv data or file
data = """orderId,itemNumber,itemName,name,street
111,123,testitem,john doe,samplestreet 1
222,345,anothertestitem,jane doe,samplestreet 1
111,345,anothertestitem,john doe,samplestreet 1
"""

# the Order tuple
Order = namedtuple('Order', 'orderId itemNumber itemName name street')

# load the csv into orders
orders = [Order(*values) for line in data.split("\n")[1:] if line for values in [line.split(",")]]

# and group it by orderId
orders = sorted(orders, key = lambda order: order.orderId)

# group it by orderId
output = list()
for key, values in groupby(orders, key=lambda order: order.orderId):
    items = list(values)
    dct = {"address": {"name": items[0].name, "street": items[0].street},
           "orderItems": [{"name": item.itemName, "sku": item.itemNumber} for item in items]}
    output.append(dct)

print(output)

This yields

[{'address': {'name': 'john doe', 'street': 'samplestreet 1'}, 'orderItems': [{'name': 'testitem', 'sku': '123'}, {'name': 'anothertestitem', 'sku': '345'}]},
 {'address': {'name': 'jane doe', 'street': 'samplestreet 1'}, 'orderItems': [{'name': 'anothertestitem', 'sku': '345'}]}]

You could even put it in a great comprehension but that would not make it more readable.

user_12314500
Related