Insert or Update Documents in MongoDB from a CSV using Python

user_6258802 0
Viewed 796

I'm trying to insert/update documents in MongoDB based on information that I have in a CSV. Where if the first header of the CSV customer_id doesn't exist then it should create a new document but, if it does exist then it should just update all the values in the document.

I have the script built that will look for the customer_id and if it doesn't exist then it will create the new document but, having trouble getting the update part working.

Do you have to specify each header that needs to be updated or is there a more efficient way of updating by utilizing the headers from the CSV in the event new headers are added later on so that the script wouldn't have to be updated to specify the new headers:

import csv
from pymongo import MongoClient
  
conn = MongoClient('localhost', 27017)

db = conn.shipping
collection = db.sales

file = csv.reader(open("shipping_list.csv"), delimiter=',')

header = ["customer_id", "customer_name", "sales_rep", "purchase_date", "region", "purchase_price", "shipping_status", "products_purchased"]

for each in file:
    if collection.count_documents({ 'customer_id': each[0] }) == 0:
        row={}
        for n in range(0,len(header)):
            row[header[n]] = each[n]
                 
        collection.insert_one(row)
    else:
        row={}
       for n in range(0,len(header)):
            row[header[n]] = each[n]
                 
        collection.update({'customer_id': each[0]}, row)
pymongo mongodb-query mongodb python-3.x python
1 Answers

If you want to use pymongo you can make you code much simpler using pandas and read_csv(). You only have to specify the key column so you can add more columns without changing the code. Use parse_dates if you want to store dates as "proper" dates not strings.

import pandas as pd
from pymongo import MongoClient

db = MongoClient()['mydatabase']

key = 'customer_id'
df = pd.read_csv('csv_pandas_mongo.csv', parse_dates=['purchase_date'])

for row in df.to_dict('records'):
    db.mycollection.update_one({key: row.get(key)}, {'$set': row}, upsert=True)
user_104906830
Related