datetime.strptime issues in working with csv.DictReader

user_13442451 0
Viewed 111

I have a CSV file showing some basic rescued dogs, including when they got into the rescue and when they got adopted. I'm trying to eventually do a visualization on a map to show where the dogs move from one area to others. One of the element I'm looking at is the length of how long a particular dog with a foster before being adopted.

This is the beginning of the CSV file.

Name,Status,Sex,Animal_ID,Birthdate,Adopter_City,Adopter_State,Adopter_Zip,Foster_City,Foster_State,Foster_Zip,Created_Date,Adopted_Date,Length
Marnie,Adopted,Female,15598143,2019-03-24,Port Orchard,Wa,98367,Auburn,WA,98002,2020-04-22,2020-07-08
Kendra,Adopted,Female,15598254,2019-12-31,Austin,Tx,78731,Austin ,Tx,78723,2020-04-22,2020-05-01
Hope,Adopted,Female,15598264,2019-10-25,Springfield ,OR,97477,Austin,TX,78737,2020-04-22,2020-05-02
...

I used datetime.strptime to format the intake (into foster care) and outtake (adopted out). Then I calculated the length of stay = outtake - intake. The code calculated the length correctly.

import csv
from datetime import datetime


with open('animals_20200826_12p46.csv', newline = '') as csvfile:
    data = csv.DictReader(csvfile)
   
    for row in data:
        if len(row['Adopted_Date']) < 1 :
            continue
        intake = datetime.strptime(row['Created_Date'], '%Y-%m-%d')
        outtake = datetime.strptime(row['Adopted_Date'], '%Y-%m-%d')
        length = outtake - intake
        row['Length'] = length
        
        print('In:', intake, 'Out:', outtake, 'In foster:', length)

The result came back as:

In: 2020-04-22 00:00:00 Out: 2020-07-08 00:00:00 In foster: 77 days, 0:00:00
In: 2020-04-22 00:00:00 Out: 2020-05-01 00:00:00 In foster: 9 days, 0:00:00
In: 2020-04-22 00:00:00 Out: 2020-05-02 00:00:00 In foster: 10 days, 0:00:00
In: 2020-04-22 00:00:00 Out: 2020-06-01 00:00:00 In foster: 40 days, 0:00:00
...

I then followed the steps suggested in How to update rows in a CSV file to update my CSV file.

import csv
from datetime import datetime
from tempfile import NamedTemporaryFile
import shutil

filename = 'animals_20200826_12p46.csv'
tempfile = NamedTemporaryFile(mode='w', delete=False)

fields = ['Name','Status','Sex','Animal_ID','Birthdate','Adopter_City','Adopter_State','Adopter_Zip','Foster_City','Foster_State','Foster_Zip','Created_Date','Adopted_Date','Length']

with open(filename, 'r') as csvfile, tempfile:
    reader = csv.DictReader(csvfile, fieldnames= fields)
    writer = csv.DictWriter(tempfile, fieldnames=fields)
    for row in reader:
        if len(row['Adopted_Date']) < 1 :  
            continue
        intake = datetime.strptime(row['Created_Date'], '%Y-%m-%d')
        outtake = datetime.strptime(row['Adopted_Date'], '%Y-%m-%d')
        row['Length'] = outtake - intake
        writer.writerow(row)

shutil.move(tempfile.name, filename)

The main part of the for-loop is the same, but now I ran into traceback error as below.

Traceback (most recent call last):
  File "CSV.py", line 18, in <module>
    intake = datetime.strptime(row['Created_Date'], '%Y-%m-%d')
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data 'Created_Date' does not match format '%Y-%m-%d'

Where did I go wrong? What would be a better way to do this than my current approach?

csv datetime python
3 Answers

The difference is that in the working example you are using

DictReader(csvfile)

whereas in the non-working example you are using

DictReader(csvfile, fieldnames=fields)

The documentation says:

If fieldnames is omitted, the values in the first row of file f will be used as the fieldnames.

In other words, if fieldnames is omitted, the first row is consumed to determine the fieldnames, otherwise it is not consumed and instead emitted like all other rows in the iteration.

This causes the initial row containing the column headers to be passed to strptime.

I suggest you just omit the fieldnames parameter for the reader again.

user_46215130

Use dateparser.parse instead of datetime.strptime.

Code:

import dateparser

intake = dateparser.parse(row['Created_Date'])
outtake = dateparser.parse(row['Adopted_Date'])
user_141875200

change this line:

reader = csv.DictReader(csvfile)

and it'll work fine

as the official document says:

fieldnames is an optional parameter. If it is omitted, the values in the first row of file will be used as the fieldnames.

When you determine fieldnames manually every row including the first one expected to be data rows.

user_52800560
Related