How to read the contents of a csv file into a class with each csv row as a class instance

Viewed 17809

I'm a Python newbie, and I've been struggling with a class assignment for days. I have a csv file that contains data as such:

id,latitude,longitude,city,label,yr1970,yr1975,yr1980,yr1985,yr1990,yr1995,yr2000,yr2005
1,35.6832085,139.8089447,Tokyo,Tokyo,23.3,26.61,28.55,30.3,32.53,33.59,34.45,35.62

There are about 40 rows in this file, each containing data related to a world city. As you can see, the top row is the header. I am supposed to create a class in Python and read the csv file into the class, where every row becomes an instance of the class. I am then to store the class instances in a list. I've been able to create one instance where all of the data is stored, but I can't seem to create an instance for each row (and I obviously do not want to do it manually).

Here's what I've got so far:

import csv
Cities = []


with open('filepath','rb') as f:
cityList = csv.reader(f)
for row in cityList:
    if row != 'label':
        for row in cityList:
            citysName = row[3]


            class City:

                def __init__(self, cityName=row[3], Label=row[4], Lat=row[1],
                         Lon=row[2], yr1970=row[5], yr1975=row[6], yr1980=row[7],
                             yr1985=row[8], yr1990=row[9], yr1995=row[10], yr2000=row[11],
                             yr2005=row[12], yr2010=row[13]):
                    self.cityName = cityName
                    self.label = Label
                    self.lat = Lat
                    self.lon = Lon
                    self.yr1970 = yr1970
                    self.yr1975 = yr1975
                    self.yr1980 = yr1980
                    self.yr1985 = yr1985
                    self.yr1990 = yr1990
                    self.yr1995 = yr1995
                    self.yr2000 = yr2000
                    self.yr2005 = yr2005
                    self.yr2010 = yr2010

            citysName = City()

            Cities.append(citysName.cityName)
            Cities.append(citysName.label)
            Cities.append(citysName.lat)
            Cities.append(citysName.lon)
            Cities.append(citysName.yr1970)
            Cities.append(citysName.yr1975)
            Cities.append(citysName.yr1980)
            Cities.append(citysName.yr1985)
            Cities.append(citysName.yr1990)
            Cities.append(citysName.yr1995)
            Cities.append(citysName.yr2000)
            Cities.append(citysName.yr2005)
            Cities.append(citysName.yr2010)

        print Cities

Again, I'm quite new with Python (and coding in general), and I realize this code is not good, but I'm having a lot of difficulty finding tips for reading csv files into a Python class.

5 Answers

You can try this:

import csv
class City:
   def __init__(self, row, header):
        self.__dict__ = dict(zip(header, row))

data = list(csv.reader(open('file.csv')))
instances = [City(i, data[0]) for i in data[1:]]

However, since you mentioned that there are many rows, you may want to create an id for each city that will act as your string representation in the list:

import csv
class City:
   def __init__(self, row, header, the_id):
       self.__dict__ = dict(zip(header, row)) 
       self.the_id = the_id
   def __repr__(self):
       return self.the_id

data = list(csv.reader(open('file.csv')))
instances = [City(a, data[0], "city_{}".format(i+1)) for i, a in enumerate(data[1:])]

Your output will be a listing like this:

[city_1, city_2, city_3...]

And any attributes can be called like so:

instances[1].latitude

Regarding your recent comment, to access city attributes by city name, you can slightly restructure instances:

instances = {a[3]:City(a, data[0], "city_{}".format(i+1)) for i, a in enumerate(data[1:])}

Some tips to help you clean up your code:

  1. Instead of this:

    self.yr1970
    

    Define a list to keep track of years and their values:

    tokyo_years = {
        1970: 23.3,
        1975: 26.61,
        # ...
    }
    

    Now pair this structure with each city:

    cities = [
        { 'city': 'Tokyo',     'years': tokyo_years },
        { 'city': 'Vancouver', 'years': vancouver_years },
        # ...
    ]
    
  2. Don't nest so deeply. Also, the following is really weird:

    for row in cityList:
        if row != 'label':
            for row in cityList:
    

    You are looping over something and then looping over it again while you are looping over it...!

  3. Classes belong at top-level. That means there should be 0 spaces preceding class.

             class City:
    

    should be:

    class City:
    

The reason I mention all this is because trying to do anything further with messy code just results in messier code. :) Try to improve your current code by:

  1. Using data structures (lists, dictionaries).
  2. Restricting levels of nested code to 2 max. (Consider using functions to help you with this.)
  3. Putting classes at top-level.

If your data is just an immutable record, use namedtuple:

>>> from collections import namedtuple

>>> City = namedtuple('City', 'lat lon cityName label '
...                   'yr1970 yr1975 yr1980 yr1985 yr1990 yr1995 yr2000 yr2005 yr2010')

You can slice the row as you don't need the first value, and unpack it using *:

>>> row = ['1', '35.6832085', '139.8089447', 'Tokyo', 'Tokyo',
...        '23.3', '26.61', '28.55', '30.3', '32.53', '33.59', '34.45', '35.62', '35.7']

>>> city = City(*row[1:])

>>> city
City(lat='35.6832085', lon='139.8089447', cityName='Tokyo', label='Tokyo',
     yr1970='23.3', yr1975='26.61', yr1980='28.55', yr1985='30.3', yr1990='32.53',
     yr1995='33.59', yr2000='34.45', yr2005='35.62', yr2010='35.7')

You need to add just this object to your lists of cities, not every attribute:

>>> cities.append(city)

Putting it together with a list comprehension filtering out the label rows:

import csv
from collections import namedtuple

City = namedtuple('City',
                  'lat lon cityName label '
                  'yr1970 yr1975 yr1980 yr1985 yr1990 yr1995 yr2000 yr2005 yr2010')

with open('filepath') as f:
    cities = [City(*row[1:]) for row in csv.reader(f)
              if row[0] != 'label']

You can do this by using a csv.DictReader which will return each row as an dictionary. It will also allow you to easily define the class based on the fieldnames defined in the first or header row of the csv file rather than hardcoding them into your program.

You'll still be able reference the fields of any record with "dot" syntax such as city.latitude, city.label, etc.

Below is Python 2 code. For Python 3, open the file with:

with open('cities.csv', 'r', newline='') as f:

instead what's shown.

import csv

with open('cities.csv', 'rb') as f:
    reader = csv.DictReader(f)
    fieldnames = reader.fieldnames

    class City:
        def __init__(self, **fields):
            self.__dict__.update(**fields)

        def __repr__(self):  # Added to make printing instances show their contents.
            fields = ', '.join(('{}={!r}'.format(fieldname, getattr(self, fieldname))
                                   for fieldname in fieldnames))
            return('{}({})'.format(self.__class__.__name__, fields))

    Cities = [City(**row) for row in reader]

print(Cities)

Sample output from your sample input file:

[City(id='1', latitude='35.6832085', longitude='139.8089447', city='Tokyo', label='Tokyo', yr1970='23.3', yr1975='26.61', yr1980='28.55', yr1985='30.3', yr1990='32.53', yr1995='33.59', yr2000='34.45', yr2005='35.62')]
import csv


class City:
    def __init__(self, **kwargs):
        self.id = kwargs.get('id')
        self.latitude = kwargs.get('latitude')
        self.longitude = kwargs.get('longitude')
        self.city = kwargs.get('city')
        self.label = kwargs.get('label')
        self.year_1970 = kwargs.get('yr1970')
        self.year_1975 = kwargs.get('yr1975')
        self.year_1980 = kwargs.get('yr1980')
        self.year_1985 = kwargs.get('yr1985')
        self.year_1990 = kwargs.get('yr1990')
        self.year_1995 = kwargs.get('yr1995')
        self.year_2000 = kwargs.get('yr2000')
        self.year_2005 = kwargs.get('yr2005')

    def __str__(self):
        return self.label


if __name__ == '__main__':
    with open('filepath', 'r') as csvfile:
        reader = csv.DictReader(csvfile)

        for row in reader:
            city = City(**row)
            print(city)
Related