how to use a list of column names to get the indices of each column in python

user19825372 0
Viewed 73

I have a list of column names and a csv file. I want to implement a function that reads the csv to return X (13 attributes as listed in x_col_names) and y (the corresponding final grade). X: numpy array: shape = [N, D], y: numpy array: shape = [N]. The data is delimited with

def load_student_data(filename):

X, y = None, None

X = np.empty((395,13))
y = np.empty((395,))
rows = []
with open(filename) as file:
  data = csv.reader(filename)
  for row in file:
    rows.append([row])
  arr = np.array(rows)
  print(arr)

return X, y

I tried this code but it didn't work. I think my logic is flawed. Also, I have no way to get the index of the columns because the only way I found was using pandas which I am not allowed to use. I am only entitled to csv, numpy and math.

I should get

1 14 4

6 5 6

(395, 13) (395,)

but the only output I got correct was the last one. I know I have not defined my X and y because I'm having difficulty and am still working on it. Any guidance would be much appreciated.

Please help. Thank you

csv numpy python
1 Answers

You should provide a small example file, and demonstrate the results. It does't need to be the full 395 rows and 14? columns.

For now I'm just going to comment on the code

def load_student_data(filename):
    X, y = None, None
    X = np.empty((395,13))    
    y = np.empty((395,))

Why did you define X,y twice? Do you understand what empty does? It is not the same as the following list [].

    rows = []
    with open(filename) as file:
          data = csv.reader(filename)
          for row in file:
              rows.append([row])

You open the file, but pass filename, not file to the reader. What's the logic there? You aren't doing anything with data!

So far I think rows will just be the same as doing rows=file.readlines(), a list of strings, one per line.

          arr = np.array(rows)
          print(arr)
    return X, y

So arr is an array of strings, just an array version of rows. X,y are still the values you created with empty.

In a comment I suggested using np.genfromtxt, but that will require doing some reading, and testing with smaller files. But it shouldn't be hard to read the csv with just the readlines, or with the help of the reader. If doing that, I'd forget about numpy for the moment, and focus on reading and parsing individual lines. Hint - string split is widely used to separate the columns in a line.

The problems with the code suggest you haven't paid much attention to details, and still have a limited grasp of Python.

hpaulj0
Related