Is there a way of storing the original data lines in a pandas dataframe

user_6534042 0
Viewed 41

I am using the read_csv method from pandas.

Say I am reading:

A,"B",C

With column names 1, 2, 3

I will get a dataframe of 3 columns, with the columns having values:

1 2 3
A B C

I want to get a dataframe of 4 columns:

orig 1 2 3
A,"B",C A B C

For this scenario, re-creating the csv line from the 3 values is not possible as the parsing will already have discarded the quotes.

Is there a good way of doing this? I am currently directly manipulating the input to add the column to the incoming data, however I was hoping there was something within pandas itself that could help?

pandas dataframe python
1 Answers

The answer heavily depends on the data in a csv-file. If you are sure that there's no separator in quoted items, then we can avoid unquoting with quoting=csv.QUOTE_NONE parameter:

# create dummy file
data = 'A,"B",C'
with open('test.csv', 'w') as f:
    f.write(data)

# read the file with quotes untouched
from csv import QUOTE_NONE
pd.read_csv('test.csv', header=None, quoting=QUOTE_NONE)

Ouput:

   0    1  2
0  A  "B"  C

In general, to see raw data we have to read them properly, i.e. without any csv-parser like pandas.read_csv or csv.reader

user_149096210
Related