Cant read content from a specific .CSV file using pandas

user_11021252 0
Viewed 101

I have a CSV file which contains coordinates in Degree Minute Seconds format. But I can't load that CSV file using read_csv

eg:

'''74° 18' 01.8963" E'''    '''32° 56' 40.2788" N'''
'''76° 05' 57.9815" E'''    '''31° 24' 25.0336" N'''
'''75° 02' 45.5176" E'''    '''30° 25' 19.6260" N'''
'''73° 23' 12.3829" E'''    '''31° 47' 47.4578" N'''
'''74° 18' 01.8963" E'''    '''32° 56' 40.2788" N'''

Or

74° 18' 01.8963" E 32° 56' 40.2788" N
76° 05' 57.9815" E 31° 24' 25.0336" N
75° 02' 45.5176" E 30° 25' 19.6260" N
73° 23' 12.3829" E 31° 47' 47.4578" N
74° 18' 01.8963" E 32° 56' 40.2788" N

Anyone have any suggestions?

ERROR:

Traceback (most recent call last):

  File "<ipython-input-7-5e6c73be55c1>", line 1, in <module>
    pd.read_csv("test.csv")

  File "C:\ProgramData\Anaconda3\envs\obspy\lib\site-packages\pandas\io\parsers.py", line 676, in parser_f
    return _read(filepath_or_buffer, kwds)

  File "C:\ProgramData\Anaconda3\envs\obspy\lib\site-packages\pandas\io\parsers.py", line 448, in _read
    parser = TextFileReader(fp_or_buf, **kwds)

  File "C:\ProgramData\Anaconda3\envs\obspy\lib\site-packages\pandas\io\parsers.py", line 880, in __init__
    self._make_engine(self.engine)

  File "C:\ProgramData\Anaconda3\envs\obspy\lib\site-packages\pandas\io\parsers.py", line 1114, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)

  File "C:\ProgramData\Anaconda3\envs\obspy\lib\site-packages\pandas\io\parsers.py", line 1891, in __init__
    self._reader = parsers.TextReader(src, **kwds)

  File "pandas\_libs\parsers.pyx", line 529, in pandas._libs.parsers.TextReader.__cinit__

  File "pandas\_libs\parsers.pyx", line 749, in pandas._libs.parsers.TextReader._get_header

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 5: invalid start byte
pandas coordinates csv python-3.x
2 Answers

Cant say that its the best way but see if this can work for you:

# using some random seperator to get the entire row as one column
df = pd.read_csv("coordinates.csv", sep="!", header=None)
df.columns = ['coord']

# added separate columns will blanks for lat and lon (I assumed them to be lat and lon)
df['lat'] = ''*len(df['coord'])
df['lon'] = ''*len(df['coord'])

df at this stage:

                                   coord lat lon
0  74° 18' 01.8963" E 32° 56' 40.2788" N        
1  76° 05' 57.9815" E 31° 24' 25.0336" N        
2  75° 02' 45.5176" E 30° 25' 19.6260" N        
3  73° 23' 12.3829" E 31° 47' 47.4578" N        
4  74° 18' 01.8963" E 32° 56' 40.2788" N

.

import re

# assuming each coordinate will end with either of one directional indicators - E, W, N or S
pattern = '[EWNS]'

for i, val in enumerate(list(df['coord'])):
    idx = re.search(pattern, val).start()
    df['lat'][i] = df['coord'][i][:idx+1]
    df['lon'][i] = df['coord'][i][idx+1:]

print(df)

df:

                                   coord                 lat                  lon
0  74° 18' 01.8963" E 32° 56' 40.2788" N  74° 18' 01.8963" E   32° 56' 40.2788" N
1  76° 05' 57.9815" E 31° 24' 25.0336" N  76° 05' 57.9815" E   31° 24' 25.0336" N
2  75° 02' 45.5176" E 30° 25' 19.6260" N  75° 02' 45.5176" E   30° 25' 19.6260" N
3  73° 23' 12.3829" E 31° 47' 47.4578" N  73° 23' 12.3829" E   31° 47' 47.4578" N
4  74° 18' 01.8963" E 32° 56' 40.2788" N  74° 18' 01.8963" E   32° 56' 40.2788" N
user_126961630

If they are fixed length records you can use read_fwf

# for this - '''74° 18' 01.8963" E'''    '''32° 56' 40.2788" N'''
df = pd.read_fwf('filename.csv',[(3,22),(32,51)], header=None)
df
                     0                    1
0  74A° 18' 01.8963" E  32A° 56' 40.2788" N
1  76A° 05' 57.9815" E  31A° 24' 25.0336" N
2  75A° 02' 45.5176" E  30A° 25' 19.6260" N
3  73A° 23' 12.3829" E  31A° 47' 47.4578" N
4  74A° 18' 01.8963" E  32A° 56' 40.2788" N

or

# for this - 74° 18' 01.8963" E 32° 56' 40.2788" N
df = pd.read_fwf('filename.csv',[(0,19),(20,39)], header=None)
user_60756990
Related