Suppose I have sample data in an Excel document:
| header1 | header2 | header3 |
|---|---|---|
| some data | testing | 123 |
| moar data | hello! | 456 |
I export this data to csv format with Excel, with File > Save as > .csv
This is my data sample.csv:
$ cat sample.csv
header1,header2,header3
some data,testing,123
moar data,hello!,456%
Note that Excel apparently does not add a newline at the end, by default -- this is indicated by % at the end.
Now let's say I want to append a row(s) to the CSV file. I can use csv module to do that:
import csv
def append_to_csv_file(file: str, row: dict, encoding=None) -> None:
# open file for reading and writing
with open(file, 'a+', newline='', encoding=encoding) as out_file:
# retrieve field names (CSV file headers)
reader = csv.reader(out_file)
out_file.seek(0)
field_names = next(reader, None)
# add new row to the CSV file
writer = csv.DictWriter(out_file, field_names)
writer.writerow(row)
row = {'header1': 'new data', 'header2': 'blah', 'header3': 789}
append_to_csv_file('sample.csv', row)
So now a newline is added to end of file, but problem is that the data is added to end of last line, rather than on a separate line:
$ cat sample.csv
header1,header2,header3
some data,testing,123
moar data,hello!,456new data,blah,789
This causes issue when I want to read back the updated data from the file:
with open('sample.csv', newline='') as f:
print(list(csv.DictReader(f)))
# [{..., 'header3': '456new data', None: ['blah', '789']}]
Question: so what is the best option to handle case when CSV file might not have newline at the end, when appending a row(s) to file.
Current attempt
This is my solution to work around case when appending to CSV file, but file may not end with a newline character:
import csv
def append_to_csv_file(file: str, row: dict, encoding=None) -> None:
with open(file, 'a+', newline='', encoding=encoding,) as out_file:
# get current file position
pos = out_file.tell()
print('pos:', pos)
# seek to one character back
out_file.seek(pos - 1)
# read in last character
c = out_file.read(1)
print(out_file.tell(), repr(c))
if c != '\n':
delta = out_file.write('\n')
pos += delta
print('new_pos:', pos)
# retrieve field names (CSV file headers)
reader = csv.reader(out_file)
out_file.seek(0)
field_names = next(reader, None)
# add new row to the CSV file
writer = csv.DictWriter(out_file, field_names)
# out_file.seek(pos + 1)
writer.writerow(row)
row = {'header1': 'new data', 'header2': 'blah', 'header3': 789}
append_to_csv_file('sample.csv', row)
This is output from running the script:
pos: 68
68 '6'
new_pos: 69
The contents of CSV file now look as expected:
$ cat sample.csv
header1,header2,header3
some data,testing,123
moar data,hello!,456
new data,blah,789
I am wondering if anyone knows of an easier way to do this. I feel like I might be overthinking this a bit. I basically want to account for cases where CSV file might need a newline added to end, before a new row is appended to end of file.
If it helps, I am running this on a Mac OS environment.