Append to a CSV file which does not end with newline

Viewed 27

Suppose I have sample data in an Excel document:

header1 header2 header3
some data testing 123
moar data hello! 456

I export this data to csv format with Excel, with File > Save as > .csv

This is my data sample.csv:

$ cat sample.csv
header1,header2,header3
some data,testing,123
moar data,hello!,456%

Note that Excel apparently does not add a newline at the end, by default -- this is indicated by % at the end.

Now let's say I want to append a row(s) to the CSV file. I can use csv module to do that:

import csv


def append_to_csv_file(file: str, row: dict, encoding=None) -> None:
    # open file for reading and writing
    with open(file, 'a+', newline='', encoding=encoding) as out_file:
        # retrieve field names (CSV file headers)
        reader = csv.reader(out_file)
        out_file.seek(0)
        field_names = next(reader, None)
        # add new row to the CSV file
        writer = csv.DictWriter(out_file, field_names)
        writer.writerow(row)


row = {'header1': 'new data', 'header2': 'blah', 'header3': 789}
append_to_csv_file('sample.csv', row)

So now a newline is added to end of file, but problem is that the data is added to end of last line, rather than on a separate line:

$ cat sample.csv 
header1,header2,header3
some data,testing,123
moar data,hello!,456new data,blah,789

This causes issue when I want to read back the updated data from the file:

with open('sample.csv', newline='') as f:
    print(list(csv.DictReader(f)))

# [{..., 'header3': '456new data', None: ['blah', '789']}]

Question: so what is the best option to handle case when CSV file might not have newline at the end, when appending a row(s) to file.

Current attempt

This is my solution to work around case when appending to CSV file, but file may not end with a newline character:

import csv


def append_to_csv_file(file: str, row: dict, encoding=None) -> None:
    with open(file, 'a+',  newline='', encoding=encoding,) as out_file:
        # get current file position
        pos = out_file.tell()
        print('pos:', pos)
        # seek to one character back
        out_file.seek(pos - 1)
        # read in last character
        c = out_file.read(1)
        print(out_file.tell(), repr(c))
        if c != '\n':
            delta = out_file.write('\n')
            pos += delta
        print('new_pos:', pos)
        # retrieve field names (CSV file headers)
        reader = csv.reader(out_file)
        out_file.seek(0)
        field_names = next(reader, None)
        # add new row to the CSV file
        writer = csv.DictWriter(out_file, field_names)
        # out_file.seek(pos + 1)
        writer.writerow(row)


row = {'header1': 'new data', 'header2': 'blah', 'header3': 789}
append_to_csv_file('sample.csv', row)

This is output from running the script:

pos: 68
68 '6'
new_pos: 69

The contents of CSV file now look as expected:

$ cat sample.csv 
header1,header2,header3
some data,testing,123
moar data,hello!,456
new data,blah,789

I am wondering if anyone knows of an easier way to do this. I feel like I might be overthinking this a bit. I basically want to account for cases where CSV file might need a newline added to end, before a new row is appended to end of file.

If it helps, I am running this on a Mac OS environment.

0 Answers
Related