What is end-of-line for python's csv.reader?

user_15086226 0
Viewed 251

Python docs specifies:

The reader is hard-coded to recognise either '\r' or '\n' as end-of-line, and ignores lineterminator. This behavior may change in the future.

I wrote a simple program

step 1

with open('test.csv','wb') as f:
    f.write(b'hello\r\rworld\n!')

step 2

import csv

with open('test.csv','r',newline='') as f:
    r = csv.reader(f)
    for row in r:
        print(row)

Output of step 2:

['hello']
[]
['world']
['!']

If I change step 1 to :

import csv

with open('test.csv','wb') as f:
    f.write(b'hello\r\nworld\n!')

Output:

['hello']
['world']
['!']

My question is why the empty list [] disappeared in this case?

If I am correct then in step 2 csv.reader encountered first \r and returned what it had read until then, but when it encountered second \r immediately after first, it had not read anything and hence returned empty list.

But why this doesn't happen in second case if \n is also end-of-line for csv.reader?

csv python
2 Answers

You are using the built-in write function.

If you look at your files, you will already see the difference. The output file when using b'hello\r\rworld\n!' contains:

hello

world
!

And when you use b'hello\r\nworld\n!', then this produces this file content:

hello
world
!

Therefore, it has nothing to do with the CSV reader.

Back to your issue: My guess is that \r\r is interpreted individually when writing b'hello\r\rworld\n!' to the file. On the other hand, the combination \r\n is interpreted as one control character.

My assumption is based on:

A manner of interpreting text streams in which all of the following are recognized as ending a line: the Unix end-of-line convention '\n', the Windows convention '\r\n', and the old Macintosh convention '\r'. See PEP 278 and PEP 3116, as well as bytes.splitlines() for an additional use.

from https://docs.python.org/3/glossary.html#term-universal-newlines

Trying out bytes.splitlines():

result = b'halifax\ndigby\n\nzuerich\rbern\r\rlugano\r\nlausanne'.splitlines()
for line in result:
    print(line)

Results in:

b'halifax'
b'digby'
b''
b'zuerich'
b'bern'
b''
b'lugano'
b'lausanne'

And, if you swap \r\n to \n\r, which is not a recognized as a line ending, then it will write two new lines. That said b'hello\n\rworld\n!' will result in this file content:

hello

world
!

I hope that helps. Cheers!

See also:

user_426590

I suspect this is a documentation bug. CR by itself is a fairly unusual end-of-line marker (i think pre-unix MacOS used it, as well as the venerable Swedish ABC80 that I grew up with :-) ), but CR+LF is very common, so it would make sense for csv.reader to interpret a CR+LF sequence as a single end-of-line.

user_41770090
Related