`_csv.Error: line contains NUL` from a downloaded csv

Admin 1
Viewed 3464

I have downloaded a csv file from an url and used csv.reader to read its content. However I am getting _csv.Error: line contains NUL when I try to iterate over the _csv.reader object.

I have to mention that if I manually copy paste (ctrl+a, ctrl+c and ctrl+v ) the contents of the csv manually to a different csv the code works.

This is the code so far.

import csv
import requests

url='https://sedo.com/fileadmin/documents/resources/expiring_domain_auctions.csv'

response=requests.get(url)

with open('downloaded_csv.csv','wb') as out_file:
    out_file.write(response.content) # file is written properly in disk, can open with editor

with open('downloaded_csv.csv',newline='') as in_file:
    csv_contents=csv.reader(in_file,delimiter=';')
    print((csv_contents))
    for row in csv_contents: # _csv.Error: line contains NUL
        print(row)

Can anyone tell me how I can read the contents of this file in my python program?

python-requests csv character-encoding python
1 Answers

The file is encoded as UTF-16, so this encoding must be specified when reading the file.

>>> # Check the first 100 characters...
>>> r = requests.get(url)
>>> r.content.decode('utf-16')[:100]
'sep=;\n"Domain Name";"Start Time";"End Time";"Reserve Price";"Domain is IDN";"Domain has hyphen";"Dom'

Depending on your platform, you need to open the file like this:

with open('downloaded_csv.csv', newline='', encoding=encoding) as in_file:

where the value of encoding is one of utf-16, utf-16-le, utf-16-be

Note that you may need to remove or skip the initial "sep=;" line.

user_53209060
Related