Read file from sys.stdin in 'rb' mode : Python

user_13048627 0
Viewed 761

I have written below code to convert a csv file to a xml file. I am reading the file from sys.stdin and writing the output back to sys.stdout. I am getting below error while reading a file.

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcc in position 7652: invalid continuation byte

I have researched the error and found that reading the input file in 'rb' mode may resolve the error. Now how do I change the below code to read the input file from sys.stdin in 'rb' mode. I could not find answer yet. 

import csv
import sys
import os
from xml.dom.minidom import Document

filename = sys.argv[1]
filename = os.path.splitext(filename)[0]+'.xml'
pathname = "/tmp/"
output_file = pathname + filename

f = sys.stdin
reader = csv.reader(f)
fields = next(reader)
fields = [x.lower() for x in fields]
fieldsR = fields

doc = Document()
dataRoot = doc.createElement("rowset")
dataRoot.setAttribute('xmlns:xsi', "http://www.w3.org/2001/XMLSchema-instance")
dataRoot.setAttribute('xsi:schemaLocation', "./schema.xsd")
doc.appendChild(dataRoot)

for line in reader:
    dataElt = doc.createElement("row")           
    for i in range(len(fieldsR)):
        dataElt.setAttribute(fieldsR[i], line[i])
        dataRoot.appendChild(dataElt)


xmlFile = open(output_file,'w')
xmlFile.write(doc.toprettyxml(indent = '\t'))
xmlFile.close()
sys.stdout.write(output_file)
unicode python
1 Answers

In Python 3, both stdin, stout and stderr are wrapped in IO buffers that apply on the fly text encoding/decoding to the streams.

If you want direct access to the underlying binary stream, it is available as attributes in these wrappers.

For stdin, instead of calling .read in sys.stdin do sys.stdin.buffer.raw.read() - (and likewise for stderr and stdout, just use ...buffer.raw to get to the underlying binary stream).

user_1082050
Related