I have been upgrading some Python 2 scripts to Python 3. I used 2to3 to refactor the code. Running with python3, I get an exception. I was able to reproduce the problems with just three lines of code;
import jsonrpclib
p = jsonrpclib.Server("http://r195/cgi-bin/streamscape_api")
print(p.nodeid())
With python2, it works:
$ python rpc.py
[u'3011']
When I run the exact same code with Python3, I get this exception:
$ python3 rpc.py
Traceback (most recent call last):
File "rpc.py", line 6, in <module>
print(p.nodeid())
File "/home/kory/.local/lib/python3.8/site-packages/jsonrpclib/jsonrpc.py", line 265, in __call__
return self.__send(self.__name, kwargs)
File "/home/kory/.local/lib/python3.8/site-packages/jsonrpclib/jsonrpc.py", line 212, in _request
response = self._run_request(request)
File "/home/kory/.local/lib/python3.8/site-packages/jsonrpclib/jsonrpc.py", line 226, in _run_request
response = self.__transport.request(
File "/usr/lib/python3.8/xmlrpc/client.py", line 1153, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib/python3.8/xmlrpc/client.py", line 1183, in single_request
raise ProtocolError(
xmlrpc.client.ProtocolError: <ProtocolError for r195/cgi-bin/streamscape_api: 400 Bad Request>
Using WireShark, I captured the traffic between the python script and the webserver. The only difference was tje header. With python2, thid is the header sent to the webserver;
Host: r195
Accept-Encoding: gzip
User-Agent: jsonrpclib/0.1 (Python 2.7.18)
Content-Type: application/json-rpc
With python3, the header is:
Host: r195
Accept-Encoding: gzip
Content-Type: text/xml
User-Agent: jsonrpclib/0.1 (Python 3.8.10)
Content-Type: application/json-rpc
Notice that python3, sends two "Content-Type" headers. Using curl
to build a request packet the headers, the problem is the "Content-Type: text/xml". Sending the request with curl without that content type, works correctly.
Just to be sure, as a test, I commented out this line from xmlrpc/client.py, and the script now works with python3. But commenting out that line is not a viable solution. I believe the webserver is running lighttpd 1.4.54.
Part of the issue is due to the way the xmlrpc.py has changed from python2 to python3. the send_content() used to be the one adding content-type header and send_request would just send the request in 2.
In 3, xmlrpc adds content-type in send_request() which is semantically incorrect. So when jsonrpclib overloads send_content() it adds the extra content-type header.
This project: https://github.com/tcalmant/jsonrpclib/tree/master/jsonrpclib corrects this by overloading send_request() as well and does not add the content-type header, which is correctly done in the overloaded send_content. So using this would fix this issue. However the underlying issue of why lighttpd fails when there are repeated content-type is weird. If there is no answer, we can go ahead and ignore, since most clients will not do this.
What I would like to know is that is this a python3 jdonrpclib bug or a lighthttpd bug?