My answer addresses the specific (and somewhat common) case where you don't really need to convert the entire xml to json, but what you need is to traverse/access specific parts of the xml, and you need it to be fast, and simple (using json/dict-like operations).
Approach
For this, it is important to note that parsing an xml to etree using lxml is super fast. The slow part in most of the other answers is the second pass: traversing the etree structure (usually in python-land), converting it to json.
Which leads me to the approach I found best for this case: parsing the xml using lxml, and then wrapping the etree nodes (lazily), providing them with a dict-like interface.
Code
Here's the code:
from collections import Mapping
import lxml.etree
class ETreeDictWrapper(Mapping):
def __init__(self, elem, attr_prefix = '@', list_tags = ()):
self.elem = elem
self.attr_prefix = attr_prefix
self.list_tags = list_tags
def _wrap(self, e):
if isinstance(e, basestring):
return e
if len(e) == 0 and len(e.attrib) == 0:
return e.text
return type(self)(
e,
attr_prefix = self.attr_prefix,
list_tags = self.list_tags,
)
def __getitem__(self, key):
if key.startswith(self.attr_prefix):
return self.elem.attrib[key[len(self.attr_prefix):]]
else:
subelems = [ e for e in self.elem.iterchildren() if e.tag == key ]
if len(subelems) > 1 or key in self.list_tags:
return [ self._wrap(x) for x in subelems ]
elif len(subelems) == 1:
return self._wrap(subelems[0])
else:
raise KeyError(key)
def __iter__(self):
return iter(set( k.tag for k in self.elem) |
set( self.attr_prefix + k for k in self.elem.attrib ))
def __len__(self):
return len(self.elem) + len(self.elem.attrib)
# defining __contains__ is not necessary, but improves speed
def __contains__(self, key):
if key.startswith(self.attr_prefix):
return key[len(self.attr_prefix):] in self.elem.attrib
else:
return any( e.tag == key for e in self.elem.iterchildren() )
def xml_to_dictlike(xmlstr, attr_prefix = '@', list_tags = ()):
t = lxml.etree.fromstring(xmlstr)
return ETreeDictWrapper(
t,
attr_prefix = '@',
list_tags = set(list_tags),
)
This implementation is not complete, e.g., it doesn't cleanly support cases where an element has both text and attributes, or both text and children (only because I didn't need it when I wrote it...) It should be easy to improve it, though.
Speed
In my specific use case, where I needed to only process specific elements of the xml, this approach gave a suprising and striking speedup by a factor of 70 (!) compared to using @Martin Blech's xmltodict and then traversing the dict directly.
Bonus
As a bonus, since our structure is already dict-like, we get another alternative implementation of xml2json for free. We just need to pass our dict-like structure to json.dumps. Something like:
def xml_to_json(xmlstr, **kwargs):
x = xml_to_dictlike(xmlstr, **kwargs)
return json.dumps(x)
If your xml includes attributes, you'd need to use some alphanumeric attr_prefix (e.g. "ATTR_"), to ensure the keys are valid json keys.
I haven't benchmarked this part.
