Retrieve many nested statements at the same time for a single object with lxml Python

user_7879557 0
Viewed 42

I am working with big xml where I am retrieving many different properties, and now I am trying to retrieve comment category property and connect it to the text between the tags. However, there are 3 different situations that I need to handle. XML example:

<comment-list>
 <comment category="Derived from sampling site"> Peripheral blood </comment>
 <comment category="Transformant">
   <cv-term terminology="NCBI-Taxonomy" accession="10376">Epstein-Barr virus (EBV)</cv-term>
 </comment>
 <comment category="Sequence variation"> Hemizygous for FMR1 &gt;200 CGG repeats (PubMed=25776194) 
 </comment>
 <comment category="Monoclonal antibody target">
   <xref-list>
     <xref database="UniProtKB" category="Sequence databases" accession="Q5T5X7">
       <property-list>
         <property name="gene/protein designation" value="Human BEND3"/>
       </property-list>
       <url><![CDATA[https://www.uniprot.org/uniprot/Q5T5X7]]></url>
     </xref>
   </xref-list>
 </comment>
 </comment-list>
  1. When <comment> does not have child tags under. Then I need to retrieve comment category property and connect it with the text between the tags.
  2. When <comment> has a <cv-term> tag nested underneath. Then I need to retrieve comment category, cv-term terminology, cv-term accession and the text between the cv-term tags.
  3. When <comment> has several tags nested underneath: <xref-list>-<xref>-<property-list>- <property>-<url>. In this case I need to retrieve: comment category, xref database property, xref accession property, and property value property.

I am using lxml to parse this XML, and I am struggling to wrap my head around how to solve case 2. Case 1 and 3 work but when an object has all three cases then the output gets messed up.

I would like to receive following output:

Derived from sampling site: Peripheral blood
Transformant: NCBI-Taxonomy, 10376, Epstein-Barr virus (EBV)
Sequence variation: Hemizygous for FMR1 &gt;200 CGG repeats (PubMed=25776194)
Monoclonal antibody target: UniProtKB, Q5T5X7, Human BEND3

Here is my very messy code which outpus the elements in wrong order. It worked fine for case 1 and 3, but when case 2 comes into play then the output is ordered wrong:

comment_cat = att.xpath('.//comment-list/comment/@category')
comment_text = att.xpath('.//comment-list/comment/text()') 
cv_term = att.xpath('.//comment-list/comment/cv-term/text()')
xref = [a + ', ' + b for a,b in zip(att.xpath('.//comment-list/comment/xref- 
list/xref/@database'),att.xpath('.//comment-list/comment/xref-list/xref/@accession'))]
property_list = att.xpath('.//comment-list/comment/xref-list/xref/property-list/property/@value')
xref_property_list = [a + ', ' + b for a,b in zip(xref, property_list)]
empty_str_in_text = ['\n      ', '\n    ', '\n      ', '\n    ']
comment_texts_all = cv_term+comment_text+xref_property_list

for e in empty_str_in_text:
    if e in comment_texts_all:
        comment_texts_all.remove(e)    
key_values['Comments'] = ';; '.join([i + ': ' + j for i, j in zip(comment_cat, 
comment_texts_all)])

Output:

Derived from sampling site: Epstein-Barr virus (EBV);; 
Transformant:  Peripheral blood ;; 
Sequence variation:  Hemizygous for FMR1 >200 CGG repeats (PubMed=25776194) ;; 
Monoclonal antibody target: UniProtKB, Q5T5X7, Human BEND3
lxml xml-parsing python
1 Answers

Here is a slightly alternative approach:

    xml = '''<comment-list>
    <comment category="Derived from sampling site"> Peripheral blood </comment>
    <comment category="Transformant">
        <cv-term terminology="NCBI-Taxonomy" accession="10376">Epstein-Barr virus (EBV)</cv-term>
    </comment>
    <comment category="Sequence variation"> Hemizygous for FMR1 &gt;200 CGG repeats (PubMed=25776194)</comment>
    <comment category="Monoclonal antibody target">
        <xref-list>
            <xref database="UniProtKB" category="Sequence databases" accession="Q5T5X7">
                <property-list>
                    <property name="gene/protein designation" value="Human BEND3"/>
                </property-list>
                <url><![CDATA[https://www.uniprot.org/uniprot/Q5T5X7]]></url>
            </xref>
        </xref-list>
    </comment>
    <comment category="Knockout cell">
        <method>KO mouse</method>
        <xref-list>
            <xref database="MGI" category="Organism-specific " accession="MGI:97740">
                <property-list>
                    <property name="gene/protein designation" value="Polb"/>
                </property-list>
                <url><![CDATA[http://www.informatics.jax.org//MGI:97740]]></url>
            </xref>
        </xref-list>
    </comment>
</comment-list>'''

from lxml import etree as ET

tree = ET.fromstring(xml)

result = ''

for comment in tree.iter('comment'):
    result += f"{comment.get('category')}: "
    cv_term = comment.find('cv-term')
    xref_list = comment.find('xref-list')
    method = comment.find('method')
    if len(list(comment)) == 0:
        result += comment.text
    elif cv_term is not None:
        result += ', '.join([cv_term.get('terminology'), cv_term.get('accession'), cv_term.text])
    elif xref_list is not None and method is None:
        result += ', '.join([xref_list.xpath('./xref/@database')[0], xref_list.xpath('./xref/@accession')[0], xref_list.xpath('./xref/property-list/property/@value')[0]])
    elif method is not None:
        result += method.text
    result += '\n'

print(result)

Output:

Derived from sampling site:  Peripheral blood 
Transformant: NCBI-Taxonomy, 10376, Epstein-Barr virus (EBV)
Sequence variation:  Hemizygous for FMR1 >200 CGG repeats (PubMed=25776194)
Monoclonal antibody target: UniProtKB, Q5T5X7, Human BEND3
Knockout cell: KO mouse
user_141686230
Related