How to find the string offsets in xml using bsoup?

Question

I have an xml file which is output for Abby's OCR software.

The xml content can contain multiple text blocks and also multiple table blocks (As the OCR'd document looks similar to a research paper which contains paragraphs of text with some tables in between those paragraphs)

I am trying to extract the text from tables, which looks like this in the xml:

The non-table text is present as part of text blocks, an example below:

Now, my problem statement is: How do I get the global offsets for only the text in the tables?

This is how my code looks like for extracting text from tables:

soup = bs.BeautifulSoup(content,'lxml')
page_definition = "Page" + str(page) + "_"
tables = soup.find_all('composedblock',{"type": "table"}, id=lambda value: value and value.startswith(page_definition))
for table in tables:
    table_content = []
    vpos = []
    # Converting table xml to str for bs4 to be able to consume
    table = str(table)
    xml_soup = bs.BeautifulSoup(table, 'lxml')
    # Finding all  tags inside the table ComposedBlock
    text_blocks = xml_soup.find_all('textblock')
    
    # Looping over all text blocks inside the table
    for text_block in text_blocks:
        print(table.text.index(text_block.text))
        # Getting all vpos numbers, as different vpos signify different rows ..
        # .. due to the virtue of vertical positions
        vpos.append(text_block.get("vpos"))

    # Collecting all vpos of a table
    all_vpos = set(vpos)
    all_vpos = sorted(list(all_vpos))
    for vpos in all_vpos:
        txt_block_row = xml_soup.find_all("textblock", {"vpos": str(vpos)})

        # Finding all text in a single
        row = []
        for txt_block in txt_block_row:
            texts = txt_block.findAll("string")
            content_text =[]
            for text in texts:
                content = text.get('content')

user_10035985 · Answer

I hope I understood your question right. You want to print all indices of string that are inside table:

from bs4 import BeautifulSoup


txt = '''

    
        
        
        
    



    
        
            
        
    

'''

soup = BeautifulSoup(txt, 'html.parser')

all_strings = []
are_we_inside_table = []
current_index, indexes = 0, []
for s in soup.select('string[content]'):
    all_strings.append(s['content'])
    are_we_inside_table.append(bool(s.find_previous('composedblock', type='table')))
    indexes.append(current_index)
    current_index += len(s['content'])

for s, t, i in zip(all_strings, are_we_inside_table, indexes):
    if t:
        print(i, s)

Prints:

5 some text

5 because Mafi and , are strings but not inside table.

EDIT: To limit the search only on string on page 1, you can do thins change:

...
for s in soup.select('[id^="Page{}"] string[content]'.format(1)):
    ...

How to find the string offsets in xml using bsoup?

1 Answers