Python PDF Parsing with Camelot and Extract the Table Title

user_2600860 0
Viewed 2961

Camelot is a fantastic Python library to extract the tables from a pdf file as a data frame. However, I'm looking for a solution that also returns the table description text written right above the table.

The code I'm using for extracting tables from pdf is this:

import camelot
tables = camelot.read_pdf('test.pdf', pages='all',lattice=True, suppress_stdout = True)

I'd like to extract the text written above the table i.e THE PARTICULARS, as shown in the image below.

What should be a best approach for me to do it? appreciate any help. thank you

enter image description here

tabula python-camelot pdfminer python
2 Answers

You can create the Lattice parser directly 

            parser = Lattice(**kwargs)
            for p in pages:
                t = parser.extract_tables(p, suppress_stdout=suppress_stdout,
                                          layout_kwargs=layout_kwargs)
                tables.extend(t)

Then you have access to parser.layout which contains all the components in the page. These components all have bbox (x0, y0, x1, y1) and the extracted tables also have a bbox object. You can find the closest component to the table on top of it and extract the text.

user_120072710

Here's my hilariously bad implementation just so that someone can laugh and get inspired to do a better one and contribute to the great camelot package :)

Caveats:

  • Will only work for non-rotated tables
  • It's a heuristic
  • The code is bad
# Helper methods for _bbox
def top_mid(bbox):
    return ((bbox[0]+bbox[2])/2, bbox[3])

def bottom_mid(bbox):
    return ((bbox[0]+bbox[2])/2, bbox[1])

def distance(p1, p2):
    return math.sqrt((p1[0]-p2[0])**2 + (p1[1]-p2[1])**2)

def get_closest_text(table, htext_objs):
    min_distance = 999  # Cause 9's are big :)
    best_guess = None
    table_mid = top_mid(table._bbox)  # Middle of the TOP of the table
    for obj in htext_objs:
        text_mid = bottom_mid(obj.bbox)  # Middle of the BOTTOM of the text
        d = distance(text_mid, table_mid)
        if d < min_distance:
            best_guess = obj.get_text().strip()
            min_distance = d
    return best_guess

def get_tables_and_titles(pdf_filename):
    """Here's my hacky code for grabbing tables and guessing at their titles"""
    my_handler = PDFHandler(pdf_filename)  # from camelot.handlers import PDFHandler
    tables = camelot.read_pdf(pdf_filename, pages='2,3,4')
    print('Extracting {:d} tables...'.format(tables.n))
    titles = []
    with camelot.utils.TemporaryDirectory() as tempdir:
        for table in tables:
            my_handler._save_page(pdf_filename, table.page, tempdir)
            tmp_file_path = os.path.join(tempdir, f'page-{table.page}.pdf')
            layout, dim = camelot.utils.get_page_layout(tmp_file_path)
            htext_objs = camelot.utils.get_text_objects(layout, ltype="horizontal_text")
            titles.append(get_closest_text(table, htext_objs))  # Might be None

    return titles, tables

See: https://github.com/atlanhq/camelot/issues/395

user_9292890
Related