Python and NLP, extract key words and person name from multiple pdf files using NLP & Python

Viewed 43

I hope to extract some information from multiple pdf files (e.g. xxx1.pdf, xxx2.pdf, xxx3.pdf...), the output dataframe including 4 fields: 1.file name, 2.the context of each pdf, 3.specific keywords, and 4.the Person's name related to the keyword.

I partly wrote some code to generate the first three fields, apart from the person's name, I assume it is more about NLP skills, e.g. Spacy, NLTK etc, which I am very new here. What I wish to generate is if the keyword ('Chair Man' in my case) be found in each pdf file, then search the closet Person's name to the key words (before or after), then put the name in the dataframe. I may sense the logic is to calculate the shortest distance between a Person name and the key word, but I could not make it, here is my part of code:

import pandas as pd
import numpy as np
from tkinter import filedialog
from pathlib import Path
import os

import PyPDF2


file_path = filedialog.askdirectory()
file_list = Path(file_path + '/').rglob('*.pdf')

file_names = os.listdir(file_path)
file_names = [file for file in file_names if file.endswith('.pdf')]
print(len(file_names))

file_df = []
context = []

for file in file_list:
    print(file.stem)
    file_name = file.stem

    pdf = PyPDF2.PdfFileReader(file)
    page_num = pdf.getNumPages()
    text = ''
    for i in range(0, page_num):
        PgOb = pdf.getPage(i)
        text += PgOb.extractText()
    # print(text)
    file_df.append(file_name)
    context.append(text)

pdf_df = pd.DataFrame({'File': file_df, 'Context': context})


pdf_df['Chair_Man_Check'] = np.nan
pdf_df.loc[pdf_df['Context'].str.contains('Chair Man'), 'Chair_Man_Check'] = 'Chair Man  - Yes'
print(pdf_df)

the ideal output is like this below:

  File_Name                    Context  Chair_Man_Check          Name
  xxx1.pdf  file context blah blah...   Chair Man -Yes   James Volks
  xxx2.pdf                  blah.....              NaN           NaN
  xxx3.pdf                   blahhhhh  Chair Man - Yes  Tim Aloepman

Could you provide some help please? many thanks!

0 Answers
Related