Appending tables generated from a loop

user_15139736 0
Viewed 71

I am a new python user here and am trying to append data together that I have pulled from a pdf using Camelot but am having trouble getting them to join together.

Here is my code:

url = 'https://www.fhfa.gov/DataTools/Downloads/Documents/HPI/HPI_AT_Tables.pdf'

tables = camelot.read_pdf(url,flavor='stream', edge_tol = 500, pages = '1-end')

i = 0

while i in range(0,tables.n):
    header = tables[i].df.index[tables[i].df.iloc[:,0]=='Metropolitan Statistical Area'].to_list()
    header = str(header)[1:-1]
    header = (int(header))
    tables[i].df = tables[i].df.rename(columns = tables[i].df.iloc[header])
    tables[i].df = tables[i].df.drop(columns = {'': 'Blank'})

    print(tables[i].df)
    #appended_data.append(tables[i].df)
    
#if i > 0:
#    dfs = tables[i-1].append(tables[i], ignore_index = True)
#pass

    i = i + 1

any help would be much appreciated

pandas append pdf loops
1 Answers

You can use pandas.concat() to concat a list of dataframe.

while i in range(0,tables.n):
    header = tables[i].df.index[tables[i].df.iloc[:,0]=='Metropolitan Statistical Area'].to_list()
    header = str(header)[1:-1]
    header = (int(header))
    tables[i].df = tables[i].df.rename(columns = tables[i].df.iloc[header])
    tables[i].df = tables[i].df.drop(columns = {'': 'Blank'})

df_ = pd.concat([table.df for table in tables])
user_103151630
Related