Compare two excel spreadsheets with different row values and coordinates using pandas

Viewed 320

I am making an excel comparing program with pandas. I have made a simple comparing tool that works well, but it compares row-by-row and displays changes that appear in other sections of the column. This happens because the row coordinates are not equal in both sheets. To clarify a little bit, here is my code:

import pandas as pd
import numpy as np
import openpyxl

wb = openpyxl.load_workbook('CK_CBF_Draft_01.2018_original.xlsx')
ws = wb['CBF']

list1 = []

for i in ws['H1':'H365']:
    for cell in i:
        list1.append(i)

# Define the diff function to show the changes in each field
def report_diff(x):
    return x[1] if x[1] in list1 else '{} ---> {}'.format(x[0],x[1])

# We want to be able to easily tell which rows have changes
def has_change(row):
    if "--->" in row.to_string():
        return "Y"
    else:
        return "N"

# Read in both excel files
df1 = pd.read_excel('Invoice1.xlsx', 'Sheet1', na_values=['NA'])
df2 = pd.read_excel('Invoice2.xlsx', 'Sheet1', na_values=['NA'])

# Make sure we order by account number so the comparisons work
df1.sort_values(by="Host Name")
df1=df1.reindex()
df2.sort_values(by="Host Name")
df2=df2.reindex()

# Create a panel of the two dataframes
diff_panel = pd.Panel(dict(df1=df1,df2=df2))

#Apply the diff function
diff_output = diff_panel.apply(report_diff, axis=0)

# Flag all the changes
diff_output['has_change'] = diff_output.apply(has_change, axis=1)

#Save the changes to excel but only include the columns we care about
diff_output[(diff_output.has_change == 'Y')].to_excel('my-diff-1.xlsx',index=False,columns=["Host Name","CPU#","Memory","Invoice Total","Quantity"])

print('Worked')

The problem, as I have stated, is the fact that this returns a row-by-row difference, and the differences are incorrect as they appear in different parts of the column. Does anyone know a way of comparing two files with different rows accurately?

Thanks for your help and sorry if the question is a little vague.

1 Answers
import pandas as pd
import numpy as np

# Read both Excel files
file1 = pd.read_excel("file1.xlsx", na_values=['NA'])
file2 = pd.read_excel("file2.xlsx", na_values=['NA'])


df2 = file1
df1 = file2


res = df1[df1['samecolname'].isin(df2['samecolname'].unique())]
                   
res2 = df2[df2['samecolname'].isin(df1['samecolname'].unique())]               

res.to_excel('diff1-insecond-but-not-in-first.xlsx',index=False)
res2.to_excel('diff2-in-first-not-in-second.xlsx',index=False)
Related