Comparing two HTML files using Python difflib package

Question

I am having issues with comparing two HTML files using Pythob difflib. While I was able to generate a comparison file that highlighted any changes, when I opened the comparison file, it displayed the raw HTML and CSS script/tags instead of the plain text.

E.g

 Text Here

instead of

Text Here

My Python Script is as follows:

import difflib

file1 = open('file1.html', 'r').readlines()
file2 = open('file2.html', 'r').readlines()
diffHTML = difflib.HtmlDiff()
htmldiffs = diffHTML.make_file(file1,file2)

with open('Comparison.html', 'w') as outFile:
     outFile.write(htmldiffs)

My input files looks something like this

                    
            
                    
                        Text here        
                    

                    
            
                            
                                Title
                            
                            
                                Sub title
                            
                            
                                Date & Time : Jul 2, 2020 2:42:48 PM
                            
            

                General
                
                
                    Text here
                
                
                        Text here
                
                
                    Description A :
                
                
                        Text here
                
                
                    Description b:
                
                
                        
                
                
                    Group:
                
                
                        Text here
                
                
                    Status:
                
                
                        
                            Active
                        
                
                
                    Version:
                
                
                        7
                
                
                    Last Modified:
                
                
                        Jun 15, 2020 2:41:48 PM

user_2663156 · Answer

Assuming that you are only looking for the text changes and not changes to the HTML, you could strip the output of HTML after the comparison. There are a number of ways to achieve this. The two that I first thought of was:

RegEx, because this is native in Python, and
BeautifulSoup, because it was created to read webpages

Create a function, using either of above methods, to strip the output of HTML

e.g. using BeautifulSoup

UPDATE

Reading through the documentation again, it seems to me that the comparison will yield the actual HTML too, however, you could create an additional output that only shows the text changes.

Also to avoid showing the whole document, I've set the context parameter to True

Using BeautifulSoup

import re
import difflib
from bs4 import BeautifulSoup

def remove_html_bs(raw_html):
    data = BeautifulSoup(raw_html, 'html.parser')
    data = data.findAll(text=True)
    
    def visible(element):
        if element.parent.name in ['style', 'script', '[document]', 'head', 'title']:
            return False
        elif re.match('', str(element.encode('utf-8'))):
            return False
        return True
    
    result = list(filter(visible, data))

    return result

def compare_files(file1, file2):
    "Creates two comparison files"
    file1 = file1.readlines()
    file2 = file2.readlines()
    
    # Difference line by line - HTML
    difference_html = difflib.HtmlDiff(tabsize=8).make_file(file1, file2, context=True, numlines=5)
    # Difference line by line - raw
    difference_file = set(file1).difference(file2)
    
    # List of differences by line index
    difference_index = []
    for dt in difference_file:
        for index, t in enumerate(file1):
            if dt in t:
                difference_index.append(f'{index}, {remove_html_bs(dt)[0]}, {remove_html_bs(file2[index])[0]}')

    # Write entire line with changes
    with open('comparison.html', 'w') as outFile:
        outFile.write(difference_html)

    # Write only text changes, by index
    with open('comparison.txt', 'w') as outFile:
        outFile.write('LineNo, File1, File2
')
        outFile.write('
'.join(difference_index))

    return difference_html, difference_file


file1 = open('file1.html', 'r')
file2 = open('file2.html', 'r')

difference_html, difference_file = compare_files(file1, file2)

Comparing two HTML files using Python difflib package

1 Answers