I have two search results from a web-service saved as html that i have to parse with BeautifulSoup in order to extract some data. I noticed that one of them takes approx. 35 times longer than the other.
Someone has an explanation for this / an idea what i can do in order to improve the performance of the slower html file?
Setup:
Python 2.7.13
Jupyter Notebook 4.3.1
beautifulsoup4 (4.5.3)
lxml (3.8.0)
Code:
from bs4 import BeautifulSoup
path = "path to the files"
file_1 = "slow.html"
file_2 = "fast.html"
with open(path+file_1) as rfile_1:
html_1 = rfile_1.read()
with open(path+file_2) as rfile_2:
html_2 = rfile_2.read()
%timeit soup = BeautifulSoup(html_1, 'lxml')
>> 1 loop, best of 3: 4.67 s per loop
%timeit soup = BeautifulSoup(html_2, 'lxml')
>> 10 loops, best of 3: 136 ms per loop