When getting a file from the internet, reading it into memory, not a file, and then comparing the contents of it to a local file.
The efficiency and ease of filecmp is what I'm looking to achieve or find but so far I'm unable to increment through the byte lists without slowing to a crawl. So if there is an existing library or a simple way to compare the contents of a BytesIO object that would be great.
newfile = file_name + e[e.rfind('.'):]
urllib.request.urlretrieve(e, directory + newfile)
print('Image sucessfully Downloaded: ',file_name)
for filename in os.listdir(directory):
f = os.path.join(directory, filename)
f2 = os.path.join(directory, newfile
if os.path.isfile(f):
if(filecmp.cmp(f, f2, shallow=False)) & (f != f2):
os.remove(f2)
print('Duplicate Image, Removing')