How to run a script on each folder in a drive?

user_19429935 0
Viewed 38

I am working on a data cleanup in a network drive. The drive has 1000+ folders, and those folders have several subfolders. The script that I got from G4G (seen below) prompts me to select a folder. I can click on one of my 1000+ folders, and the data is cleaned up properly (duplicates are deleted). However, I'd like to loop the command through the whole drive to avoid clicking on folders for hours. I cannot select the drive as my folder because duplicate file names between the first folders in the drive should not be considered duplicates.

Example: Z:/Folder1 and Z:/Folder2 both have several files named text.txt, immediately inside of the folders and within the subdirectories of the folders. Folder1 and Folder2, among all text.txt files immediately inside and within its subdirectories, should each be left with one text.txt. If the current script is applied to Folder1 and Folder2 individually, then the desired result of one text.txt file existing in Folder1 and one existing in Folder2 is accomplished. If the script is applied to the Z: drive, then between Folder1 and Folder2, there would only be one text.txt, and one of the folders would be without a file named text.txt.

How can I apply this script to each first folder in the drive without having to manually click on each folder?

from tkinter.filedialog import askdirectory

# Importing required libraries.
from tkinter import Tk
import os
import hashlib
from pathlib import Path

# We don't want the GUI window of
# tkinter to be appearing on our screen
Tk().withdraw()

# Dialog box for selecting a folder.
file_path = askdirectory(title="Select a folder")

# Listing out all the files
# inside our root folder.
list_of_files = os.walk(file_path)

# In order to detect the duplicate
# files we are going to define an empty dictionary.
unique_files = dict()

for root, folders, files in list_of_files:

    # Running a for loop on all the files
    for file in files:

        # Finding complete file path
        file_path = Path(os.path.join(root, file))

        # Converting all the content of
        # our file into md5 hash.
        Hash_file = hashlib.md5(open(file_path, 'rb').read()).hexdigest()

        # If file hash has already #
        # been added we'll simply delete that file
        if Hash_file not in unique_files:
            unique_files[Hash_file] = file_path
        else:
            if file.endswith((".txt",".bmp")):
                os.remove(file_path)
                print(f"{file_path} has been deleted")
data-cleaning duplicates python
1 Answers

You can change the script as below. Basically below script gets absolute path of all the directories in the current directory, and feeds them one by one for cleanup.

from tkinter.filedialog import askdirectory

# Importing required libraries.
from tkinter import Tk
import os
import hashlib
from pathlib import Path

# We don't want the GUI window of
# tkinter to be appearing on our screen
Tk().withdraw()

# Dialog box for selecting a folder.
file_paths = [os.path.abspath(i) for i in os.listdir() if os.path.isdir(i)]

for file_path in file_paths:
    # Listing out all the files
    # inside our root folder.
    list_of_files = os.walk(file_path)

    # In order to detect the duplicate
    # files we are going to define an empty dictionary.
    unique_files = dict()

    for root, folders, files in list_of_files:

        # Running a for loop on all the files
        for file in files:

            # Finding complete file path
            file_path = Path(os.path.join(root, file))

            # Converting all the content of
            # our file into md5 hash.
            Hash_file = hashlib.md5(open(file_path, 'rb').read()).hexdigest()

            # If file hash has already #
            # been added we'll simply delete that file
            if Hash_file not in unique_files:
                unique_files[Hash_file] = file_path
            else:
                if file.endswith((".txt",".bmp")):
                    os.remove(file_path)
                    print(f"{file_path} has been deleted")
user_136082280
Related