Handling File Pathing - Python 3.8

Question

I have a question regarding glob and file pathing - lets say I have a static file path in the form of

G:\ML_CDetector_ImageArchive\BreastCancer\RawFolder\Duke-Breast-Cancer-MRI\Duke-Breast-Cancer-MRI

I want to add variable pathing for the folders afterwards since the folder naming convention is dissimilar in each patient - while the patient folder convention is standard across the board like:

Breast_MRI_XXX - Where x is a number from 1-922 (Which I have been able to handle though the use of a while loop) and finally the inner folder is where it gets a bit funky but once again I havent been able to handle this through glob with the following use of a wildcard operator:

f"{currentPatient}\*\ Now within the folder there are several more folders which I want to enter such that there is a partial match of a name:

gl.glob(f"{currentPatient}\*\[3rd]*\*.dcm") but to my dismay I have not been able to acquire the correct folder as when I do

for item in globInnerFolder:
                print(item)

I am returned an empty print in the form of:

[]
[]
[]
[]
[]
[]
[]
[]
[]
[]

Nothing shows up and there are no errors - how would I go about doing the following - Quick note I have tried the use of glob and iglob to see if there is ANYTHING returned and nada:

    fpMainFolder        = gl.glob("G:\ML_CDetector_ImageArchive\BreastCancer\RawFolder\Duke-Breast-Cancer-MRI\Duke-Breast-Cancer-MRI\")
    # Loop Variable
    j = 0
    
    # while (j < len(csvList)):
    while (j < 10):
        # Image Configuration
        currentPatient      = str(csvList[j][0])
        # yStartPixel       = int(csvList[j][1])
        # yEndPixel         = int(csvList[j][2])
        # xStartPixel       = int(csvList[j][3])
        # xEndPixel         = int(csvList[j][4])
        # sliceStart        = int(csvList[j][5])
        # sliceEnd          = int(csvList[j][6])
        
        #imageSize          = (yEndPixel - yStartPixel), (xEndPixel - xStartPixel)
        
        # Patient Folder Loop Variable
        try:
            globInnerFolder     = gl.glob(f"{currentPatient}\*\[3rd]*\*.dcm")
            print(globInnerFolder)
            for item in globInnerFolder:
                print(item)
        except:
            try:
                globInnerFolder = gl.iglob(f"{currentPatient}\*\[ph3ax]*\*.dcm")
                for item in globInnerFolder:
                    print(item)
            except:
                print("Exiting - No File Structure Found")
                exit()

Here is an example of a full file path

G:\ML_CDetector_ImageArchive\BreastCancer\RawFolder\Duke-Breast-Cancer-MRI\Duke-Breast-Cancer-MRI\Breast_MRI_001\01-01-1990-MRI BREAST BILATERAL WWO-97538\3.000000-ax dyn pre-93877

user_11714936 · Answer

I found a solution to my problem - Using glob I can iterate through a folder and return a list with all given folders using this methodology I can select the appropriate element from the list and considering that the number of inner file paths after the patient is static (meaning only one folder is located within each patient followed by more folders within that structure)

I used glob again to match the specific inner folder and used the first element in the list to return a string of the folder path and then using string concatenation I built a dynamic folder search by using glob to match the inner folders.

    fpMainFolder        = "G:\ML_CDetector_ImageArchive\BreastCancer\RawFolder\Duke-Breast-Cancer-MRI\Duke-Breast-Cancer-MRI\"
    j = 0
    while j < len(csvList):
        currentPatient      = str(csvList[j][0])
        yStartPixel         = int(csvList[j][1])
        yEndPixel           = int(csvList[j][2])
        xStartPixel         = int(csvList[j][3])
        xEndPixel           = int(csvList[j][4])
        sliceStart          = int(csvList[j][5])
        sliceEnd            = int(csvList[j][6])
        fpPatientPath       = f"{currentPatient}"
        fpMriPass           = glob((fpMainFolder + fpPatientPath + "\*\" ))
        fpMriPass           = fpMriPass[0]
        fpThirdPass         = glob((fpMriPass + "\*3rd*\"))
        if len(fpThirdPass) == 0:
            fpThirdPass     = glob((fpMriPass + "\*Ph3ax*\"))
        print(fpThirdPass)
        
        for i in range((sliceEnd - sliceStart) + 1):
            try:
                fpDCMImage          = "1-" + "0" + str(sliceStart + i) + ".dcm"
                completeFp          = fpThirdPass[0] + fpDCMImage
                openDicom           = dcmread(completeFp)
                # Matplotlib -> Image Show -> Pixel Limits of X and Y -> Tumor Location
                plt.imsave("G:\ML_CDetector_ImageArchive\BreastCancer\TrainingSet\PatientImages\TFJPEG_"      + currentPatient + "_TumourImageSlice"      + str(i) + ".jpg",
                           openDicom.pixel_array[yStartPixel:yEndPixel,xStartPixel:xEndPixel],
                           cmap=plt.cm.twilight,
                           origin = 'lower')
            except:
                try:
                    fpDCMImage          = "1-" + str(sliceStart + i) + ".dcm"
                    completeFp          = fpThirdPass[0] + fpDCMImage
                    openDicom           = dcmread(completeFp)
                    # Matplotlib -> Image Show -> Pixel Limits of X and Y -> Tumor Location
                    plt.imsave("G:\ML_CDetector_ImageArchive\BreastCancer\TrainingSet\PatientImages\TFJPEG_"   + currentPatient + "_TumourImageSlice"      + str(i) + ".jpg",
                           openDicom.pixel_array[yStartPixel:yEndPixel,xStartPixel:xEndPixel],
                           cmap=plt.cm.twilight,
                           origin = 'lower')
                except Exception as e:
                    print(e)
        j += 1

In the instance of no matches being found, I used an if statement to check the size of the list that was being returned. If glob was not able to match the first format I had the list would return a length of 0, I ran a second matching parameter which would return an array with a size greater than 0 thus I could once again use the first element in the list and grab the file path from that.

        fpMriPass           = glob((fpMainFolder + fpPatientPath + "\*\" ))
        fpMriPass           = fpMriPass[0]
        fpThirdPass         = glob((fpMriPass + "\*3rd*\"))
        if len(fpThirdPass) == 0:
            fpThirdPass     = glob((fpMriPass + "\*Ph3ax*\"))
        print(fpThirdPass)

This may not be the most efficient solution but it did solve my problem of not being able to access folders that were not following a standard naming convention.

I hope this helps as future reference.

Handling File Pathing - Python 3.8

1 Answers