How to create Numpy Arrays from images of a folder including sub-folders

Question

Here is my folder structure:

Human
├── Man
│   ├── img02.jpg
│   └── img02.jpg
├── Woman
│   ├── img03.jpg
│   └── img03.jpg

I want to convert pixels of all images into numpy array. I have tried with this:

import cv2
import glob
import numpy as np

X_data = []
files = glob.glob("Human/*.jpg")
for myFile in files:
    print(myFile)
    image = cv2.imread (myFile)
    X_data.append (image)

type(X_data)

It's giving me list as output. But, I want that, if I run print(X_data), my final output will be something like this:

{'data': array([[5.1, 3.5, 1.4, 0.2],
       [4.9, 3. , 1.4, 0.2],
       ....................
       [5.9, 3. , 5.1, 1.8]]), 'target': array([0, 0, ...... 1, 1]), 'target_names': array([],....)

How to convert the pixels into python data structure?

user_176769 · Answer

The structure that you are looking for is called a dictionary in Python.

The main idea is to create X_data as a dictionary with predefined keys. Then, loop through the files to extract relevant information from each image (pixels, filename, subdirectory) and store them in separate lists. Later, convert each list into a numpy array and just add them to the dictionary.

The source code below demonstrates how to accomplish this and shows how to iterate through the data on the dictionary and display the images along with their filename and subdirectory:

import cv2
import glob
import numpy as np
import pprint

X_data = { 'data': None, 'target': None, 'target_name': None }

# iterate through the target directory (recursively) finding .jpg files
target_dir = 'Human'
files = []
for f in glob.glob(target_dir + '/**/*', recursive=True):
    if f[-4:] == '.jpg':
        files.append(f)
#print('files=', files)

# create separate lists to store the data
data_list = []
target_list = []
target_name_list = []

# iterate through the files
for filename in files:
    # load input image
    image = cv2.imread(filename)
    #print('f=', filename)

    # remove the target folder name from the path
    subdir_plus_file = filename[len(target_dir)+1:]
    #print('subdir_plus_file=', subdir_plus_file)

    # extract the subdirectory
    slash_idx = subdir_plus_file.index('\')
    if (slash_idx):
        subdir = subdir_plus_file[:slash_idx]
    else:
        subdir = 'None'
    #print('subdir=', subdir)

    # append each piece of data into its respective list
    data_list.append(image)
    target_list.append(filename)
    target_name_list.append(subdir)

# convert each list into a np.array and then assemble them into a single data structure (dictionary)
X_data['data'] = np.array(data_list)
X_data['target'] = np.array(target_list)
X_data['target_name'] = np.array(target_name_list)

# print the dictionary contents
#pprint.pprint(X_data, depth=1)

# display each image along with its respective target and target_name
for idx, _ in enumerate(X_data['target']):    
    cv2.imshow('output', X_data['data'][idx])
    print('File #' , idx, '  target=', X_data['target'][idx], '  target_name=', X_data['target_name'][idx])
    cv2.waitKey(0)

Output:

{'data': array([array([[[255, 255, 255],
        [255, 255, 255],
        [255, 255, 255],
        ...,
        [176, 171, 170],
        [174, 169, 168],
        [174, 169, 168]]], dtype=uint8)], dtype=object),
 'target': array(['Human\Man\dates.jpg', 'Human\Woman\legos.jpg'], dtype='

How to create Numpy Arrays from images of a folder including sub-folders

1 Answers