image_dataset_from_directory much faster than flow_from_dataframe. How to speed it up?

user_3578468 0
Viewed 419

I have an existing evaluation suite that was built for the scikit-learn API. I work with datasets that fit into memory and don't require flowing the data from disk. However, I observe that when I load data with my low level loading function load_images I get worse model performance than when I load with either image_dataset_from_directory or flow_from_dataframe. I could refactor my existing code base to work based on pandas dataframes and use flow_from_dataframe, since I need to make these loaders work with scikit-learn KFold splits. There seems to be no straight forward way to do that with image_dataset_from_directory, but with flow_from_dataframe the index selection makes that possible. The problem now it, that flow_from_dataframe is INCREDIBLY slow. I trained 30 models, and it took over 3 hours with flow_from_dataframe while with flow_from_dataframe it took 8 minutes!

So, I now have 2 options:

  • Speed up flow_from_dataframe
  • Find out what magic preprocessing flow_from_dataframe and image_dataset_from_directory do that makes the model fit the data better than when loading the data with load_images.

I spend a day now on these two options, but failed to do either.

TL;DR:

a) Do you know what the critical difference in my low level way of loading the data is compared to the other two high level methods?

b) Do you know how to speed up flow_from_dataframe?

Measurements and code:

Here are prediction quality measurements I made:

with flow_from_dataframe:

enter image description here

with image_dataset_from_directory:

enter image description here

with load_images:

enter image description here

I did pairwise t-tests. flow_from_dataframe and image_dataset_from_directory are not significantly different from each other on alpha = .05 and load_images is significantly different (plain to see, actually).

Here are the load functions I used for the evaluation:

def load_images(self, image_paths):

    def load_image(path):
        img = keras.preprocessing.image.load_img(path, target_size=input_shape)
        img = keras.preprocessing.image.img_to_array(img)
        img = img.reshape(1, *input_shape, 3)
        return img

    input_shape = self.model.input.shape[1], self.model.input.shape[2]

    images = np.vstack(list(map(load_image, image_paths)))

    return image        
   

def load_train_data_with_image_dataset_from_directory(data_dir, batch_size, img_height, img_width):
    
    def make_split_ds(split):
        
        ds = tf.keras.preprocessing.image_dataset_from_directory(data_dir, validation_split=0.2, subset=split,
                                                                 image_size=(img_height, img_width), seed=0,
                                                                 batch_size=batch_size, interpolation="nearest",
                                                                 label_mode="categorical")
        return ds
    
    train_ds = make_split_ds("training")
    val_ds = make_split_ds("validation")
    class_names = train_ds.class_names
        
    return train_ds, val_ds, class_names


def load_train_data_with_flow_from_dataframe(data_dir, batch_size, img_height, img_width):
    
    def flow_split(df):
        
        splitgen = datagen.flow_from_dataframe(df, directory=TRAIN_DATA_DIR, shuffle=True,
                                               batch_size=batch_size, target_size=(img_height, img_width),
                                               interpolation="nearest")
        return splitgen

    data, classes = load_data(["train"])

    X_train, y_train = data["train"]["paths"], data["train"]["labels"]
    
    df = pd.DataFrame({"filename": X_train, "class": y_train}).sample(frac=1).reset_index(drop=True)
    
    datagen = ImageDataGenerator()
    
    train_ds = flow_split(df.iloc[: int(len(df) * .8)])
    val_ds = flow_split(df.iloc[int(len(df) * .8): ])
    
    return train_ds, val_ds, classes
keras tensorflow dataframe image python
1 Answers

I found a way to wrap image_dataset_from_directory in a way that allows to use in a more flexible way by unrolling the built-in iterator to get a normal X and y arrays.

def load_data(*splits, label_mode="int", **kwargs):

    def load_data_from_directory(data_dir):
        return tf.keras.preprocessing.image_dataset_from_directory(data_dir, label_mode=label_mode, **kwargs)

    def memoize(dataset):
        nonlocal classes

        if not classes:
            classes = dataset.class_names
        elif classes and dataset.class_names != classes:
            raise logger.warning("Splits differ in represented classes.")

        dataset = dataset.cache().prefetch(buffer_size=tf.data.AUTOTUNE)

        return dataset

    def unroll(dataset):

        X, y = map(np.concatenate, zip(*[b for b in dataset.as_numpy_iterator()]))
        return X, y

    logger = logging.getLogger(__name__)
    classes = None

    data_dirs = ([TRAIN_DATA_DIR] * ("train" in splits)) + ([TEST_DATA_DIR] * ("test" in splits))

    split_ds = map(memoize, map(load_data_from_directory, data_dirs))
    X, y = map(np.array, map(list, map(chain.from_iterable, zip(*map(unroll, split_ds)))))

    return X, y, classes
user_35784680
Related