How to change batch size of ImageDataGenerator after instantiating it?

user_10729292 0
Viewed 1379

The method I am aware of is something like this

from tensorflow.keras.preprocessing.image import ImageDataGenerator

train_datagen = ImageDataGenerator(rescale=1./255)
val_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
        train_dir,  
        target_size=(150, 150), 

        batch_size=20,             <---------------------------

        class_mode='binary')

But I want to change batch size while training in model.fit() method and it won't happen because batch_size has already been set in flow_from_directory()

So how do I load this dataset so that I've the freedom to change the batch_size while training? All the efforts are highly appreciated

keras tensorflow deep-learning python
3 Answers

You can change the batch size after creating the ImageDataGenerator object:

train_generator.batch_size = 2

The batches will then be of size 2.

user_109083750

The answer from @Nicolas is correct. You can easily choose the batch size layer after creating a generator. One additional piece of information I like brings here about batch_size in the model.fit. According to the doc

batch_size: ... Do not specify the batch_size if your data is in the form of datasets, generators, or keras. utils.Sequence instances (since they generate batches).

So, according to the documentation, we shouldn't specify the batch_size if we use generator as they generate batches for the training.

user_92157800

Here's another way it worked for me I copied the code from keras.dataset.cifar10 and used the link of cats_vs_dogs dataset

from tensorflow.python.keras import backend
from tensorflow.python.keras.datasets.cifar import load_batch
from tensorflow.python.keras.utils.data_utils import get_file
from tensorflow.python.util.tf_export import keras_export

dirname = 'cifar-10-batches-py'
origin = 'https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip '
path = get_file(
    dirname,
    origin=origin,
    untar=True,
    file_hash=
    '6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce')

num_train_samples = 50000

x_train = np.empty((num_train_samples, 3, 32, 32), dtype='uint8')
y_train = np.empty((num_train_samples,), dtype='uint8')

for i in range(1, 6):
  fpath = os.path.join(path, 'data_batch_' + str(i))
  (x_train[(i - 1) * 10000:i * 10000, :, :, :],
    y_train[(i - 1) * 10000:i * 10000]) = load_batch(fpath)

fpath = os.path.join(path, 'test_batch')
x_test, y_test = load_batch(fpath)

y_train = np.reshape(y_train, (len(y_train), 1))
y_test = np.reshape(y_test, (len(y_test), 1))

if backend.image_data_format() == 'channels_last':
  x_train = x_train.transpose(0, 2, 3, 1)
  x_test = x_test.transpose(0, 2, 3, 1)

x_test = x_test.astype(x_train.dtype)
y_test = y_test.astype(y_train.dtype)
user_107292920
Related