generator called at the wrong time (keras)

Viewed 430

I use fit_generator() in keras 2.0.2 with batch size 10 and steps 320 because I have 3209 samples for training. Before the first epoch begins, the generator was called 11 times, showing:

Train -- get ind: 0 to 10
    ...    
Train -- get ind: 100 to 110

Then, after the first batch (1/320), it prints out Train -- get ind: 110 to 120, but I think it should be Train -- get ind: 0 to 10. Is my implementation for the train_generator() function incorrect? Or why am I having this issue?

Here is my code for the generator:

EPOCH = 10
x_train_img = img[:train_size] # shape: (3209,512,512)
x_test_img = img[train_size:]  # shape: (357,512,512)

def train_generator():
    global x_train_img

    last_ind = 0

    while 1:
        x_train = x_train_img[last_ind:last_ind+BATCH_SIZE]
        print('Train -- get ind: ',last_ind," to ",last_ind+BATCH_SIZE)
        last_ind = last_ind+BATCH_SIZE
        x_train = x_train.astype('float32') / 255.
        x_train = np.reshape(x_train, (len(x_train), 512, 512, 1)) 
        yield (x_train, x_train)
        if last_ind >= x_train_img.shape[0]:
             last_ind = 0

def test_generator():
     ...

train_steps = x_train_img.shape[0]//BATCH_SIZE #320
test_steps = x_test_img.shape[0]//BATCH_SIZE   #35

autoencoder.fit_generator(train_generator(), 
                steps_per_epoch=train_steps, 
                epochs=EPOCH,
                validation_data=test_generator(),
                validation_steps=test_steps,
                callbacks=[csv_logger] )

A better? way of writing the generator:

def train_generator():
    global x_train_img

    while 1:
        for i in range(0, x_train_img.shape[0], BATCH_SIZE):
            x_train = x_train_img[i:i+BATCH_SIZE]
            print('Train -- get ind: ',i," to ",i+BATCH_SIZE)
            x_train = x_train.astype('float32') / 255.
            x_train = np.reshape(x_train, (len(x_train), 512, 512, 1)) 
            yield (x_train, x_train)
1 Answers
Related