Shuffling the training dataset with Tensorflow object detection api

Viewed 2037

I'm working on a logo detection algorithm using the Faster-RCNN model with the Tensorflow object detection api. My dataset is alphabetically ordered (so there are a hundred adidas logo, then hundred apple logo etc.). And i would like it to be shuffled while training.

I've put some values in the config file:

train_input_reader:{
          shuffle: true
          queue_capacity: some value
          min_after_dequeue : some other value}

However whatever are the values, I'm putting in, algorithm is at first training on all of the a's logos (adidas, apple and so on) and only a lapse of time after starting to see the b's logos (bmw etc.) and the c's one etc.

Of course I could just shuffle my input dataset directly, but I would like to understand the logic behind it.

PS: I've seen this post about shuffling and min_after_dequeue, but I still dont quite get it. My batch size is 1 so it shouldn't be using tf.train.shuffle_batch() but only tf.RandomShuffleQueue

My training dataset size is 5000 and if I write min_after_dequeue: 4000 or 5000 it is still not shuffled right. Why though?


Update: @AllenLavoie It's a bit hard for me; as there is a lot of dependencies and I'm new to Tensorflow. But in the end the queue is constructed by

tf.contrib.slim.parallel_reader.parallel_read(    _, string_tensor = parallel_reader.parallel_read(
        config.input_path,
        reader_class=tf.TFRecordReader,
        num_epochs=(input_reader_config.num_epochs
                    if input_reader_config.num_epochs else None),
        num_readers=input_reader_config.num_readers,
        shuffle=input_reader_config.shuffle,
        dtypes=[tf.string, tf.string],
        capacity=input_reader_config.queue_capacity,
        min_after_dequeue=input_reader_config.min_after_dequeue)

It seems that when I'm putting num_readers = 1 in the config file the dataset is finally shuffling as I want, (at least in the beginning), but when there are more somehow on the start the logos are getting in the alphabetical order.

1 Answers
Related