Tensorflow Datasets: Is there a way to only modify a certain percentage of labels?

user_15126516 0
Viewed 476

I'm using the following example to analyse the performance of Computer Vision system depending on the data quality.

Keras Implementation Retinanet: https://keras.io/examples/vision/retinanet/

My goal is to corrupt(stretch, shift) certain percentages (10%,20%,30%) of the total bounding boxes across all images. This means that images should be randomly picked and them some of the bounding boxes corrupted so that in total the target percentage is affected.

I'm using the tensorflow datasets as my training data (e.g. https://www.tensorflow.org/datasets/catalog/kitti).

My basic idea was to generate an array in the size of the total amout of boxes and fill it with 1 (modify box) and 0 (ignore box) and then iterate through all boxes:

random_array = np.concatenate((np.ones(int(error_rate_size*TOTAL_NUMBER_OF_BOXES)+1,dtype=int),np.zeros(int((1-error_rate_size)*TOTAL_NUMBER_OF_BOXES)+1,dtype=int)))

The problem is that the implementation I'm using is heavily relying on graph implementation and specifially on the map function (https://www.tensorflow.org/api_docs/python/tf/data/Dataset#map). I would like to follow this pattern in order to keep the implemented data pipeline.

What I am hopeing to do is to use map function in combination with a global counter so I can loop through the array and modify whenever a condition is given. It should roughly look something like this:

COUNT = 0
def damage_data(box):

    scaling_range = 2.0
    global COUNT
    COUNT += 1
    if random_array[COUNT]== 1:

        new_box = tf.stack(
        [
            box[0]*scaling_range*tf.random.uniform(shape=(),minval=0.0,maxval=1.0,dtype=tf.float32,seed=1), # x center
            box[1]*scaling_range*tf.random.uniform(shape=(),minval=0.0,maxval=1.0,dtype=tf.float32,seed=2), # y center
            box[2]*scaling_range*tf.random.uniform(shape=(),minval=0.0,maxval=1.0,dtype=tf.float32,seed=3), # width,
            box[3]*scaling_range*tf.random.uniform(shape=(),minval=0.0,maxval=1.0,dtype=tf.float32,seed=4), # height,
        ],
        axis=-1,)
    else:
        tf.print("Not Changed")
        new_box = tf.stack(
        [
            box[0],
            box[1], # y center
            box[2], # width,
            box[3], # height,
        ],
        axis=-1,)         

    return new_box


def damage_data_cross_sequential(image, bbox, class_id):
    
    # bbox format [x_center, y_center, width, height]

    bbox = tf.map_fn(damage_data,bbox)
    
    return image, bbox, class_id


train_dataset = train_dataset.map(damage_data_cross_sequential,num_parallel_calls=1)

But using this code the variable COUNT is not incremented globally but rather every map() call starts from the initial value 0. I assume this somehow is caused through the graph implementation and the parallel processes in map().

The question is now if there is any way to globally increase a counter through the map function or if I could extend the given dataset with a unique identifier (e.g. add box[5] = id).

I hope the problem is clear and thanks already! :)

--------------UPDATE 1-------------------------------

The second approach as described by @Lescurel is what I'm trying to do.

Some clarifications about the dataset structure.

The number of boxes per image is not identical.It changes from image to image.

e.g. sample 1: ((x_dim, y_dim, 3), (4,4)), sample 2: ((x_dim, y_dim, 3), (2,4))

For a better understanding the structure can be reproduced with the following:

import tensorflow as tf
import tensorflow_datasets as tfds
import numpy as np

valid_ds = tfds.load('kitti', split='validation') # validation is a smaller set


def select_relevant_info(sample):
    image = sample["image"]
    bbox = sample["objects"]["bbox"]
    class_id = tf.cast(sample["objects"]["type"], dtype=tf.int32)
    
    return image, bbox, class_id


valid_ds = valid_ds.map(select_relevant_info)

for sample in valid_ds.take(1):
    print(sample)
tensorflow-datasets tensorflow2.0 keras tensorflow python
1 Answers

For plenty of reasons, using a global state is not a terribly good idea, but it's probably even worse in a concurrent context like this one.

There is at least two other ways of implementing what you want:

  • using a random sample with a threshold as condition to modify the label
  • put your random array in the dataset as the condition to modify the label.

I personally prefer the first option, which is simpler.

An example.

Lets generate some random data, and create a tf.Dataset. In that example, the total number of sample is 1000:

imgs = tf.random.uniform((1000, 4, 4))
boxes = tf.ones((1000, 4))

ds = tf.data.Dataset.from_tensor_slices((imgs, boxes))

First option: Random Sample

This function will draw a number uniformly between 0 and 1. If this number is higher than the threshold prob, then nothing happens. Otherwise, we modify the label. In that example, it gives a 0.05% chance of modifying the label.

def change_label_with_prob(label, prob=0.05, scaling_range=2.):
    return tf.cond(
        tf.random.uniform(()) > prob,
        lambda: label,
        lambda: label*scaling_range*tf.random.uniform((4,), 0., 1., dtype=tf.float32),
    )

You can simply call it with Dataset.map:

new_ds = ds.map(lambda img, box: (img, change_label_with_prob(box)))

Second Option : Pass the condition array around

First, we generate an array filled with our conditions: 1 if we want to modify the array, 0 if not.

# lets set the number to change to 200
N_TO_CHANGE = 200
# randomly generated array with 200 "1" and "800" 0.
cond_array = tf.random.shuffle(
    tf.concat([tf.ones((N_TO_CHANGE,),dtype=tf.bool), tf.zeros((1000 - N_TO_CHANGE,),dtype=tf.bool)], axis=0)
)

Then we can create a dataset from that array of conditions, and zip it with our previous dataset:

# creating a dataset from the conditional array
ds_cond = tf.data.Dataset.from_tensor_slices(cond_array)

# zipping the two datasets together
ds_data_and_cond = tf.data.Dataset.zip((ds, ds_cond))
# each element of that dataset is ((img, box), cond)

We can write our function, roughly the same as before:

def change_label_with_cond(label, cond, scaling_range=2.0):
    # if true, modifies, do nothing otherwise
    return tf.cond(
        cond,
        lambda: label
        * scaling_range
        * tf.random.uniform((4,), 0.0, 1.0, dtype=tf.float32),
        lambda: label,
    )

And then map the function on our new dataset, paying attention to the nested shape of each element of the dataset:

ds_changed_label = ds_data_and_cond.map(
    lambda img_and_box, z: (img_and_box[0], change_label_with_cond(img_and_box[1], z))
)
# New dataset has a shape (img, box), same as before the zipping
user_73701530
Related