TensorFlow - TFRecords load and transform images with bounding boxes

Question

I'm trying to build a 'Car Classifier' using TensorFlow.

I have 1000 labelled JPG images, 800x800, complete with bounding boxes and associated annotations.coco.json; split into train/validate/test folders.

I've managed to load the TFRecordDataset's using the code below:

TFRecord Data Set Loading Steps

# Load TfRecord data sets
raw_train = tf.data.TFRecordDataset([training_file])
raw_validation = tf.data.TFRecordDataset([validation_file])
raw_test = tf.data.TFRecordDataset([testing_file])

# Load label map
category_index = label_map_util.create_category_index_from_labelmap(label_map_file, use_display_name=True)

------------------------------------------------------------------------------------------
def extract_features(tfrecord):
    # Extract features using the keys set during creation
    features = {
        'image/object/bbox/xmin': tf.io.VarLenFeature(dtype=tf.float32),
        'image/object/bbox/ymin': tf.io.VarLenFeature(dtype=tf.float32),
        'image/object/bbox/xmax': tf.io.VarLenFeature(dtype=tf.float32),
        'image/object/bbox/ymax': tf.io.VarLenFeature(dtype=tf.float32),        
        'image/object/class/label': tf.io.VarLenFeature(dtype=tf.int64),        
        'image/width': tf.io.FixedLenFeature([], tf.int64),
        'image/height': tf.io.FixedLenFeature([], tf.int64),
        'image/encoded': tf.io.FixedLenFeature([], tf.string)
    }

    # Extract the data record
    sample = tf.io.parse_single_example(tfrecord, features)

    image = tf.io.decode_image(sample['image/encoded'])        
    label = sample['image/object/class/label']
        
    return [image, label]

raw_train = raw_train.map(extract_features)
raw_validation = raw_validation.map(extract_features)
raw_test = raw_test.map(extract_features)

Transform/Resize images for Training

ORIGINAL_IMG_SIZE = 800
RESIZE_IMG_SIZE = 160 # All images will be resized to 160x160 or 614x614 maybe for Yolo?

def format_example(image, label):
    #https://stackoverflow.com/questions/62957726/i-got-value-error-that-image-has-no-shape-while-converting-image-to-tensor-for-p
    image.set_shape([ORIGINAL_IMG_SIZE, ORIGINAL_IMG_SIZE, 3])
    image = tf.cast(image, tf.float32)
    image = (image/127.5) - 1
    image = tf.image.resize(image, (RESIZE_IMG_SIZE, RESIZE_IMG_SIZE))
    return image, label

Tensorflow examples only seem to talk about resizing the whole image and not about how to handle resizing of bounding boxes within the image, and bounding box labels.
Does anyone have any examples of how to handle the resizing of images together with bounding boxes contained within the image?

Training Pipeline

Again Tensorflow examples only seem to train with whole images, not with images with bounding boxes and associated bounding box labels.
Does anyone have any examples of TensorFlow Transfer Learning training with images with bounding boxes and associated bounding box labels?

TensorFlow - TFRecords load and transform images with bounding boxes

0 Answers