Custom Object Detection TFLite model error (pascal voc) - ValueError: The size of the train_data (0) couldn't be smaller than batch_size (2)

Question

I am trying to build a custom object detection model in jupyter notebook using tflite model maker but I have some problems.

I am getting images with pascal_voc (not csv) so I splited the train data/test data into different files


os.mkdir('C:/Users/user/Desktop/GradProject5/train_test_split/train')
os.mkdir('C:/Users/user/Desktop/GradProject5/train_test_split/test')

image_paths = os.listdir('C:/Users/user/anaconda3/envs/DarkflowTest/data/dataset')
random.shuffle(image_paths)

for i, image_path in enumerate(image_paths):
    if i < int(len(image_paths) * 0.8):
        shutil.copy(f'C:/Users/user/anaconda3/envs/DarkflowTest/data/dataset/{image_path}', 'C:/Users/user/Desktop/GradProject5/train_test_split/train')
        shutil.copy(f'C:/Users/user/anaconda3/envs/DarkflowTest/data/annotations/{image_path.replace("jpg", "xml")}', 'C:/Users/user/Desktop/GradProject5/train_test_split/train')
    else:
        shutil.copy(f'C:/Users/user/anaconda3/envs/DarkflowTest/data/dataset/{image_path}', 'C:/Users/user/Desktop/GradProject5/train_test_split/test')
        shutil.copy(f'C:/Users/user/anaconda3/envs/DarkflowTest/data/annotations/{image_path.replace("jpg", "xml")}', 'C:/Users/user/Desktop/GradProject5/train_test_split/test')

test_image_dir='C:/Users/user/Desktop/GradProject/	rain_test_split/test/'
#annotations_dir = 'C:/Users/user/anaconda3/envs/DarkflowTest/data/annotations/'
train_data=object_detector.DataLoader.from_pascal_voc(train_image_dir+'image/',train_image_dir+'xml/',label_map={1:"pill",2:"text"})
test_datal=object_detector.DataLoader.from_pascal_voc(test_image_dir+'image/',test_image_dir+'xml/',label_map={1:"pill",2:"text"})

Then loaded train data and test data with DataLoader

model = object_detector.create(train_data, model_spec=spec, batch_size=2, train_whole_model=True)

Then I tried to create a model but I am getting this error.

[[1]: https://i.stack.imgur.com/ZBQtk.png][1]

ValueError Traceback (most recent call last) in ----> 1 model = object_detector.create(train_data, model_spec=spec, batch_size=2, train_whole_model=True)

~\anaconda3\lib\site-packages ensorflow_examples\lite\model_maker\core ask\object_detector.py in create(cls, train_data, model_spec, validation_data, epochs, batch_size, train_whole_model, do_train) 285 if do_train: 286 tf.compat.v1.logging.info('Retraining the models...') --> 287 object_detector.train(train_data, validation_data, epochs, batch_size) 288 else: 289 object_detector.create_model()

~\anaconda3\lib\site-packages ensorflow_examples\lite\model_maker\core ask\object_detector.py in train(self, train_data, validation_data, epochs, batch_size) 137 # TODO(b/171449557): Upstream this to the parent class. 138 if len(train_data) < batch_size: --> 139 raise ValueError('The size of the train_data (%d) couldn't be smaller ' 140 'than batch_size (%d). To solve this problem, set ' 141 'the batch_size smaller or increase the size of the '

ValueError: The size of the train_data (0) couldn't be smaller than batch_size (2). To solve this problem, set the batch_size smaller or increase the size of the train_data.

Am I getting error because train_data=object_detector.DataLoader.from_pascal_voc(train_image_dir+'image/',train_image_dir+'xml/',label_map={1:"pill",2:"text"}) it failed to load train data..? I think I did everything right and still struggling with this error.. Please help me if you know the solution to this problem!

user_18519781 · Answer

I met the same issue, and simply remove the trailing / from the test image dir, then the issue is gone.

train_image_dir = '/Users/twinssbc/VSProject/mobile_phone/train'
train_process_image_dir = '/Users/twinssbc/VSProject/mobile_phone/train_preprocess'
train_label_dir = '/Users/twinssbc/VSProject/mobile_phone/train_label'

train_data_loader = object_detector.DataLoader.from_pascal_voc(train_process_image_dir, train_label_dir, label_map=label_map)
spec = model_spec.get('efficientdet_lite4')
model = object_detector.create(train_data_loader, spec, batch_size=5, epochs=50, train_whole_model=True)

test_data_loader = object_detector.DataLoader.from_pascal_voc(test_process_image_dir, test_label_dir, label_map=label_map)
model.evaluate(test_data_loader)

user_7931989 · Answer

check the format of your pascal voc xml..... I solved this issue by reformating the xml which had you might need to remove this line.. and if you still get another file not found error... check that the extension of your xml filename is same as what your file name is in the folder in use. ps this happened to me because I used label studio to annotate and tried to use the data in tflie model builder directly

Custom Object Detection TFLite model error (pascal voc) - ValueError: The size of the train_data (0) couldn't be smaller than batch_size (2)

2 Answers