Over-fitting on using retinanet by fizyr for car damage detection(regression and classification)

user_10020966 0
Viewed 184

I am trying to detect the damages type(scratch,dent,crack) and their bounding box on car images. I trained this on retinanet by fizyr but model is heavly overfitting. On training i am getting mAP of 0.89 while on test i just get mAP of 0.07. I have used 1200 images for training and 350 test, The detail are mentioned below.

-Training result
-22 instances of class rusted with average precision: 0.7371
-93 instances of class missing with average precision: 0.9160
-517 instances of class major dent with average precision: 0.9461
-1201 instances of class scratch with average precision: 0.9356
-84 instances of class tear with average precision: 0.8001
-470 instances of class minor dent with average precision: 0.9523
-435 instances of class cracked with average precision: 0.9276
-mAP using the weighted average of precisions among classes: 0.9329
-mAP: 0.8879

Test
-3 instances of class rusted with average precision: 0.0000
-13 instances of class missing with average precision: 0.0874
-160 instances of class major dent with average precision: 0.1057
-318 instances of class scratch with average precision: 0.1434
-27 instances of class tear with average precision: 0.0462
-111 instances of class minor dent with average precision: 0.0205
-127 instances of class cracked with average precision: 0.1159
--mAP using the weighted average of precisions among classes: 0.1079
mAP: 0.0742

I know classes are unbalanced but such huge difference between test and train. Also model is performing poor on scratch class which has huge number of training examples.

data-science computer-vision training-data machine-learning artificial-intelligence
1 Answers

Yes, your classes are unbalanced, but you have enough images that you should be doing better than this. How did you select your train/test partition? The results suggest that there's a randomization problem there. However they're chosen, your model is learning the wrong criteria for classification.

This reminds me of the dog/cat model that learned to discriminate the backgrounds in which people photograph their pets.

First, try rotating or shuffling your training and testing sets. Can you use cross-validation in this application? That alone could solve your problem.

Failing that, can you alter the model -- perhaps include a dropout layer? That would help un-learn invalid hypotheses.

user_47851850
Related