ml.net image classification categories problem

user_17903077 0
Viewed 65

So i made a Machine Learning project with images classification auto trained and it actually work good!

The only problem im facing is:

let say i have 2 categories, Cars and Boats. The machine is 100% accurate when it come to cars or boats but if i put a random picture like a dog or an house, it still validate it as a car or a boat...? So i guess its just run an algorythm and chose the higher score of similarity.

So i checked on youtube, stack and many others forums and website for an answer but i found nothing about. So my last hope is to ask the question in here.

Can someone can tell me what should i do to have the machine have a mismatch when the image analyzed is not one of the categories? I tried to implement a code that would be based on score to reject scores under X number but the thing is that some image that are out of the categories get an higher score than an image in the categories... at this point im pretty lost.

thank you

ml.net c#
1 Answers

The reason for the misclassification is because the model only knows about cars and boats. It doesn't know what a dog is.

Imagine you didn't know anything about the world and were taught about the colors Red and Yellow. And that's all you knew. Then all of a sudden, someone showed you something Green. You don't know Green but you do know Red and Yellow. So you'll probably say Yellow because that's what it looks most like based on your limited knowledge. Machine learning models behave in similar ways. They can only choose from the set of categories you've trained them on.

Therefore, in order to take into account more categories, you'd need to retrain your model with examples of these other categories so the model knows about them.

As far as detecting whether something was mislabeled is a bit more tricky. A way to do it is to have a human-in-the-loop. Where every so often, you have someone verify whether the model is behaving the way it's supposed to. Another which is more automated but more complex is to calculate the data drift.

https://towardsdatascience.com/detecting-semantic-drift-within-image-data-6a59a0e768c6

user_151841840
Related