I found an interesting video please do watch it https://www.youtube.com/watch?v=yeS8TJwBAFs.
Here the objects/brands detected when only in hands. how is it done any ideas?
collecting all datasets/brand items within the hand and feeding into a CNN is not a good idea or any OpenCV functions can be applied?
First of all, I want to point out that this project seems that there are tons of images used to train such kind of an algorithm. From my past experience one of the possible solution is to use keypoint detection. Let me clarify it.
Using a keypoint detection, specifically a wrist keypoint is vital to achieve such kind of an algorithm. Since the model first detects the wrist keypoints of a human, finding the wrist keypoints and defining an area around the wrist keypoints e.g 50x50 pixels is used for classification. (please see: keypoint detection algorithms.)
However, the challange is defining a small area like 50x50 pixels will not have enough features to distinguish between e.g nutella and peanut butter. Therefore, I think the model is very well designed and also the dataset size must be very large to do such kind of a classification.
It seems we can do this with simple object detection models like YOLOV3, SSD with hand(wrist) as an additional class.
So, if we want to detect N, class of brands, then total classes for object detection model would be N+1.
After getting the detection from model we can filter the objects whose bounding box overlaps with the person hand bounding box (For example IOU( intersection over union) of (objects, hand)
