ML for object search

user_2265758 0
Viewed 86

I'm trying to find a way to build ML using AWS, preferably using their services such as SageMaker and not just EC2, for object detection in images using an image as input.

AWS Rekognition offers Image Comparison and Object detection APIs, but they are not exactly what I'm looking for, the comparison works only with faces and not objects and object detection is too basic.

AlibabCloud has that functionality as a service (https://www.alibabacloud.com/product/imagesearch) but I would like to use something similar on AWS, rather than Alibaba.

How would I go about and build something like this?

Thank you.

amazon-sagemaker computer-vision machine-learning amazon-web-services
1 Answers

edited 03/08/2020 to add pointers for visual search

Since you seem interested both in the tasks of object detection (input an image, and return bounding boxes with object classes) and visual search (input an image and return relevant images) let me give you pointers for both :)

For object detection you have 3 options:

  • Using the managed service Amazon Rekognition Custom Labels. The key benefits of this service is that (1) it doesn't require writing ML code, as the service runs autoML internally to find the best model, (2) it is very flexible in terms of interaction (SDKs, console), data loading and annotation and (3) it can work even with small datasets (typically a few hundred images or less).
  • Using SageMaker Object Detection model (documentation, demo). In this option, the model is also already written (SSD architecture with Resnet or VGG backbone) and you just need to choose or tune hyperparameters
  • Using your own model on Amazon SageMaker. This could be your own code in docker, or code from an ML framework in a SageMaker ML Framework container. There are such containers for Pytorch, Tensorflow, MXNet, Chainer and Sklearn. In terms of model code, I recommend considering gluoncv, a compact python computer vision toolkit (based on mxnet backend) that comes with many state-of-the-art models and tutorials for object detection

The task of visual search requires more customization, since you need to provide the info of (1) what you define as search relevancy (eg is it visual similarity? or object complementarity? etc) and (2) the collection among which to search. If all you need is visual similarity, a popular option is to transform images into vectors with a pre-trained neural network and run kNN search between the query image and the collection of transformed images. There are 2 tutos showing how to build such systems on AWS here:

user_53318340
Related