Trying the pretrained Faceboook DETR model for object detection using the HuggingFace implementation. The sample code listed below from https://huggingface.co/facebook/detr-resnet-50 is straightforward.
from transformers import DetrFeatureExtractor, DetrForObjectDetection
from PIL import Image
import requests
import numpy as np
url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)
feature_extractor = DetrFeatureExtractor.from_pretrained('facebook/detr-resnet-50')
model = DetrForObjectDetection.from_pretrained('facebook/detr-resnet-50')
inputs = feature_extractor(images=image, return_tensors="pt")
outputs = model(**inputs)
# model predicts bounding boxes and corresponding COCO classes
logits = outputs.logits
bboxes = outputs.pred_boxes
I can use
threshod = 0.7
labels =['background', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',
'train', 'truck', 'boat', 'traffic light', 'fire hydrant',
'street sign', 'stop sign', 'parking meter', 'bench', 'bird',
'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra',
'giraffe', 'hat', 'backpack', 'umbrella', 'shoe', 'eye glasses',
'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard',
'sports ball', 'kite', 'baseball bat', 'baseball glove',
'skateboard', 'surfboard', 'tennis racket', 'bottle', 'plate',
'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana',
'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog',
'pizza', 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed',
'mirror', 'dining table', 'window', 'desk', 'toilet', 'door', 'tv',
'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave',
'oven', 'toaster', 'sink', 'refrigerator', 'blender', 'book',
'clock', 'vase', 'scissors', 'teddy bear', 'hair drier',
'toothbrush']
np_softmax = (logits.softmax(-1)[0, :, :-1]).detach().numpy()
classes = []
probability = []
idx = []
for i, j in enumerate(np_softmax):
if np.max(j) > threshold:
classes.append(labels[np.argmax(j)])
probability.append(np.max(j))
idx.append(i)
to retrieve the detected classes. But I did not fully understand the coordinates in the bboxes.
This is a torch tensor with 100 bounded boxes coordinates of 4 dimensions. With idx I can get the index of the classes so I can get their corresponding boxes. Seems the coordinates are normalized because they are all between 0 and 1. I have a difficulty to remap the coordinates into pixels so I can draw the bounded boxes on the original images. Could not find documentation on this, any suggestions? Thanks