How to generate bounding box data from heatmap data of an image?

user_3539683 0
Viewed 2170

I have a group of images and some separate heatmap data which (imperfectly) explains where subject of the image is. The heatmap data is in a numpy array with shape (224,224,3). I would like to generate bounding box data from this heatmap data.

The heatmaps are not always perfect, So I guess I'm wondering if anyone can think of an intelligent way to do this.

Here are some examples of what happens when I apply the heatmap data to the image:

Image of a cat with a heatmap illuminating the subject of the image

enter image description here

I found a solution to this in matlab, but I have no idea how to read this code! I am a python programmer, unfortunately. https://github.com/metalbubble/CAM/tree/master/bboxgenerator

Anyone have any ideas about how to approach something like this?

computer-vision machine-learning opencv matlab python
3 Answers

I'm attempting a similar method for automating the creation of bounding boxes (since, lets face it: creating boxes manually takes along time)

this other stackpost covers a similar idea: EDIT: (i originally had put a link to the current stack post - but here is the stack post i was referring to) Generating bounding boxes from heatmap data

the problem at hand that i recognize is that heatmaps can be fragmented and a bit arbitrary. the solution that comes to mind initially is setting a threshold of the heat map. So in the case of the example heat map images - when applying a bounding box cover all regions that are yellow/orange/red than say green/blue.

user_128974550

It depends on how many bounding boxes you need. You can set a threshold and have multiple bounding boxes for each of the highly activated regions, or try connecting the regions (by a morphological operation maybe) and calculate a single bounding box for connected activated pixels.

user_176040950
Related