Merge Similar Bins of Image Histogram

user_8483689 0
Viewed 438

I am working on a CV project using Python (no AI, classic tools only) and ran into a problem. I am trying to detect hand and skin color from a known ROI that has the hand on white background (since it is with webcam the color might not be accurate - gray color might be considered as white for me for example). I tried doing an histogram of colors, from there I will extract the hand color. For getting the list of the colors I used Image.getcolors(width*height), and the putted it in a histogram. Unfortunately I am getting a huge list of colors which a lot of them are alike the others (for example, (255,0,0) and (255,0,1) are represented in separated bins) (due to the bad camera quality, lighting etc..). My question is how can I merge those bins and get an a reliable histogram from which I would be able to extract the skin color. Here is some of the code I wrote:

pilRoi = Image.fromarray(coloredRoi)
w,h = pilRoi.size
colorsInRoi = pilRoi.getcolors(w*h)
sortedColors = sorted(colorsInRoi, key=lambda tup: tup[0])[::-1]

for sorting the colors. And:

    for idx, color in enumerate(sortedColors):
    if(idx<config.NUM_COLORS):
        plt.bar(idx, color[0], color=helper.toHex(color[1]),edgecolor=helper.toHex(color[1]))
    else:
        break
plt.show()

for the histogram. I did try removing sum of the white range pixels, the problem remains with other colors:

    for color in sortedColors:
    if isInWhiteRange(color[1]) or color[1] == config.BLUE:
        toRemove.append(color)

for color in toRemove:
    sortedColors.remove(color)

Thank You!

computer-vision histogram opencv image-processing python
1 Answers

Usually with histograms you place colours into bins. For example, if you had 256 intensities and 32 bins, this means that each bin has a width of 4. Intensities ranging from 0-3 get lumped into the first bin, intensities ranging from 4-7 get lumped into the second bin, etc. This is what is formally known as uniform colour quantization where we quantize a pixel so that it goes into one of a predetermined set of bins.

In your specific case, you can specify a number of bins per colour channel, then you can simply compute a 1D histogram such that with each colour channel, you figure out which bin for that channel it belongs to, then convert this sequence of 3 bins into a single value. The reason why I advocate a 1D histogram is to allow for computing similarity measures between images easier.

Since you have the image already in NumPy form stored in coloredRoi, I assume you have this already in three planes so a 3D array with the channels being the last dimension. I also assume you're dealing with 8-bit unsigned integer values per channel. Something simple like this could work:

# Define number of bins per channel
num_red_bins = 8
num_green_bins = 8
num_blue_bins = 8

# Define threshold per bin
thresh_red = 256 // num_red_bins
thresh_green = 256 // num_green_bins
thresh_blue = 256 // num_blue_bins

# Extract planes
red = coloredRoi[..., 0]
green = coloredRoi[..., 1]
blue = coloredRoi[..., 2]

# Calculate bin number per location
bin_red = red // thresh_red
bin_green = green // thresh_green
bin_blue = blue // thresh_blue

# Calculate 1D bin locations
bins = num_red_bins * num_green_bins * bin_blue + num_green_bins * bin_red + bin_green

# Calculate histogram
histo = np.bincount(bins, minlength=num_red_bins * num_green_bins * num_blue_bins)

The code is pretty self-explanatory, but the last two lines may be confusing. Before this point, we have converted RGB pixels into their bin locations in the red, green and blue channels. The collection of these will give us where this pixel will map to with respect to the final 3D bin. This is a unique tuple that will map to a single location in the 1D histogram. To calculate the final 1D bin number, think of red navigating the rows of this space and green navigating the columns of this space. Assuming we only have to deal with red and green, every time we need to go to a new space in red, we have to hop num_green_bins over so that's why we have num_green_bins * bin_red. Each time we go to a new space in green, we just need to offset the columns so we can add bin_green to num_green_bins * bin_red + bin_green. Finally if we want to go to blue, we need to hop over num_red_bins * num_green_bins for each blue space we want as we're now going to 3D, hence we now also add with num_red_bins * num_green_bins * bin_blue. We then use numpy.bincount to calculate the final histogram based on the 1D bins we've just calculated.

Now that you have this 1D histogram, you can use any histogram similarity measure to see if the colour distribution of what you expect from a hand match the patch of interest. As a final note, if you're curious to see what this quantized image looks like, simply take your bin values and multiply the bin values by the threshold per bin as I've outlined above, then stack everything together into a final image.

out_img = np.dstack((thresh_red * bin_red, thresh_green * bin_green, thresh_blue * bin_blue))

numpy.dstack takes 2D arrays and stacks them in the third dimension to produce a consolidated 3D array. If you did this right, small variations of colour will disappear when you visualize the quantized result which is stored in out_img. Note that the number of bins per colour channel are parameters you need to tune. The higher the number of bins, the more fine-grained you get for the colours which thus increases the dynamic range of what you're represented, but with granular colours comes at a cost of treating very similar RGB pixels to be different. Similarly, the smaller the number of bins, the more similar colours are going to look like over a wider range of values which will make the discriminative power of your classification to be weaker. I would suggest changing the number of bins such that you place more exaggeration on the bins which reflect the human skin tone (red/green), and less emphasis on the colours that don't (blue).

user_32508290
Related