Cut mask out of image with certain pixel margin numpy opencv

user_4509378 0
Viewed 3991

I am playing around with the Mask RCNN (https://github.com/matterport/Mask_RCNN) segmentation program that is trained on the COCO data set. It detects persons (along with many other objects that I further neglect) in an image and returns one or multiple Person masks, i.e. Boolean Numpy arrays containing True values for all pixels that are classified as a 'Person' and False values for all other pixels:

Overlay Image

So an inputted image (uint8 array of shape (3900,2922,3)) becomes a mask (Boolean array of shape (3900,2922)) or multiple masks when multiple persons are detected in the picture.

Now I can use this mask to cut the person out of the image with some simply Numpy array indexing:

mask3d = np.dstack([mask]*3)
cut_out_mask = np.invert(mask3d)
res = np.where(cut_out_mask, 0, image)

This returns the following image: Cut_out Image

Since the masks that are returned by the Mask_RCNN program are quite tight, I would like to add a margin of a few pixels (let's say 15px), so that I get something like this:

manual

Which Numpy/ OpenCV functions can I leverage to cut out the mask from the original image (similar to np.where), adding a margin of 15 pixels around the mask?

opencv image-processing numpy image python
3 Answers

One method to do this is to use cv2.dilate to increase the surface area of the mask. Depending on your mask shape, you can create varying structuring element shapes and sizes with cv2.getStructuringElement.

For instance, if your mask shape is rectangular you might want to use cv2.MORPH_RECT or if your mask shape is circular you can use cv2.MORPH_ELLIPSE. In addition, you can change the kernel size and the number of iterations to dilate. After dilating the mask, you can use cv2.bitwise_and to get your result. Here's a minimum reproducible example:

Original image

Mask

Dilate

Bitwise-and for result

import cv2

# Load image and mask
image = cv2.imread('1.png')
mask = cv2.imread('mask.png')

# Create structuring element, dilate and bitwise-and
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (25,25))
dilate = cv2.dilate(mask, kernel, iterations=3)
result = cv2.bitwise_and(image, dilate)

cv2.imshow('dilate', dilate)
cv2.imshow('result', result)
cv2.waitKey()
user_111621650

You may use cv2.dilate morphological operation for widening the mask:

new_mask = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (50,50)))

Sample code that loads a mask and widens the mask:

import cv2

# Read the mask from a file (for testing).
mask = cv2.imread("sample_mask.png", cv2.IMREAD_GRAYSCALE);

new_mask = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (50,50)))

# Display mask and new_mask
cv2.imshow('mask', mask)
cv2.imshow('new_mask', new_mask)
cv2.waitKey(0)
cv2.destroyAllWindows()

mask:
enter image description here

new_mask:
enter image description here

user_49267570

The function you are looking for is the cv2.filter2D().

I wrote a short demo for you:

import numpy as np
import cv2

First we create a demo mask, with contains a square in the middle:

mask = np.zeros((300, 300))
mask[100:200, 100:200] = 1

Then we create our kernel for the filter2D function:

kernel = np.ones((32, 32))

I used the size 32, to get the desired padding of 15 pixel.

paddedMask = cv2.filter2D(mask, 1, kernel)

Just for demoing I display the Image with the following code:

image = cv2.threshold(np.array(paddedMask, dtype=np.uint8), 1, 255, 1)[1]
cv2.imshow("Test", image)
cv2.imshow("Org", mask)
cv2.waitKey(0)
cv2.destroyAllWindows()

Hope it helps.

user_129527280
Related