Problem with superpixel segmentation (opencv in python) - holes in contours of the segmentation mask

Question

I am trying superpixel segmentation with OpenCV in Python. I have tried implementing LSC, SLIC and SEEDS algorithms https://docs.opencv.org/3.4/df/d6c/group__ximgproc__superpixel.html in order to segment cell nuclei, however, so far only SEEDS algorithm works. Countours in segmentation masks created via .getLabelContourMask() method contained holes for both LSC and SLIC algorithms, which led to flooding of the whole (LSC) or large portion (SLIC) of the mask when using floodfill() method https://docs.opencv.org/3.4/d7/d1b/group__imgproc__misc.html. The trend is - the better the segmentation (eg the lower the ratio value in LSC), the leakier the contours in the mask (which I am assuming should not happen). Only SEEDS worked as expected - no apparent holes/openings in contours of the mask, thus only areas around seedpoints flooded after using floodfill(). Anyone had the same experience? Any idea why this might be the case (may OpenCV implementation of LSC and SLIC be problematic?)? Can anything be done to close the contours/edges in the LSC/SLIC mask? Newbie in comp sci over here, any help appreciated.

Test Code for LSC:

lsc = cv2.ximgproc.createSuperpixelLSC(image, region_size=25, ratio=0.1)
lsc.iterate(100)
lsc_mask = lsc.getLabelContourMask() 
cv2.imwrite("lsc_mask.tif", lsc_mask) #holes visible when zooming in

x, y = lsc_mask.shape
m = np.zeros((x+2, y+2),dtype=np.uint8)

for point in nuclei_coordinates:
    retval, lsc_mask_flooded, m_flooded, rect = cv2.floodFill(image=lsc_mask, mask=m, seedPoint=point, newVal=(255,255,255), loDiff=0, upDiff=0) 

cv2.imwrite("lsc_mask_flooded.tif", lsc_mask_flooded) #almost whole image flooded
cv2.imwrite("m_flooded.tif", m_flooded*255)

original image (.tif)

lsc_mask for the image (.tif)

lsc_mask zoomed in

image - lsc_mask overlay (.tif)

lsc_mask_flooded (.tif)

m_flooded (.tif)

nuclei_coordinates = [(56, 106), (32, 116), (13, 125), (18, 145), (32, 147), (13, 160), (10, 182), (46, 192), (39, 208), (33, 227), (14, 231), (27, 255), (31, 272), (38, 284), (45, 301), (82, 312), (70, 252), (56, 261), (51, 244), (63, 219), (103, 334), (121, 334), (131, 316), (148, 322), (170, 295), (219, 261), (224, 227), (220, 180), (192, 178), (196, 162), (211, 157), (215, 138), (207, 116), (190, 109), (192, 97), (186, 81), (170, 73), (174, 106), (158, 115), (147, 130), (152, 168), (156, 195), (173, 189), (131, 205), (148, 59), (125, 84), (134, 105), (150, 101), (119, 102), (124, 124), (141, 147), (122, 167), (103, 168), (112, 188), (78, 202), (104, 206), (102, 229), (116, 228), (134, 221), (146, 221), (81, 177), (110, 153), (116, 148), (111, 140), (100, 144), (91, 130), (88, 146), (74, 142), (79, 155), (54, 137), (71, 125), (82, 100), (72, 96), (76, 77), (95, 70), (104, 60), (198, 292), (207, 276), (162, 309), (148, 292), (148, 305), (131, 283), (100, 298), (232, 15), (212, 24), (225, 42), (215, 68), (247, 83), (238, 57), (264, 54), (282, 44), (317, 27), (318, 11), (309, 68), (319, 56), (306, 97), (297, 91), (233, 117), (309, 127), (314, 139), (319, 165), (326, 176), (339, 197), (350, 214), (351, 240), (364, 253), (366, 279), (378, 300), (355, 308), (332, 298), (310, 296), (314, 271), (336, 273), (283, 279), (271, 285), (258, 270), (264, 256), (250, 260), (259, 246), (245, 228), (247, 211), (255, 163), (280, 162), (293, 148), (370, 452), (368, 421), (358, 399), (358, 380), (339, 358), (337, 397), (342, 373), (328, 338), (336, 466), (325, 452), (338, 431), (324, 433), (306, 421), (317, 412), (324, 387), (323, 370), (313, 356), (298, 388), (289, 428), (265, 454), (244, 450), (241, 432), (269, 426), (275, 397), (303, 337), (283, 344), (278, 323), (264, 332), (270, 345), (242, 352), (218, 369), (225, 345), (219, 332), (208, 316), (190, 327), (183, 342), (171, 347), (136, 369), (134, 386), (152, 400), (130, 411), (154, 454), (147, 431), (183, 443), (208, 440), (194, 422), (183, 403), (232, 414), (215, 466), (296, 468), (452, 8), (464, 24), (480, 43), (431, 41), (445, 34), (427, 23), (406, 12), (380, 17), (373, 34), (351, 33), (389, 42), (409, 38), (421, 54), (414, 78), (395, 79), (403, 59), (378, 83), (363, 85), (345, 91), (342, 66), (349, 53), (375, 55), (489, 58), (492, 37), (511, 49), (536, 48), (540, 66), (544, 84), (547, 101), (548, 120), (529, 96), (520, 124), (540, 134), (554, 147), (500, 115), (482, 111), (471, 85), (459, 67), (448, 86), (449, 117), (442, 140), (427, 133), (418, 153), (405, 117), (358, 125), (355, 110), (341, 121), (353, 146), (371, 154), (387, 151), (394, 135), (384, 190), (384, 166), (368, 175), (368, 203), (383, 217), (394, 203), (389, 232), (404, 238), (409, 262), (419, 250), (419, 232), (450, 279), (456, 265), (476, 257), (433, 262), (435, 226), (451, 242), (424, 181), (320, 221), (443, 194), (455, 179), (474, 198), (488, 186), (504, 197), (475, 176), (555, 170), (551, 190), (532, 187), (527, 198), (504, 209), (481, 227), (468, 224), (456, 228), (556, 213), (539, 216), (536, 235), (520, 235), (492, 313), (509, 321), (488, 296), (491, 340), (508, 359), (501, 387), (490, 374), (480, 398), (475, 379), (470, 353), (452, 367), (449, 395), (436, 412), (431, 430), (461, 417), (447, 455), (468, 454), (488, 433), (428, 452), (419, 464), (505, 424), (493, 415), (519, 414), (539, 390), (547, 367), (522, 367), (543, 333), (518, 348), (556, 315), (569, 294), (643, 314), (628, 321), (623, 304), (611, 305), (595, 307), (669, 347), (673, 289), (661, 272), (632, 247), (630, 231), (621, 211), (608, 224), (611, 201), (605, 175), (635, 188), (644, 202), (662, 248), (688, 254), (713, 270), (729, 286), (720, 309), (739, 340), (733, 351), (724, 341), (736, 251), (729, 228), (724, 239), (714, 249), (711, 229), (699, 241), (680, 236), (675, 220), (695, 206), (727, 212), (734, 198), (737, 178), (720, 181), (709, 162), (738, 158), (739, 140), (721, 147), (702, 141), (681, 149), (662, 168), (663, 189), (675, 200), (658, 151), (652, 131), (667, 122), (648, 152), (617, 148), (619, 125), (583, 125), (581, 148), (574, 106), (573, 87), (576, 74), (711, 10), (725, 12), (736, 47), (724, 63), (733, 70), (713, 82), (710, 100), (696, 72), (691, 47), (691, 23), (677, 9), (694, 4), (649, 18), (654, 3), (635, 8), (622, 38), (612, 26), (598, 47), (619, 77), (630, 105), (653, 102), (640, 90), (676, 91), (661, 66), (670, 35), (703, 367), (712, 387), (684, 358), (656, 362), (626, 363), (611, 358), (600, 452), (633, 454), (574, 440), (589, 421), (580, 463), (335, 236), (721, 459), (733, 445), (717, 444), (728, 421), (740, 430), (718, 30), (584, 48), (589, 198)]

user_8639595 · Answer

I think that the idea behind using LSC or LIS is to use it with watershed algorithm. That way you can get an image that has visible and continous segments contours.

Java code example:

SuperpixelLSC superpixelLSC = Ximgproc.createSuperpixelLSC(src, region_size);
superPixelLSC.iterate();
superPixelLSC.getLabels(markers);
Imgproc.watershed(src, markers);

You can then detect the contours on watershed algorithm results.

user_4118300 · Answer

Had the same problem, when working with the SLIC-Segmentation. Used a dilate function after getting the mask, to fill the gaps on the lines:

dst = cv2.dilate(lsc_mask , kernel)

The dilation is a basic morphologic function to expand the boudries of a figure. The kernel is the matrix you expand the boudries with. You can control the expansion of the boundries with the size of the kernel. If you need more info I can suggest R.Gonzales' book-Digital Image Processing.

Here is also a quite good example with code: https://cvexplained.wordpress.com/2020/05/18/dilation/

Problem with superpixel segmentation (opencv in python) - holes in contours of the segmentation mask

2 Answers