How to segment gray areas of image with OpenCV

I would suggest the following step for your task:

1. Blurr the image to get rid of small stuctures such as the line

2. Binarization via thresholding. But using the global otsu method instead.

3. Apply opening morphological operation to the binarized mask. This connects the binary masks of the text blocks

4. Contour extraction the extract the blobs and bounding rect estimation to get the rectangle borders of the blobs.

   img_rgb = cv2.imread(...)
   
   # color 2 gray conversion
   img = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
   
   # image bluring
   img = cv2.GaussianBlur(img, (55,55), -1)
   
   # binarization make a mask image out of the gray value image
   _,img = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
   
   # morphological operation closing. Scattered areas will be combined
   kernel = np.ones((15,15),np.uint8)
   img = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel, iterations=2)
   
   # contour extraction for each combinded region in the mask the outer contour will be extracted
   contours,_ = cv2.findContours(img, mode=cv2.RETR_EXTERNAL,method=cv2.CHAIN_APPROX_SIMPLE)
   # list containing the image content of each found region
   img_roi = []
   img_rgb_raw = img_rgb.copy()
   for contour in contours:
      # computed bounding rect of contour, contour contains a list of points
      roi = cv2.boundingRect(contour)
      cv2.rectangle(img=img_rgb, pt1=(roi[0], roi[1]), pt2=(roi[0] + roi[2], roi[1] + roi[3]), color=(0,0,255), thickness=2)
      # extract image content of a region for further processing
      img_roi.append(img_rgb_raw[roi[1] : roi[1] + roi[3], roi[0] : roi[0] + roi[2], :])
   
   # display results
   cv2.imshow("binary", img)
   cv2.imshow("output",img_rgb)
   cv2.waitKey(-1)
   

Binarization and closing provides a good segmentation result for the example.

You are trying to crop each info box, right? You can apply a very aggressive morphology chain with big structuring elements to create big blocks of text. Then, extract contours and approximate them to bounding rectangles. Afterwards, just crop each bounding rectangle from the original image. This is the code:

import cv2
import numpy as np

# image path
path = "D://opencvImages//"
fileName = "sBREr.jpg"

# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Deep copy for results:
inputCopy = inputImage.copy()

# Convert the image to grayscale:
grayImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)

# Threshold via Otsu:
_, binaryImage = cv2.threshold(grayImage, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)

This bit will get you a nice binary image, note that the target pixels are in white:

Now, apply a couple of dilate operations. First, with a big vertical kernel, then with a horizontal kernel same size as the first. These are big kernels, of size 1 x 15 in the first case and 15 x 1 in the last case. Also set a couple of iterations. For this input, I tried 4 iterations. This is done as follows:

# Set morph operation iterations:
opIterations = 4

# Get the structuring element:
verticalKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1, 15))
# Perform Dilate (vertical):
dilateImage = cv2.morphologyEx(binaryImage, cv2.MORPH_DILATE, verticalKernel, None, None, opIterations, cv2.BORDER_REFLECT101)

horizontalKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15, 1))
# Perform Dilate (horizontal):
dilateImage = cv2.morphologyEx(dilateImage, cv2.MORPH_DILATE, horizontalKernel, None, None, opIterations, cv2.BORDER_REFLECT101)

Now, if you process this blobs your bounding rectangles might be too big. That is because the dilate operations effectively increased the blob size. Let's try to bring the blobs to their original size by applying an erode, same number of iterations but this time with a more traditional kernel - I use a 3 x 3 rectangular one:

# Perform Erode:

# Set kernel (structuring element) size:
kernelSize = 3
# Set morph operation iterations:
opIterations = 8

# Get the structuring element:
maxKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))

# Perform closing:
erodeImage = cv2.morphologyEx(dilateImage, cv2.MORPH_ERODE, maxKernel, None, None, opIterations, cv2.BORDER_REFLECT101)

This is the final result of the morphology chain:

We now have nice blocks where the original info used to me. Furthermore, the blocks are thick and connected. Let's detect external contours now - be careful here and use external mode, because some blocks could have nested holes. We are looking only for parent contours with no children. Additionally, I included a minimum area filter to filter out possible noise:

# Get each bounding box
# Find the big contours/blobs on the filtered image:
contours, hierarchy = cv2.findContours(erodeImage, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)


# Look for the outer bounding boxes (no children):
for _, c in enumerate(contours):

    # Approximate the contour to a polygon:
    contoursPoly = cv2.approxPolyDP(c, 3, True)
    # Get the polygon's bounding rectangle:
    boundRect = cv2.boundingRect(contoursPoly)

    # Get the dimensions of the bounding rect:
    rectX = boundRect[0]
    rectY = boundRect[1]
    rectWidth = boundRect[2]
    rectHeight = boundRect[3]

    # Compute the rect's area:
    rectArea = rectWidth * rectHeight
    # Set a min area threshold:
    minArea = 100
    # Process rectangle only if its above the min threshold:
    if rectArea > minArea:
        # Draw rectangle:
        color = (0, 255, 0)
        cv2.rectangle(inputCopy, (int(rectX), int(rectY)), (int(rectX + rectWidth), int(rectY + rectHeight)), color, 3)

        cv2.imshow("Bounding Rectangle", inputCopy)
        cv2.waitKey(0)

        # Crop the current ROI:
        croppedImage = inputImage[rectY:rectY + rectHeight, rectX:rectX + rectWidth]

        cv2.imshow("croppedImage", croppedImage)
        cv2.waitKey(0)

These are the bounding rectangles of the original image:

These are the cropped areas. Note that these are individual images:

After detecting the edges of the image using the canny edge detector, you can use the cv2.dilate method to dilate the contours, clumping scattered contours that are within a certain distance from each others into chuncks:

import cv2
import numpy as np

img = cv2.imread("blur_doc.jpg")
img_canny = cv2.Canny(img, 5, 20)
img_dilate = cv2.dilate(img_canny, np.ones((5, 5)), iterations=9)

contours, _ = cv2.findContours(img_dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
for cnt in contours:
    x, y, w, h = cv2.boundingRect(cnt)
    cv2.rectangle(img, (x, y), (x + w, y + h), (72, 67, 146), 2)

cv2.imshow("Image", img)
cv2.waitKey(0)

Output: