How to crop multiple rectangles or squares from JPEG?

Question

I have a jpeg from where I want to crop a portion containing graph (the one in the bottom portion).

As of now I used this code to achieve the same:

from PIL import Image

img = Image.open(r'D:\aakash\graph2.jpg')
area = (20, 320, 1040, 590)
img2 = img.crop(area)
# img.show()
img2.show()

But I achieved this by guessing the x1, y1, x2, y2 multiple times to arrive at this (guess work).

Image before cropping:

Image after cropping:

I'm totally novice in image cropping based on some logic. How can I successfully crop all graphs to create separate images given the positions are same?

Update: I believe, it is not a possible duplicate of that problem because, even though logically that's same, but the way the clustering logic will work is different. In that question, there are only 2 vertical white lines to divide, but here, there's two horizontal and two vertical lines, and I hardly have a clue on how to use KMeans to solve this kind of image clustering.

Help from someone who's an expert with sklearn's KMeans to solve this kind of problem shall be highly appreciated.

user_2836621 · Answer

Here's another way to do it, but using PIL/Pillow and skimage rather than OpenCV:

#!/usr/local/bin/python3

import numpy as np
from PIL import Image, ImageFilter
from skimage.measure import label, regionprops

# Load image and make Numpy version and greyscale PIL version
pim = Image.open('article.jpg')
n   = np.array(pim)
pgr = pim.convert('L')

# Threshold to make black and white
thr = pgr.point(lambda p: p < 230 and 255)
# Following line is just for debug
thr.save('result-1.png')

# Median filter to remove noise
fil = thr.filter(ImageFilter.MedianFilter(11))
# Following line is just for debug
fil.save('result-2.png')

# Make Numpy version for skimage to use
nim = np.array(fil)

# Image is now white blobs on black background, so label() it
label_image=label(nim)

# Iterate through blobs, saving each to disk
i=0
for region in regionprops(label_image):
   if region.area >= 100:
      # Extract rectangle containing blob and save
      name="blob-" + str(i) + ".png"
      minr, minc, maxr, maxc = region.bbox
      Image.fromarray(n[minr:maxr,minc:maxc,:]).save(name)
      i = i + 1

That gives these output images:

And the intermediate, debug images are result-1.png:

and result-2.png:

user_2836621 · Answer

Here's a third way to do it without needing to write any Python. It just uses ImageMagick in the Terminal - it's installed on most Linux distros and is available for macOS and Windows.

Basically, it uses the same techniques as my other answers - threshold, median filter, and "Connected Components Analysis", a.k.a. "labelling".

magick article.jpg -colorspace gray -threshold 95% -median 19x19  \
    -define connected-components:verbose=true                     \
    -define connected-components:area-threshold=100               \
    -connected-components 4 -auto-level output.png

Sample Output

Objects (id: bounding-box centroid area mean-color):
  4: 963x241+38+333 519.0,453.0 231939 srgb(0,0,0)
  0: 1045x590+0+0 528.0,204.0 155279 srgb(255,255,255)
  2: 393x246+292+73 488.0,195.5 96534 srgb(0,0,0)
  3: 303x246+698+73 849.0,195.5 74394 srgb(0,0,0)
  1: 238x246+39+73 157.5,195.5 58404 srgb(0,0,0)

The output has a header line telling you what the fields are, then one line for each blob found in the image. Let's look at the line:

2: 393x246+292+73 488.0,195.5 96534 srgb(0,0,0)

That means there is a blob 393 px wide and 246 px tall, at offset 292,73 from the top-left corner that we can draw on in half-transparent blue with this:

magick article.jpg -fill "rgba(0,0,255,0.5)" -draw "rectangle 292,73 685,319" result.png

And we can crop out with this:

magick article.jpg -crop 393x246+292+73 result.png

The labelled image (output.png) from the very first command looks like this - you will see that each blob is labelled with a different colour (shade of grey):

Note that if your ImageMagick version is v6 or older, you need to use convert instead of magick in all the foregoing commands.

user_2836621 · Answer

Here's a way to do it, using OpenCV findContours() method.

#!/usr/bin/env python3

import numpy as np
import cv2

# Load image
im = cv2.imread('article.jpg',cv2.IMREAD_UNCHANGED)

# Create greyscale version
gr = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)

# Threshold to get black and white
_,grthresh = cv2.threshold(gr,230,255,cv2.THRESH_BINARY)
cv2.imwrite('result-1.png',grthresh)

# Median filter to remove JPEG noise
grthresh = cv2.medianBlur(grthresh,11)
cv2.imwrite('result-2.png',grthresh)

# Find contours
im2, contours, hierarchy = cv2.findContours(grthresh,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)

# Look through contours, checking what we found
blob = 0
for i in range(len(contours)):
    area  = cv2.contourArea(contours[i])
    # Only consider ones taller than around 100 pixels and wider than about 300 pixels
    if area > 30000:
        # Get cropping box and crop
        rc = cv2.minAreaRect(contours[i])
        box = cv2.boxPoints(rc)
        Xs = [ box[0,0], box[1,0], box[2,0], box[3,0]]
        Ys = [ box[0,1], box[1,1], box[2,1], box[3,1]]
        x0 = int(round(min(Xs)))
        x1 = int(round(max(Xs)))
        y0 = int(round(min(Ys)))
        y1 = int(round(max(Ys)))
        cv2.imwrite(f'blob-{blob}.png', im[y0:y1,x0:x1])
        blob += 1

It gives you these files:

-rw-r--r--@ 1 mark  staff  248686  6 Jun 09:00 blob-0.png
-rw-r--r--@ 1 mark  staff   92451  6 Jun 09:00 blob-1.png
-rw-r--r--@ 1 mark  staff  101954  6 Jun 09:00 blob-2.png
-rw-r--r--@ 1 mark  staff  102373  6 Jun 09:00 blob-3.png
-rw-r--r--@ 1 mark  staff  633624  6 Jun 09:00 blob-4.png

The intermediate, debug files are result-1.png:

And result-2.png:

How to crop multiple rectangles or squares from JPEG?

3 Answers