How do I merge image back after splitting into blocks

user_18012096 0
Viewed 86

I splitted an image into blocks with the code below

block = []
for x in range(0, 224,16):
    for y in range(0, 224,16):
        block.append(im[y:y+16, x:x+16])
np_block = np.array(block)

enter image description here

The image shape is 224,224,3 and each block is 16,16,3. How do I combine these blocks together to form the complete image?. When I reshaped it (np_block.reshape(224, 224,3)), I got the below image which is completely different from the real image

enter image description here

Edit: This has been answered in the comment section as thus:

reshaped_img = np_block.reshape(14,14,16,16,3).transpose(0,3,1,2,4).reshape(14*16,14*16,3)

The resulting image is rotated and also flipped compared to the original image. Here is the original image

original image

And here is the resulting image from the provided method to merge the blocks together

merged image

Doing restored_img = np.flip(np.rot90(reshaped_img, k=1, axes=(0, 1)), axis=0) will correctly restore the resulting image to its original form.

However, converting restored_img into a tensor in pytorch, tensor_img = torch.from_numpy(restored_img)throws the following error:

ValueError: At least one stride in the given numpy array is negative, and tensors with negative strides are not currently supported. (You can probably work around this by making a copy of your array  with array.copy().)

I worked around this as suggested in the error message.

tensor_img = torch.from_numpy(restored_img.copy()) got rid of the error but the transformations (rotation and flipping) made on restored_img is not preserved. tensor_img stil remains the same as the last image above.

I noticed that the splitted blocks are not appended in the correct order when splitting the original image. This is evident in np_block when I visualized it (the first image above looks flipped and rotated). So, I changed the line block.append(im[y:y+16, x:x+16]) to block.append(im[x:x+16, y:y+16])

Revisualizing np_block consequently gives the following image

enter image description here

This seem closer to the original image and the blocks also looks like they are in the correct order (this is what I've observed). But the blocks do not overlap with each other.

How do I work around this, please?

pytorch image-processing numpy python
1 Answers

Hi there the problem with original solution proposed is that you have to rearange the blocks 0,3 and 1,2 if you swap those dimensions then it will be solved. check:

block = [] #create blocks
for x in range(0, 224,16):
    for y in range(0, 224,16):
        block.append(im[y:y+16, x:x+16])
np_block = np.array(block)

reshaped_img = np_block.reshape(14,14,16,16,3).transpose(1,2,0,3,4).reshape(14*16,14*16,3)
#check indices are now (1,2,0,3,4) instead of (0,3,1,2,4) 

plt.imshow(np.hstack([im,reshaped_img]))

Edit

also from original comment this is just transpose output dimensions reshaped_img.transpose(1,0), but if you can correct in the original reshape it's a better solution

Edit 2

using the corrected stacking (with x in first dimension) the code would be:

block = []
for x in range(0, 224,16):
    for y in range(0, 224,16):
        block.append(im[x:x+16, y:y+16])
np_block = np.array(block)

reshaped_img = np_block.reshape(14,14,16,16,3).transpose(0,2,1,3,4).reshape(14*16,14*16,3)

Here makes morse sence: the first dimension of reshape in block (dim 0) with the first dimension of the subblock (dim 2), are grouped, then the same for the seccond dimensions of the block and and subblock (1,3).

In any case you can check the good solution by:

plt.imshow(np.hstack([im,reshaped_img]))

Edit 3 Last edit, here a code to check the original block reshape (with code of edit 2). Here you can see every block has some sense of getting a cut in the image:


b = np_block.reshape(14,14,16,16,3)
plt.figure()
for i in range(14):
    for j in range(14):
        plt.subplot(14,14,14*i+j+1)
        plt.imshow(b[i,j])
        plt.axis('off')
user_171944180
Related