Python cv2.remap from mesh creates pixelated distortions

user_19583119 0
Viewed 46

I have a mesh that is 4 times smaller than an image, I want to distort the image with the information from the mesh but, when using cv2.remap it makes the distortion pixelated (See image below) . How could I make a smoother distortion?

Original:

enter image description here

Desired output:

enter image description here

My output:

enter image description here

My code:

img = np.array(Image.open('astronaut.jpg')) # Shape -> (512, 512, 3)
mesh = Mesh('astronaut.msh').get_uvs() # Shape -> (128, 128, 2), 2 channels for x and y 

new_mesh = np.zeros((img.shape[1], img.shape[0], 2))
new_mesh[:,:,0] = np.repeat(np.repeat(mesh[:,:,0], 4, axis=0), 4, axis=1)
new_mesh[:,:,1] = np.repeat(np.repeat(mesh[:,:,1], 4, axis=0), 4, axis=1)

nh, nw = img.shape[:2]
xs, ys = np.meshgrid(np.arange(0, nw), np.arange(0, nh))
xs = xs + new_mesh[:,:,0] * 4 # multiply by constant to modulate distort strength
ys = ys + new_mesh[:,:,1] * 4
xs = np.float32(xs)
ys = np.float32(ys)

dst= cv2.remap(img.astype(np.uint8), xs, ys, cv2.INTER_CUBIC)
opencv image-processing numpy python
1 Answers

OpenCV is not to blame. It does exactly what you tell it to.

Those artefacts come from your use of np.repeat. That merely repeats each index element in your map array. You're not upsampling your mesh properly, you effectively just copy 4x4 patches with that code.

Properly upsample your mesh (np.repeat is wrong), then you get good results. You can easily do that using cv.resize and anything other than nearest-neighbor interpolation. If you need to control boundary behavior exactly, you'll need warpAffine and a custom transformation matrix. You could even use cv.pyrUp (twice).

When you've presented a MRE (some data for the mesh), I'll update my answer with working code.

user_26028770
Related