Implementing my own algorithm to scale and rotate images in python

user_4789830 0
Viewed 5003

I am trying to implement an algorithm in python to scale images by a factor or rotate them by a given angle (or both at the same time). I am using opencv to handle the images and I know opencv has these functions built in, however I want to do this myself to better understand image transformations. I believe I calculate the rotation matrix correctly. However, when I try to implement the affine transformation, it does not come out correctly. 

import numpy as np 
import cv2
import math as m 
import sys

img = cv2.imread(sys.argv[1])
angle = sys.argv[2]

#get rotation matrix
def getRMat((cx, cy), angle, scale):
    a = scale*m.cos(angle*np.pi/180)
    b = scale*(m.sin(angle*np.pi/180))
    u = (1-a)*cx-b*cy
    v = b*cx+(1-a)*cy
    return np.array([[a,b,u], [-b,a,v]]) 

#determine shape of img
h, w = img.shape[:2]
#print h, w
#determine center of image
cx, cy = (w / 2, h / 2)

#calculate rotation matrix 
#then grab sine and cosine of the matrix
mat = getRMat((cx,cy), -int(angle), 1)
print mat
cos = np.abs(mat[0,0])
sin  = np.abs(mat[0,1])

#calculate new height and width to account for rotation
newWidth = int((h * sin) + (w * cos))
newHeight = int((h * cos) + (w * sin))
#print newWidth, newHeight

mat[0,2] += (newWidth / 2) - cx
mat[1,2] += (newHeight / 2) - cy

#this is how the image SHOULD look
dst = cv2.warpAffine(img, mat, (newWidth, newHeight))

cv2.imshow('dst', dst)
cv2.waitKey(0)
cv2.destroyAllWindows()

#apply transform
#attempt at my own warp affine function...still buggy tho
def warpAff(image, matrix, (width, height)):
    dst = np.zeros((width, height, 3), dtype=np.uint8)
    oldh, oldw = image.shape[:2]
    #print oldh, oldw
    #loop through old img and transform its coords
    for x in range(oldh):
        for y in range(oldw):
            #print y, x
            #transform the coordinates
            u = int(x*matrix[0,0]+y*matrix[0,1]+matrix[0,2])
            v = int(x*matrix[1,0]+y*matrix[1,1]+matrix[1,2])
            #print u, v
            #v -= width / 1.5
            if (u >= 0 and u < height) and (v >= 0 and v < width):
                dst[u,v] = image[x,y]
    return dst


dst = warpAff(img, mat, (newWidth, newHeight))

cv2.imshow('dst', dst)
cv2.waitKey(0)
cv2.destroyAllWindows()

Image I am using for testing

transformation opencv python
1 Answers

You're applying the rotation backward.

This means that for an angle of 20, instead of rotating 20 degrees clockwise, you rotate 20 degrees counterclockwise. That on its own would be easy to fix—just negate the angle.

But it also means that, for each destination pixel, if no source pixel exactly rotates to it, you end up with an all-black pixel. You could solve that by using any interpolation algorithm, but it's making things more complicated.

If we instead just reverse the process, and instead of calculating the destination (u, v) for each (x, y), we calculate the source (x, y) for every destination (u, v), that solves both problems:

def warpAff(image, matrix, width, height):
    dst = np.zeros((width, height, 3), dtype=np.uint8)
    oldh, oldw = image.shape[:2]
    # Loop over the destination, not the source, to ensure that you cover
    # every destination pixel exactly 1 time, rather than 0-4 times.
    for u in range(width):
        for v in range(height):
            x = u*matrix[0,0]+v*matrix[0,1]+matrix[0,2]
            y = u*matrix[1,0]+v*matrix[1,1]+matrix[1,2]
            intx, inty = int(x), int(y)
            # We could interpolate here by using something like this linear
            # interpolation matrix, but let's keep it simple and not do that.
            # fracx, fracy = x%1, y%1
            # interp = np.array([[fracx*fracy, (1-fracx)*fracy],
            #                    [fracx*(1-fracy), (1-fracx)*(1-fracy)]])
            if 0 < x < oldw and 0 < y < oldh:
                dst[u, v] = image[intx, inty]
    return dst

Now the only remaining problem is that you didn't apply the shift backward, so we end up shifting the image in the wrong direction when we turn everything else around. That's trivial to fix:

mat[0,2] += cx - (newWidth / 2)
mat[1,2] += cy - (newHeight / 2)

You do have one more problem: your code (and this updated code) only works for square images. You're getting height and width backward multiple times, and they almost all cancel out, but apparently one of them doesn't. In general, you're treating your arrays as (width, height) rather than (height, width), but you end up comparing to (original version) or looping over (new version) (height, width). So, if height and width are different, you end up trying to write past the end of the array.

Trying to find all of these and fix them is probably as much work as just starting over and doing it consistently everywhere from the start:

mat = getRMat(cx, cy, int(angle), 1)
cos = np.abs(mat[0,0])
sin  = np.abs(mat[0,1])
newWidth = int((h * sin) + (w * cos))
newHeight = int((h * cos) + (w * sin))
mat[0,2] += cx - (newWidth / 2)
mat[1,2] += cy - (newHeight / 2)

def warpAff2(image, matrix, width, height):
    dst = np.zeros((height, width, 3), dtype=np.uint8)
    oldh, oldw = image.shape[:2]
    for u in range(width):
        for v in range(height):
            x = u*matrix[0,0]+v*matrix[0,1]+matrix[0,2]
            y = u*matrix[1,0]+v*matrix[1,1]+matrix[1,2]
            intx, inty = int(x), int(y)
            if 0 < intx < oldw and 0 < inty < oldh:
                pix = image[inty, intx]
                dst[v, u] = pix
    return dst

dst = warpAff2(img, mat, newWidth, newHeight)

It's worth noting that there are much simpler (and more efficient) ways to implement this. If you build a 3x3 square matrix, you can vectorize the multiplication. Also, you can create the matrix more simply by just multiplying a shift matrix @ a rotation matrix @ an unshift matrix instead of manually fixing things up after the fact. But hopefully this version, since it's as close as possible to your original, should be easiest to understand.

user_9084940
Related