How to align images by translation and rotation only (no warp) in Python

Viewed 1262

I need to align two images which are slightly shifted and rotated 180 deg. relative to each other. I tried several ways using opencv (in Python), but no luck.

Method 1 was using MOTION_AFFINE:

   im1 = cv2.imread(file1)    # Reference image. 
   im2 = cv2.imread(file2)  # Image to be aligned. 

   # Convert images to grayscale for computing the rotation via ECC method
   im1_gray = cv2.cvtColor(im1,cv2.COLOR_BGR2GRAY)
   im2_gray = cv2.cvtColor(im2,cv2.COLOR_BGR2GRAY)

   # Find size of image1
   sz = im1.shape

   # Define the motion model - euclidean is rigid (SRT)
   warp_mode = cv2.MOTION_AFFINE

   # Define 2x3 matrix and initialize the matrix to identity matrix I (eye)
   warp_matrix = np.eye(2, 3, dtype=np.float32)

   # Specify the number of iterations.
   number_of_iterations = 5000;

   # Specify the threshold of the increment
   # in the correlation coefficient between two iterations
   termination_eps = 1e-3;

   # Define termination criteria
   criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, number_of_iterations,  termination_eps)

   # Run the ECC algorithm. The results are stored in warp_matrix.
   (cc, warp_matrix) = cv2.findTransformECC (im1_gray, im2_gray, warp_matrix, warp_mode, criteria, None, 1)

   # Warp im2 using affine
   im2_aligned = cv2.warpAffine(im2, warp_matrix, (sz[1],sz[0]))#, flags=cv2.INTER_LINEAR + cv2.WARP_INVERSE_MAP);
   
   # Save the output. 
   cv2.imwrite(outfile, im2_aligned)

This didn't even converge.

Method 2 was using feature matching, like so:

    im1 = cv2.imread(file1)    # Reference image. 
    im2 = cv2.imread(file2)  # Image to be aligned. 

    img1 = cv2.cvtColor(img1_color, cv2.COLOR_BGR2GRAY) 
    img2 = cv2.cvtColor(img2_color, cv2.COLOR_BGR2GRAY) 
 
    height, width = img2.shape 
  
    # Create ORB detector with 4000 features. 
    orb_detector = cv2.ORB_create(4000) 


    # The first arg is the image, second arg is the mask 
    #  (which is not reqiured in this case). 
    kp1, d1 = orb_detector.detectAndCompute(img1, None) 
    kp2, d2 = orb_detector.detectAndCompute(img2, None) 
  
    # Match features between the two images. 
    # We create a Brute Force matcher with  
    # Hamming distance as measurement mode. 
    matcher = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True) 
      
    # Match the two sets of descriptors. 
    matches = matcher.match(d1, d2) 
      
    # Sort matches on the basis of their Hamming distance. 
    matches.sort(key = lambda x: x.distance) 
      
    # Take the top 90 % matches forward. 
    matches = matches[:int(len(matches)*90)] 
    no_of_matches = len(matches) 
  
    # Define empty matrices of shape no_of_matches * 2. 
    p1 = np.zeros((no_of_matches, 2)) 
    p2 = np.zeros((no_of_matches, 2)) 
  
    for i in range(len(matches)): 
        p1[i, :] = kp1[matches[i].queryIdx].pt 
        p2[i, :] = kp2[matches[i].trainIdx].pt 
  
    # Find the homography matrix. 
    homography, mask = cv2.findHomography(p1, p2, cv2.RANSAC) 
  
    # Use this matrix to transform the 
    # colored image wrt the reference image. 
    transformed_img = cv2.warpPerspective(img1_color, 
                        homography, (width, height)) 
    # Save the output. 
    cv2.imwrite(outfile, transformed_img)

This ended up rotating the second image to the first image's orientation, but warping it too much, so it looks like it's not even in the same plane.

Is there any way to combine feature-based matching of two images with a transform that only rotates and translates, but does not warp perspective?

Thank you!

0 Answers
Related