How to find Phase Correlation between 2 3D matrices in python?

Question

Phase Correlation is calculated as follows:

The task is to detect duplicated content in the 3D domain by cross-correlating small 3D blocks.

R : residual matrix (209*64*48)

splitting R into non overlapping 3D blocks B of size 30 × 16 × 16

Find phase correlation between R and B

I have done the following:

import cv2
import numpy as np
import scipy

#the video is read
cap = cv2.VideoCapture('C:\Users\user\Downloads\project\rewind_dataset\videos\h264_lossless\01_forged.mp4')

#no of frames in video
total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))

#initializing video residual matrix
r = np.zeros((total-1,64,48),dtype=np.int)
#time frame
t=0

#calculating residual matrix
ret, frame = cap.read()
#grayscaled the frame
prev = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
while(1):

ret, frame = cap.read()
if ret==False: break

next = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
for i in range(64):
    for j in range(48):
        r[t][i][j]= int(prev[j*5][i*5])-int(next[j*5][i*5])

t+=1   
prev=next

#considering first 30 frames of R we have 12 blocks
#initializing the 12 blocks(30,16,16)
b1 = np.zeros((30,16,16),dtype=np.int)
b2 = np.zeros((30,16,16),dtype=np.int)
b3 = np.zeros((30,16,16),dtype=np.int)
b4 = np.zeros((30,16,16),dtype=np.int)
b5 = np.zeros((30,16,16),dtype=np.int)
b6 = np.zeros((30,16,16),dtype=np.int)
b7 = np.zeros((30,16,16),dtype=np.int)
b8 = np.zeros((30,16,16),dtype=np.int)
b9 = np.zeros((30,16,16),dtype=np.int)
b10 = np.zeros((30,16,16),dtype=np.int)
b11 = np.zeros((30,16,16),dtype=np.int)
b12 = np.zeros((30,16,16),dtype=np.int)

#assigning values to 12 blocks
for i in range(30):
  b1[i] = r[i][0:16,0:16]
  b2[i] = r[i][0:16,16:32]
  b3[i] = r[i][0:16,32:48]
  b4[i] = r[i][16:32,0:16]
  b5[i] = r[i][16:32,16:32]
  b6[i] = r[i][16:32,32:48]
  b7[i] = r[i][32:48,0:16]
  b8[i] = r[i][32:48,16:32]
  b9[i] = r[i][32:48,32:48]
  b10[i] = r[i][48:64,0:16]
  b11[i] = r[i][48:64,16:32]
  b12[i] = r[i][48:64,32:48]

# a,b,c,d,e,f are dummy matrices used for the calculation of Phase Correlation
#operations performed for a single block

#Phase Correlation Calculation

b1.resize(209,64,48)

#fourier transform of the block
a = np.fft.fft2(b1)

#conjugate of Fourier transform of residual matrix
b= np.conjugate(np.fft.fft2(r))

#Hadamard product of a and b
c=a*b

#absolute value of c
d=np.absolute(a*b)

#dividing d by c
e = np.zeros((209,64,48),dtype=np.cdouble)
np.true_divide(c,d, out=e, where=c!=0) 

#inverse fourier transform of e
f=np.fft.ifft2(e).real
print(f)
#f is the phase correlation matrix

cap.release()
cv2.destroyAllWindows()

I have the following doubts in what I have done:

1. is it necessary to grayscale the frames while calculating residual matrix? { residual matrix is difference between 2 consecutive frames in a video }

2. any better suggestion for dividing 3D matrix into non-overlapping sub blocks(3D matrices)?

3. using fft, fft2 and fftn gives different output.How does it impact phase correlation in videos?

4. finding Hadamard product a*b requires the 2 matrices to be of similar time * height * breadth. So I reshaped b(30,16,16) to b(210,64,48). Does this affect phase correlation calculation?

How to find Phase Correlation between 2 3D matrices in python?

0 Answers