How do I design a test for the partial_tucker function from tensorly?

user_4803860 0
Viewed 199

I try to design a test in order to verify that the partial_tucker function from tensorly works as I expect it to work. In other words, I want to design an input for the partial_tucker function along with its associated expected output.

So, what I have tried to do is to take an initial random tensor A (of order 4), compute its "low rank" tucker decomposition by hand then reconstruct the tensor of same shape than the initial tensor, say A_tilde. I think the A_tilde tensor is then the "low rank approximation" of the initial tensor A. Am I correct?

Then I would like to us the partial_tucker function on that A_tilde tensor and I expect the result to be the same as the tucker decomposition that I have computed by hand. It is not the case so I guess my handcrafted tucker decomposition is wrong. If so, why?

import tensorly
import numpy as np

h, w, c, f = 3, 3, 64, 128
c_prim, f_prim = 16, 32
base_tensor = np.random.rand(h, w, c, f)


# compute tucker decomposition by hand using higher order svd describred here: https://www.alexejgossmann.com/tensor_decomposition_tucker/.
lst_fac = []
for k in [2, 3]:
    mod_k_unfold = tensorly.base.unfold(base_tensor, k)
    U, _, _ = np.linalg.svd(mod_k_unfold)
    lst_fac.append(U)

real_in_fac, real_out_fac = lst_fac[0], lst_fac[1]
real_core = multi_mode_dot(base_tensor, [real_in_fac.T, real_out_fac.T], modes=(2,3))
del base_tensor  # no need of it anymore

# what i call the "low rank tucker decomposition"
real_core = real_core[:,:,:c_prim,:f_prim]
real_in_fac = real_in_fac[:, :c_prim]
real_out_fac = real_out_fac[:, :f_prim]

# low rank approximation
base_tensor_low_rank = multi_mode_dot(real_core, [real_in_fac, real_out_fac], modes=(2,3))
in_rank, out_rank = c_prim, f_prim
core_tilde, (in_fac_tilde, out_fac_tilde) = partial_tucker(base_tensor_low_rank, modes=(2, 3), ranks=(in_rank, out_rank), init='svd')
base_tensor_tilde = multi_mode_dot(core_tilde, [in_fac_tilde, out_fac_tilde], modes=(2,3))
assert np.allclose(base_tensor_tilde, base_tensor_low_rank) # this is OK

assert np.allclose(in_fac_tilde, real_in_fac) # this fails

Note that I have tried to compute in_fac_tilde.T @ real_in_fac to see if it was identity or something like that, and I noticed that only the first column was colinear in both matrices, and orthogonal to all others.

tensorly
1 Answers

You are implicitely making a lot of assumptions here: you are assuming, for instance that you can just trim a rank-R decomposition to get the rank-(R-1) decomposition. This is in general not true. Also, note that the Tucker decomposition you are using is not just Higher-Order SVD (HO-SVD). Rather, HO-SVD is used for initialization and followed by Higher Order Orthogonal Iteration (HOOI).

You are also assuming that the low-rank decomposition is unique, for any given rank, which would allow you to compare the factors of the decomposition directly. This is also not the case (even in the matrix case, and with strong constraints such as orthonormality, you still would have sign indeterminacy).

Instead you could for example check the relative reconstruction error. I suggest you have a look at the tests in TensorLy. There are lots of good references on this, if you are starting with tensors. For instance, the seminal work by Kolda and Bader; for Tucker in particular, the work by De Lathauwer et al (eg.on best low-rank approximation of tensors), etc.

user_48021960
Related