singular matrix or not: conflict between determinant and rank

user_17502081 0
Viewed 38

I made a correlation matrix from exponentially smoothed returns, using Appendix C in https://openresearch-repository.anu.edu.au/bitstream/1885/65527/2/01_Pozzi_Exponential_smoothing_weighted_2012.pdf as a guide.

It's a 101 by 101 matrix, but I don't know if it is singular or not, due to the following conflicting results:

  • pracma::Rank says its rank is 101;

  • matrixcalc::is.singular.matrix returns TRUE;

  • base::determinant.matrix gives a very close-to-zero value.

pracma::Rank(try.wgtd.cor)
#[1] 101
matrixcalc::is.singular.matrix(try.wgtd.cor)
#[1] TRUE
base::determinant.matrix(try.wgtd.cor, logarithm = FALSE)
#$modulus
#[1] 2.368591e-55
#attr(,"logarithm")
#[1] FALSE
#
#$sign
#[1] 1
#
#attr(,"class")
#[1] "det"

Does anyone know why/how this could be?

r matrix
1 Answers

No no no, don't rely on determinant. A small determinant does not necessarily means singularity. For example, the following diagonal matrix is not singular at all, but has a very small determinant.

## all diagonal elements are 0.1; dimension 101 x 101
D <- diag(0.1, nrow = 101, ncol = 101)

## the obvious way to compute det(D)
prod(diag(D))
#[1] 1e-101

## use base::determinant.matrix
determinant.matrix(D, logarithm = FALSE)$modulus
#[1] 1e-101

The determinant equals the product of eigenvalues. So in general, if all eigenvalues of a matrix are smaller than 1, then the determinant will be very small for sure!

matrixcalc::is.singular.matrix is based on determinant, so do not trust it. In addition, its result is too subjective because you can tweak tol:

doc

By contrast, pracma::Rank uses both QR and SVD factorizations to determine rank. The result is extremely reliable. Here is the source code of Rank (with my comments):

function (M) 
{
    if (length(M) == 0) 
        return(0)
    if (!is.numeric(M)) 
        stop("Argument 'M' must be a numeric matrix.")
    if (is.vector(M)) 
        M <- matrix(c(M), nrow = length(M), ncol = 1)
    ## detect rank by QR factorization
    r1 <- qr(M)$rank
    ## detect rank by SVD factorization
    sigma <- svd(M)$d
    tol <- max(dim(M)) * max(sigma) * .Machine$double.eps
    r2 <- sum(sigma > tol)
    ## check consistency
    if (r1 != r2) 
        warning("Rank calculation may be problematic.")
    return(r2)
}

In conclusion, your 101 x 101 matrix try.wgtd.cor actually has full rank!

user_48917380
Related