I'm inverting very large matrices on an HPC. Obviously, this has high RAM requirements. To avoid out-of-memory errors, as a temporary solution I've just been requesting a lot of memory (TBs). How might I predict the required memory from the input matrix size for a matrix inversion using numpy.linalg.inv to more efficiently run HPC jobs?
1 Answers
TL;DR: up to O(32 n²) bytes for a (n,n) input matrix of type float64.
numpy.linalg.inv calls _umath_linalg.inv internally without performing any copy or creating any additional big temporary arrays. This internal function itself calls LAPACK functions internally. As far as I understand, the wrapping layer of Numpy is responsible for allocating the output Numpy matrix. The C code itself allocates a temporary array (see: here). No other array allocations appear to be performed by Numpy for this operation. There are several Lapack implementation and so it is not possible to generally know how much memory is requested by Lapack calls. However, AFAIK, almost all Lapack implementations does not allocate data in your back: the caller has to do it (especially for sgesv/dgesv which is used here). Assuming the (n, n) input matrix is of type float64 and FORTRAN integers are 4-byte wise (which should be the case on most platform, especially on Windows), then the actual memory required (taken by both the input matrix, the output matrix and the temporary matrix) is 8 n² + 8 n² + (8 n² + 8 n² + 4 n) bytes which is equal to (32 n + 4) n or simply O(32 n²) bytes. Note that the temporary buffer is the maximum size and may not be fully written which mean that the OS can physically map (ie. reserve in physical RAM) only a fraction of the allocated space. This is what happens on my (Linux) machine with OpenBLAS: only 24 n² bytes appear to be actually physically mapped. For float32 matrices, it is half the space.
Related