I have a data set with the size of (400,40). Some of the columns are completely zero. They are not necessary for the calculations (I need to ignore them), but they are needed to rewrite the file.
So I'm using numpy to import it as an array, get the initialization done. But a problem occurs when I try to inverse the matrix (again, needed for the calculations). As far as I know, if a matrix has a full zero column, it is impossible to inverse it (det(M) = 0).
So I am using this to get non-zero columns:
nonZero = dataSet[:, np.all(dataSet != 0, axis=0)]
(I also tried summing the column with np.sum within np.all) but it skips some columns for no reason.
For example, my first row has:
[ 0, -1, -2, -3, 181, 5451, 0, 0, 8, 8, 1, 9, 9, 1, 0.11, 0, 0 ] etc.
When I run the above code, I get:
[ -1. -2. -3. 181. 8. 8. 1. 9. 9. 1. ]
5451 and 0.11 disappears even though the whole column does not equal 0 nor they are 0.
I also need to get the removed column indexes, since I need to rewite them after the calculations...
I am not the best Python coder, but I can't seem to fix the problem, or understand why it happens. I recently learned how to use numpy and I am pretty beginner in it. This has been troubling me for 2 days now. Any recommendation/help is appreciated.