I am using gam in mgcv to fit the model
m <- gam(y ~ s(x) + s(Group, bs = "re"))
however this makes the covariance-matrix
vcov(m)
non-positive definite. Further, the estimated s(x) is just a straight line.
Now, removing the s(x) part fixes the vcov problem, and removing the s(Group) also fixes these problems and then the estimated curve s(x) is not a straight line.
Does anybody know why this happens, and how to fix it? Such that I can include both s(x) and s(Group), and also get a positive-definite vcov matrix, and properly estimated curves (even if they were to be insignificant)?
I believe it is probably caused by the fact that x and Group are confounded to a degree, such that only one of them is needed?