I am trying to find how one variable (EVI) may predict a binary outcome (an_larv_bin) using glmer from lme4 package. The code I input is:
univ_points_evi <- glmer(an_larv_bin ~ EVI + (1|grid_no), family="binomial", data=univ_points)
and I come up against the following warning message:
In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model failed to converge with max|grad| = 0.0331857 (tol = 0.002, component 1)
I have rescaled and centered the variable as such:
scale(EVI, center = TRUE, scale = TRUE)
And still receive the same warning. The next steps I took from Rstudio troubleshooting for lme4 convergence warnings. I checked for singularity, but it was not a problem:
> tt <- getME(univ_points_evi,"theta")
> ll <- getME(univ_points_evi,"lower")
> min(tt[ll==0])
[1] 0.80632
I restarted the function from a different fit:
ss <- getME(univ_points_evi,c("theta","fixef"))
m2 <- update(univ_points_evi,start=ss,control=glmerControl(optCtrl=list(maxfun=2e4)))
And get the same warning again. I checked the gradient calculations and got the embedded results:
#pre-computed information
derivs1 <- univ_points_evi@optinfo$derivs
sc_grad1 <- with(derivs1,solve(Hessian,gradient))
max(abs(sc_grad1))
[1] 4.991948e-05
max(pmin(abs(sc_grad1),abs(derivs1$gradient)))
[1] 4.991948e-05
#numDeriv
dd <- update(univ_points_evi,devFunOnly=TRUE)
pars <- unlist(getME(univ_points_evi,c("theta","fixef")))
grad2 <- grad(dd,pars)
hess2 <- hessian(dd,pars)
sc_grad2 <- solve(hess2,grad2)
max(pmin(abs(sc_grad2),abs(grad2)))
[1] 3.308277
The third figure is obviously quite different from the first two. I am unsure what this is actually telling me, and whether it is the root of the warning?
Lastly, I checked the fit of all optimisers:
univ_points_evi.all <- allFit(univ_points_evi)
And got the following:
bobyqa : [OK]
Nelder_Mead : [OK]
nlminbwrap : [OK]
nmkbw : [OK]
optimx.L-BFGS-B : [OK]
nloptwrap.NLOPT_LN_NELDERMEAD : [OK]
nloptwrap.NLOPT_LN_BOBYQA : [OK]
And tried all available optimisers, which converge to values that are practically equivalent.
ss <- summary(univ_points_evi.all)
ss$ fixef
ss$ llik
ss$ sdcor
ss$ theta
The lme4 package guidelines state that they would in this case consider the convergence warnings to be false positives. I am wondering whether this is true in my case, or if there is an issue with the gradient calculations which is causing the issue.
My data:
> dput(EVI)
c(0.499929994, 0.589900017, 0.593994021, 0.589900017, 0.601158023,
0.492922992, 0.546519995, 0.601045012, 0.536565006, 0.592272997,
0.592227995, 0.645565987, 0.61619997, 0.516200006, 0.516200006,
0.4639, 0.4639, 0.561200023, 0.5898, 0.564800024, 0.5898, 0.5898,
0.605099976, 0.595300019, 0.545300007, 0.572000027, 0.599600017,
0.585300028, 0.591700017, 0.533399999, 0.552100003, 0.569700003,
0.592499971, 0.596199989, 0.53490001, 0.53490001, 0.53490001,
0.553300023, 0.582899988, 0.545000017, 0.592100024, 0.582899988,
0.59009999, 0.569299996, 0.612900019, 0.533500016, 0.583299994,
0.772599995, 0.772599995, 0.682500005, 0.682500005, 0.682500005,
0.772599995, 0.628099978, 0.626299977, 0.628099978, 0.747399986,
0.640200019, 0.531899989, 0.680199981, 0.535099983, 0.680199981,
0.535099983, 0.565299988, 0.680199981, 0.703199983, 0.703199983,
0.541700006, 0.678200006, 0.678200006, 0.547100008, 0.634899974,
0.696399987, 0.688199997, 0.574899971, 0.574899971, 0.669799984,
0.611000001, 0.61559999, 0.639100015, 0.669799984, 0.669799984,
0.611000001, 0.59890002, 0.639100015, 0.604799986, 0.604799986,
0.604799986, 0.606599987, 0.606599987, 0.640600026, 0.624899983,
0.640600026, 0.624899983, 0.624899983, 0.640600026, 0.640600026,
0.516200006, 0.507499993, 0.507499993, 0.46540001, 0.530300021,
0.530300021, 0.565100014, 0.546599984, 0.546599984, 0.530399978,
0.530399978, 0.530399978, 0.523199975, 0.523199975, 0.546400011,
0.546599984, 0.496600002, 0.530799985, 0.537800014, 0.545000017,
0.496600002, 0.496600002, 0.514100015, 0.530799985, 0.530799985,
0.537800014, 0.530200005, 0.530200005, 0.546599984, 0.546599984,
0.576399982, 0.46540001, 0.516200006, 0.530399978, 0.655300021,
0.680999994, 0.660000026, 0.661499977, 0.661499977, 0.680999994,
0.655300021, 0.617799997, 0.647099972, 0.647099972, 0.617799997,
0.673300028, 0.673300028, 0.507700026, 0.507700026, 0.507700026,
0.651799977, 0.591799974, 0.591799974, 0.688300014, 0.661499977,
0.661499977, 0.661499977, 0.661499977, 0.648500025, 0.648500025,
0.495799989, 0.495799989, 0.495799989, 0.648899972, 0.648899972,
0.673300028, 0.673300028, 0.648500025, 0.647099972, 0.691999972,
0.647099972, 0.647099972, 0.617799997, 0.657199979, 0.706499994,
0.591799974, 0.661499977, 0.661499977, 0.641600013, 0.648500025,
0.648500025, 0.688300014, 0.495799989, 0.495799989, 0.688300014,
0.582000017, 0.582000017, 0.57069999, 0.582000017, 0.62559998,
0.565500021, 0.565500021, 0.62559998, 0.593599975, 0.604700029,
0.599699974, 0.536800027, 0.600300014, 0.600300014, 0.604700029,
0.566699982, 0.566699982, 0.626900017, 0.626900017, 0.594900012,
0.594900012, 0.584500015, 0.586199999, 0.605700016, 0.584699988,
0.553799987, 0.542900026, 0.584699988, 0.584699988, 0.575399995,
0.579999983, 0.579299986, 0.596899986, 0.594900012, 0.565500021,
0.579299986, 0.594900012, 0.565500021, 0.549499989, 0.549499989,
0.549499989, 0.549499989, 0.606899977, 0.539600015, 0.584699988,
0.571699977, 0.56129998, 0.595600009, 0.62559998, 0.565500021,
0.565500021, 0.620299995, 0.620299995, 0.594900012, 0.579999983,
0.654299974, 0.654299974, 0.627600014, 0.627600014, 0.64349997,
0.687699974, 0.64349997, 0.59859997, 0.59859997, 0.649999976,
0.518299997, 0.658299983, 0.658299983, 0.658299983, 0.627600014,
0.658299983, 0.658299983, 0.627600014, 0.667500019, 0.653100014,
0.564899981, 0.561999977, 0.629000008, 0.639999986, 0.639999986,
0.675100029, 0.675100029, 0.658299983, 0.659300029, 0.658299983,
0.659300029, 0.657400012, 0.645299971, 0.425599992, 0.425599992,
0.474299997, 0.598800004, 0.595200002, 0.416399986, 0.564899981,
0.564899981, 0.70599997, 0.70599997, 0.664699972, 0.484299988,
0.496199995, 0.496199995, 0.484299988, 0.517499983, 0.517499983,
0.517499983, 0.535899997, 0.51730001, 0.562399983, 0.540000021,
0.540000021, 0.501299977, 0.501299977, 0.528599977, 0.532400012,
0.51730001, 0.562399983, 0.501299977, 0.574299991, 0.528599977,
0.528599977, 0.528599977, 0.503499985, 0.568700016, 0.521799982,
0.503499985, 0.521799982, 0.557699978, 0.557699978, 0.545099974,
0.532400012, 0.563399971, 0.530700028, 0.431100011, 0.431100011,
0.510900021, 0.556400001, 0.501299977, 0.48120001, 0.48120001,
0.528800011, 0.528800011, 0.62470001, 0.62470001, 0.707899988,
0.707899988, 0.62529999, 0.62529999, 0.630500019, 0.646300018,
0.604900002, 0.62529999, 0.669799984, 0.634199977, 0.634199977,
0.634199977, 0.612999976, 0.662400007, 0.698700011, 0.632799983,
0.682099998, 0.428499997, 0.513300002, 0.569700003, 0.519500017,
0.519500017, 0.48120001, 0.48120001, 0.646399975, 0.559899986,
0.564899981, 0.564899981, 0.564899981, 0.602699995, 0.602699995,
0.60650003, 0.575699985, 0.5722, 0.584299982, 0.584900022, 0.584900022,
0.5722, 0.584299982, 0.5722, 0.560699999, 0.560699999)
> dput(an_larv_bin)
c(1L, 0L, 1L, 1L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L,
0L, 1L, 0L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L,
0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 1L, 1L, 1L, 1L, 0L,
0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L,
1L, 0L, 0L, 0L, 1L, 1L, 1L, 0L, 1L, 1L, 0L, 0L, 1L, 0L, 0L, 0L,
0L, 1L, 1L, 1L, 1L, 1L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 0L, 0L, 0L,
1L, 1L, 0L, 1L, 1L, 1L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 1L, 1L, 1L,
0L, 1L, 0L, 0L, 0L, 0L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 0L,
0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 1L, 1L,
1L, 1L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 1L, 1L, 1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L,
0L, 1L, 1L, 1L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L,
0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L,
0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L,
0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 1L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L,
0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 1L, 0L,
1L)
> dput(grid_no)
c(4937L, 3270L, 2854L, 3270L, 2582L, 2584L, 2585L, 2584L, 4663L,
3416L, 3416L, 3979L, 2986L, 4839L, 4839L, 4937L, 4937L, 3264L,
2854L, 2854L, 2289L, 2289L, 2582L, 3978L, 3834L, 3416L, 3416L,
3547L, 3687L, 2852L, 4388L, 4388L, 4538L, 4538L, 4937L, 4937L,
4937L, 2854L, 2854L, 2996L, 2996L, 2289L, 2582L, 3416L, 3692L,
2983L, 2301L, 4937L, 3264L, 3264L, 3547L, 3547L, 3547L, 3264L,
3822L, 3683L, 3683L, 3678L, 2427L, 2427L, 2427L, 2289L, 2427L,
2289L, 4117L, 2710L, 2582L, 2303L, 2854L, 2854L, 4520L, 3692L,
3692L, 3416L, 4526L, 4527L, 3264L, 3685L, 3685L, 4937L, 3264L,
3264L, 3685L, 4801L, 4937L, 2290L, 2289L, 2289L, 2854L, 2854L,
2581L, 2719L, 2719L, 2578L, 2578L, 2582L, 2581L, 3416L, 3978L,
3978L, 3416L, 3549L, 3549L, 2986L, 2700L, 2700L, 4680L, 4680L,
4680L, 4670L, 4670L, 2428L, 4527L, 3264L, 2854L, 4937L, 2582L,
3264L, 3264L, 3264L, 2854L, 2854L, 4937L, 2289L, 2289L, 4527L,
4680L, 4680L, 3416L, 3416L, 4680L, 3409L, 3547L, 3685L, 3685L,
3685L, 3409L, 3547L, 2861L, 2581L, 2578L, 2861L, 2430L, 2430L,
2293L, 2293L, 2293L, 3977L, 3684L, 4523L, 4669L, 3264L, 3264L,
3264L, 3264L, 2854L, 2854L, 2289L, 2289L, 2289L, 2577L, 2577L,
4937L, 4937L, 2577L, 2582L, 2582L, 2578L, 2578L, 3416L, 3416L,
4527L, 4801L, 3685L, 3822L, 2302L, 2855L, 2855L, 4669L, 2287L,
2287L, 4669L, 3549L, 3549L, 4798L, 3549L, 4680L, 4680L, 4680L,
4822L, 4258L, 4948L, 3273L, 4677L, 4677L, 4677L, 4948L, 2854L,
2854L, 3264L, 3264L, 3264L, 4937L, 4937L, 2582L, 2582L, 2578L,
2578L, 2289L, 2289L, 2289L, 3416L, 2573L, 3416L, 4527L, 3685L,
3547L, 4801L, 3685L, 3547L, 2287L, 2287L, 2287L, 2287L, 2436L,
2291L, 2718L, 2718L, 4099L, 3131L, 4680L, 4680L, 4680L, 3260L,
3260L, 3977L, 2571L, 2578L, 2578L, 2854L, 2854L, 3264L, 3264L,
3264L, 4937L, 4937L, 2582L, 2582L, 2289L, 2289L, 2289L, 2573L,
2573L, 2573L, 2573L, 3132L, 3407L, 3416L, 3416L, 3685L, 3685L,
3685L, 4527L, 4801L, 2991L, 2287L, 2287L, 2426L, 3399L, 2301L,
4680L, 4680L, 4680L, 4541L, 4390L, 3277L, 3277L, 3277L, 3978L,
3978L, 3978L, 4937L, 4801L, 4801L, 4937L, 2289L, 2289L, 2289L,
2573L, 2854L, 3264L, 3264L, 3264L, 3684L, 3684L, 2582L, 2582L,
2854L, 3264L, 3684L, 4527L, 2578L, 2578L, 2718L, 2718L, 2296L,
4665L, 4665L, 4665L, 3416L, 3416L, 3277L, 3277L, 2443L, 2300L,
2302L, 4680L, 4680L, 4680L, 3546L, 3546L, 4937L, 4937L, 4801L,
4801L, 2854L, 2854L, 3264L, 3264L, 2289L, 2289L, 2582L, 2582L,
2578L, 2289L, 3416L, 3416L, 3416L, 3556L, 3277L, 3685L, 3978L,
4680L, 4110L, 4237L, 4527L, 4237L, 4937L, 4937L, 4801L, 4801L,
3264L, 3685L, 3416L, 3416L, 3416L, 2289L, 2289L, 2289L, 2582L,
2578L, 2582L, 2293L, 2857L, 2721L, 2443L, 4680L, 4680L)
Thanks in advance!