Use valgrind with `R CMD check`

user_3438001 0
Viewed 333

I want to run valgrind on the tests, examples and vignettes of my package. Various sources insinuate that the way to do this should be:

R CMD build my-pkg
R CMD check --use-valgrind my-pkg_0.0.tar.gz

R CMD check seems to run fine, but shows no evidence of valgrind output, even after setting the environment variable VALGRIND_OPTS: --memcheck:leak-check=full. I've found sources that hint that R needs to run interactively for valgrind to show output, but R -d CMD check (or R -d "CMD check") seems to be the wrong format.

R -d "valgrind --tool=memcheck --leak-check=full" --vanilla < my-pkg.Rcheck/my-pkg-Ex.R does work, but only on the example files; I can't see a simple way to run this against my vignettes and testthat tests.

What is the best way to run all relevant scripts through valgrind? For what it's worth, the goal is to integrate this in a GitHub actions script.

r package unit-testing valgrind
2 Answers

Edit Mar 2022: The R CMD check case is actually simpler, running R CMD check --use-valgrind [other options you may want] will run the tests and examples under valgrind and then append the standard valgrind summary at the end of the examples output (i.e., pkg.Rcheck/pkg-Ex.Rout) and test output (i.e., pkg.Rcheck/tinytest.Rout as I use tinytest)_. What puzzles me now is that an error detected by valgrind does not seem to fail the test.

Original answer below the separator.

There is a bit more to this: you helps to ensure that the R build is instrumented for it. See Section 4.3.2 of Writing R Extensions:

On platforms where valgrind is installed you can build a version of R with extra instrumentation to help valgrind detect errors in the use of memory allocated from the R heap. The configure option is --with-valgrind-instrumentation=level, where level is 0, 1 or 2. Level 0 is the default and does not add anything. Level 1 will detect some uses117 of uninitialised memory and has little impact on speed (compared to level 0). Level 2 will detect many other memory-use bugs118 but make R much slower when running under valgrind. Using this in conjunction with gctorture can be even more effective (and even slower).

So you probably want to build yourself a Docker-ized version of R to call from your GitHub Action. I think the excellent 'sumo' container by Winston has a valgrind build as well so you could try that as well. It's huge at over 4gb:

edd@rob:~$ docker images| grep wch     # some whitespace edit out
wch1/r-debug       latest     a88fabe8ec81      8 days ago     4.49GB
edd@rob:~$

And of course if you test dependent packages you have to get them into the valgrind session too...

user_1433050

Unfortunately, I found the learning curve involved in dockerizing R in GitHub actions, per @dirk-eddelbuettel's suggestion, too steep.

I came up with the hacky approach of adding a file memcheck.R to the root of the package with the contents

devtools::load_all()
devtools::run_examples()
devtools::build_vignettes()
devtools::test()

(remembering to add to .Rbuildignore).

Running R -d "valgrind --tool=memcheck --leak-check=full" --vanilla < memcheck.R then seems to work, albeit with the reporting of some issues that appear to be false positives, or at least issues that are not identified by CRAN's valgrind implementation.

Here's an example of this in action in a GitHub actions script.

(Readers who know what they are doing are invited to suggest shortcomings of this approach in the comments!)

user_34380010
Related