Is it acceptable to run `test_that` in a for loop?

Viewed 160

I have a package that converts a specific type of file into a CSV file. There are a number of parameters that might affect the conversion, and not all of them are knowable (because the original process of conversion is only really known by the manufacturer of the original file).

I have unit tests for each part of the conversion and some tests on the full conversion of some very small files, but the only way I feel I truly know the conversion works is to convert some files using the software of the manufacturer, and with the package, and compare the two. If I find a set of parameters that aren't correctly converted, I'd want to know that fixing this doesn't mess up the conversions that already work.

Since I don't have files that cover all of the possible parameters, I'd like to build the testing so that it is easy to just add more files as they become available. The obvious way to do this seems to be in a loop. It would look something like this:

test_files <- list.files(<folder of test files>)
true_files <- list.files(<folder of known-good files>)

for (file in test_files){
    # Read in test and equivalent true file
    test_that(paste0("Conversion works for file ", file_name), {
        # Run some tests to check that they are equal
        ...
        })
}

My question is: are there reasons not to do it this way? Should I be writing a test for each file instead?

Thanks in advance.

Edit: In response to a comment, let me see if I can give a concrete example of why I want to test full conversion, and not just parts.

The files are measurements taken by a device. If a period is missing, it is imputed with the last observation, unless the last observation was already imputed, or if the last observation had a vector magnitude (sqrt of the sum of the squares of the measures) above a threshold. That threshold is dependent on one of the parameters and can be expressed as a regression line. The only way to find that dependency was to 'guess' the regression line from some observations. But now, if I find some new files with different parameters, I might have to update my guess of the regression. If I do this, I risk messing up the thresholds for existing files.

Why not just right a test for the threshold? Because I don't know the true values. I can only see if my guess is holding in some observed data.

0 Answers
Related