Requirement
for(i in 1:nsim)
{
for(fi in 1:length(FUNCTIONS))
for(p1 in 1:length(param1))
{
for(p2 in 1:length(param2))
{
# ... [mathematically and so on]
for(pN in length(paramN))
{
# do something here
# with (i, fi, p1, p2, ... pN)
Variadic Functions
Below is the stackoverflow definition. To me, it represents the right amount of abstraction for a problem that can be solved using a function (generally procedural).
Motive: Benchmarking
So I was reviewing some of my code and options using microbenchmark based on some string-manipulation issues I have been having. Going down the "rabbit hole", I reviewed the microbenchmark nano-timings and decided to see if they agreed with standard protocols. The microbenchmark computations are relatively fast but the print and summary are slow, and I wanted some benchmark references so I wrote a function ggg.benchmark that lead to this output.
time.is ... [milliseconds (ms)] per call
idx expression min lower-trecile median upper-trecile max relative.efficiency relative.factor Rank
2 2 cpp_time() 0.00000035 0.00000057 0.00000065 0.00000074 0.00024615 52.90 0.47101 1
4 4 cpp_milli()/1000 0.00000040 0.00000060 0.00000069 0.00000080 0.00011427 50.00 0.50000 2
6 6 cpp_micro()/1000000 0.00000039 0.00000062 0.00000071 0.00000082 0.00011449 48.55 0.51449 3
5 5 cpp_now("milli")/1000 0.00000049 0.00000068 0.00000083 0.00000098 0.00010462 39.86 0.60145 4
7 7 cpp_now("micro")/1000000 0.00000049 0.00000068 0.00000084 0.00000100 0.00145392 39.13 0.60870 5
9 9 cpp_now("nano")/1000000000 0.00000050 0.00000069 0.00000084 0.00000100 0.00011454 39.13 0.60870 6
1 1 as.numeric(Sys.time()) 0.00000102 0.00000129 0.00000138 0.00000149 0.00000814 0.00 1.00000 7
8 8 cpp_nano()/1000000000 0.00000123 0.00000150 0.00000157 0.00000171 0.00000986 -13.77 1.13768 8
3 3 .now() 0.00000130 0.00000160 0.00000169 0.00000186 0.00001599 -22.46 1.22464 9
11 11 time.now("cpp") 0.00000598 0.00000684 0.00000714 0.00000769 0.00141655 -417.39 5.17391 10
12 12 time.now("base") 0.00000599 0.00000686 0.00000716 0.00000770 0.00002909 -418.84 5.18841 11
10 10 time.now("first") 0.00000710 0.00000811 0.00000851 0.00000918 0.00128414 -516.67 6.16667 12
13 13 str.uniqid() 0.00017347 0.00017944 0.00018093 0.00018328 0.00289352 -13010.87 131.10870 13
My function behaved okay, but I added a caching algorithm to speed it up (see deviation of max from median), and ran the following:
> ggg.benchmark(mb.irony)
"milliseconds (ms)"
"mil"
"057b92b71ccfa406fc5d509f8b459d0e"
time.is ... [milliseconds (ms)] per call
idx expression min lower-trecile median upper-trecile max relative.efficiency relative.factor Rank
5 5 ggg.benchmark(mb.res) 53.57657 54.12626 54.17876 54.88361 218.67247 99.00 0.00998 1
4 4 summary(mb.res, "eps") 4475.81297 4488.51471 4489.81239 4523.80740 4533.85203 17.32 0.82683 2
1 1 summary(mb.res, "ns") 5413.74017 5424.41295 5430.13278 5456.10914 5600.27456 0.00 1.00000 3
2 2 summary(mb.res, "us") 5416.56048 5446.56292 5448.34089 5459.35504 5474.32398 -0.34 1.00335 4
3 3 summary(mb.res, "ms") 5418.25978 5449.20442 5462.20611 5493.99542 5578.11555 -0.59 1.00591 5
Features of Benchmarking
- FUNCTIONS: Allow a
variablenumber of expressions or functions to be evaluated - PARAMS with OPTIONS: Allow a
variablenumber of parameters withvariableoptions on each to be considered - Include a NULL function to above to capture system-performance norms
- For a given trial (nsim), randomize: FUNCTION order, PARAM order
- For good data provenance, record
seedson randomness - Allow for
Rprofand/orRprofmem - Allow for comparison checks to the first FUNCTION (the
referent) or a popular results (thecommon) to test accuracy at the same time
As I reviewed my needs for a benchmarking simulation, I came up with the following function parameters:
bm = function(..., list=NULL,
setup = list("nsim" = 100, "compare.values" = TRUE, "identical.to" = "referent", "tol" = 0, "Rprof" = FALSE, "Rprofmem" = FALSE),
params = list(
"str" = c("<i>hello friend</i>", " hello there world ", " ¡hola!, ¿que tal? "),
"sep" = c(" ", "", "¿", "</i>")
)
)
{
Stumped:
In the above scenario, which was the basis for the design. I have two parameters: the string str and the separator sep. I can pass into these a collection of parameter CHOICES or OPTIONS. And on a given nest for loop, I will use one str CHOICE and one sep CHOICE to perform a call. This implies that for a single trial as nsim, I have (1+FUNCTIONS * length(param1) * length(param2)) runs to perform. For nsim=1000 and 12 functions, 3 choices on param1, 4 choices on param2, I have the following:
Combinations to be performed: 1000×13×3×4 = 156,000
Maybe I can use a while loop. I could capture all of the unique combinations as a string to later perform a single run ...
12-2-3 ... FUNCTION[12] ... PARAM1[2] ... PARAM2[3] ...
For a given nsim ... I will loop over all of the FUNCTIONS in a random order, all of each PARAM in a random order ...
Maybe if I wasn't thinking so linearly/procedurally, there would be another way (recursion). Suggestions for other strategies are welcome, as I want to get it working.
Ultimately, however, this has been a lingering programming question I have had for a long time. I have hard-coded many a for-loop with fixed loops and can't think how to make it variadic. Any C-based solution is welcome: C++, C, javascript, R
QUESTION: How to create a variadic loop?
for(i in 1:nsim)
{
for(fi in 1:length(FUNCTIONS))
for(p1 in 1:length(param1))
{
for(p2 in 1:length(param2))
{
# ... [mathematically and so on]
for(pN in length(paramN))
{
# do something here
# with (i, fi, p1, p2, ... pN)