Julia: inject code containing keyword into scipt

user_12142942 0
Viewed 42

I have a function containing a for loop. I would like to add an argument to that function that lets me chose to run the loop parrallel using Threads.@threads for i in .... I thus just need to inject Threads.@threads in front of the loop. Macros don´t work as they can´t handle keywords. Alterntively I could have something like

if parrallel
    inject("Threads.@threads for i in 1:n")
else
    inject("for i in 1:n")
end

    loop content....
end

I can´t find any way to insert code like that. What to do?

It would of course be an option to put the whole loop in a function and just use an if else containing the for loops over the function but I would prefer the rest of the code as is.

julia macros keyword
2 Answers

It would of course be an option to put the whole loop in a function and just use an if else containing the for loops over the function but I would prefer the rest of the code as is.

Note that higher order functions and syntactic sugar like do blocks make such solutions relatively straightforward to develop and easy to read:

You can start defining two higher-order functions that abstract away the for loop.

# This one is basically `Base.foreach`
function sequential_for(f, iterable)
    for i in iterable
        f(i)
    end
end

# A thread-parallel version
function parallel_for(f, iterable)
    Threads.@threads for i in iterable
        f(i)
    end
end

Then your function can dynamically decide which version of the for loop it wants to use:

function my_fun(n; parallel=false)
    for_loop = parallel ? parallel_for : sequential_for
    
    x = zeros(Int, n)

    # The do syntax avoids having to either
    # - define the loop body as a named function elsewhere, or
    # - put an hard-to-read lambda directly as argument to `for_loop`
    for_loop(1:n) do i
        x[i] = Threads.threadid()
        sleep(0.1)  # Let's make sure we see the effect of parallelism :-)
    end
    return x
end

Example use:

julia> @time my_fun(10)
  1.025307 seconds (299 allocations: 17.109 KiB)
10-element Array{Int64,1}:
 1
 1
 1
 1
 1
 1
 1
 1
 1
 1

julia> @time my_fun(10, parallel=true)
  0.235430 seconds (18.44 k allocations: 979.714 KiB)
10-element Array{Int64,1}:
 1
 1
 2
 2
 3
 4
 5
 6
 7
 8
user_12256070

Inserting code just in time doesn't work nicely or easily. The compiler would not know what to do and it would not be optimized at all and would not be safe. For a similar question see: Julia: inject code into function .

I would write the code either by writing a separate function for the for loop (in my opinion the cleaner way), or by leaving the for loop inside your first function and just writing it double.

Something like:

function forloopcontent()
    println(Threads.threadid())
end

function f(parallel::Bool)
    if parallel
        Threads.@threads for i in 1:10
            forloopcontent()
        end
    else for i in 1:10
            forloopcontent()
        end
    end
end

Otherwise, you could also write another function for the parallel version and two different methods for the normal one. I.e.:

function f(;parallel::Bool=false)
    parallel ? (return par_f()) : (return f())
end 

function f()
    for i in 1:10
        println(Threads.threadid())
    end
end

function par_f()
    Threads.@threads for i in 1:10
        println(Threads.threadid())
    end
end

This version can be called as f(;parallel=true) for the parallel version or f(;parallel=false) for the not parallel version.

user_68509010
Related