How declare external functions in modules

user_2312304 0
Viewed 113

I have the following algorithm I've written in Julia module

module two_algo

export two

function two(type)
  a = construct(type)
  b = construct(type)
  result = construct(type)
  unity(a)
  unity(b)
  add(a,b,result)
  return result
end

end

At one time I had this algorithm defined outside a module and it would compile and could dispatch to any type that defined the construct, unity, and add methods. But within a module it won't compile. It doesn't know how to find construct, unity, and add. I don't want to import them from some specific module because I might have many modules that each define construct, unity, and add for example. What do I need to do to get this to compile? In my old C/C++ days I remember doing something as simple as declaring "extern construct;" within the module. I'm really hoping I can leave the two() function to take a single type rather than passing in functions that implement construct, unity, and add. Ideally I'd also like to avoid generics. Thanks for any pointers.

julia
2 Answers

First, let me say that you perhaps don't need modules here: keeping your project organized in separate files (for you to easily navigate in the sources) is probably a good idea, but having separate modules brings some complexity which might not be needed here (for example, there is no risk of name clashes that would require using separate modules/namespaces to avoid).

That being said, one way of organizing things here would be to make helper functions (like unity or add) belong to the same module as the generic algorithm that makes use of them. Here is a minimal example:

module Algo
export algo

# These functions are only declared; they have no method
function unity end
function add end

function algo(T)
    a = unity(T)
    add(a, a)
end

end

Other parts of the code that define new types would then have to extend the functions from the module to add specific methods:

using .Algo

struct MyType
    val :: Int
end

# Explicitly extend the `unity` and `add` function from module `Algo`
# (instead of merely defining new functions with the same name
Algo.unity(::Type{MyType}) = MyType(1)
Algo.add(a::MyType, b::MyType) = MyType(a.val + b.val)

And this should work as expected:

julia> Algo.algo(MyType)
MyType(2)


EDIT: A more complex organization allows to isolate all algorithms from one another (and potentially all types, too), by having generic functions declared in a "Base" module, that all other modules know about:

module AlgoBase
    export unity, add
    
    function unity end
    function add end
end

module Algo1
    using ..AlgoBase  # allows using unity and add    
    export algo1

    function algo1(T)
        a = unity(T)
        add(a, a)
    end
end

module Types1
    using ..AlgoBase
    export Type1
    
    struct Type1
        val :: Int
    end

    # Explicitly extend the `unity` and `add` function from module `AlgoBase`
    # (instead of merely defining new functions with the same name
    AlgoBase.unity(::Type{Type1}) = Type1(1)
    AlgoBase.add(a::Type1, b::Type1) = Type1(a.val + b.val)
end

using .Algo1
using .Types1

algo1(Type1)

This is more or less the way entire parts of the Julia ecosystem are designed, with several packages collaborating to provide algorithms related to a given field, without having to depend on one another: they only need to know about one "Base" package (think for example about StatsBase, which declares a lot of useful building blocks that types can implement, and algorithms can use). However, such an architecture is usually used to make entire (and unrelated) packages cooperate, i.e. not really at the scale of submodules within a single package. YMMV

user_12256070

This is a slight modification of Francois' answer that solved my problem. I am still accepting his answer because he did the lion's share of the discovery and work.

module UnityBase
    export unity
    
    function unity end
end

module AddBase
    export add
    
    function add end
end

module TwoAlgo
    using ..UnityBase, ..AddBase
    export two

    function two(T)
        a = unity(T)
        add(a, a)
    end
end

module MyTypeMod
    using ..AddBase, ..UnityBase
    export MyType
    
    struct MyType
        val :: Int
    end

    UnityBase.unity(::Type{MyType}) = MyType(1)
    AddBase.add(a::MyType, b::MyType) = MyType(a.val + b.val)
end

using .TwoAlgo
using .MyTypeMod

println(two(MyType))
user_23123040
Related