Most simple network protocol for a remote function call?

user_1156675 0
Viewed 102

I am looking for the most simple protocol to program a remote function call, e.g. from Matlab to Julia.

[out1, out2, ...] = jlcall(socket, fname, arg1, arg2, ...);

fname is a string, all other input and output variables are numerical arrays (or other structures known to both sides) on Linux, Windows as option.

  1. Client: open connection, block, pack and transmit
  2. Server: receive, unpack, process, pack and transmit back
  3. Client: receive, unpack, close connection and continue

The solutions I've seen (tcp, zmq) were built with old versions and do no longer work.

Protocol could (should?) be limited to do the pack/transmit - receive/unpack work.

UPDATE

Here is what I have come up with using pipes:

    function result = jlcall(varargin)
    % result = jlcall('fname', arg1, arg2, ...)
    % call a Julia function with arguments from Matlab
    
    if nargin == 0 % demo
      task = {'foo', 2, 3}; % demo fun, defined in jsoncall.jl
    else
      task = varargin;
    end
    
    % create pipe and write function and parameter(s) to pipe
    pipename = tempname;
    pipe = java.io.FileOutputStream(pipename);
    pipe.write(uint8(jsonencode(task)));
    pipe.close;
    
    % run Julia and read result back
    system(sprintf('julia jsoncall.jl %s', unixpath(pipename)))
    fid = fopen(pipename, 'r');
    c = fread(fid);
    result = jsondecode(char(c'));
    fclose(fid);
    
    function path_unix = unixpath(path_pc)
    %convert path to unix version
    path_unix = path_pc;
    path_unix(strfind(path_unix,'\'))='/';

    # jsoncall.jl
    
    using JSON3                             # get JSON3.jl from repository first
    
    function foo(a,b)                       # demo function
      a+b, a*b
    end
    
    jsonfile = ARGS[1]                      # called as > julia jsoncall.jl <json_cmdfile>
    io = open(jsonfile, "r")                # open IOStream for read
    data = read(io)                         # read UTF8 data from stream
    close(io)                               # close stream
    
    cmd = JSON3.read(String(data))          # unpack stream into [fun, farg] array
    fun = Symbol(cmd[1])                    # first element is Julia function name,
    result = @eval $fun(cmd[2:end]...)      # others are function arguments
    io = open(jsonfile, "w")                # open IOStream for write
    write(io, JSON3.write(result))          # (over-)write result back to stream
    close(io)                               # close stream

Open points:

  • my first use of pipes/streams
  • output formatting: where Julia outputs a tuple of two, Matlab creates an an nx2 array.
  • replace json by msgpack for performance, might help with type formatting as well.

Your comments are welcome!

julia remote-access zeromq tcp matlab
1 Answers

Here is a stripped down way of doing it. If you are going to vary your functions and arguments, a REST as in the comments server is going to be more flexible and less likely to pose a security risk (as you are eval()ing arbitrary code in some cases).

#server code
using Sockets

const port = 6001
const addr = ip"127.0.0.1"
const server = listen(addr, port)

while true
    try
        @info "Server on $port awaiting request..."
          sock = accept(server)
          @info "Server connected."
          msg = strip(readline(sock))
          @info "got message $msg"

          fstr, argstr = split(msg, limit=2)
          x = parse(Float64, argstr)  # or other taint checks here...
          ans = eval(Meta.parse(fstr * "($x)"))
          @info "server answer: $ans"
          write(sock, "$ans\n")
    catch y
        @info "exiting on condition: $y"
    end
end

#  client code
using Sockets

port = 6001
server = ip"127.0.0.1"

sock = connect(server, port)
@info "Client connected to $server"
func = "sinpi"
x = 0.5

@info "starting send"
write(sock, "$func $x\n")
flush(sock)
@info "flushed send"

msg = strip(readline(sock))  # read one line of input and \n, remove \n
ans = parse(Float64, msg)
println("answer is $ans")
close(sock)
user_42828470
Related