Publishing protocol buffer messages over websockets in Julia

user_888379 0
Viewed 325

I'm working on a project where I want to have a Julia server process periodically publishing messages that involve some binary data. The initial client will be written in javascript, but we'd ultimately like the capability of having multiple clients, implemented in various ways. For that reason, I'd like to define the message using protocol buffers. I've turned up 3 Julia websocket implementations: WebSockets.jl, SimpleSockets.jl and HTTP.jl's WebSockets. I've done some naive experimenting with WebSockets.jl and got an error ("WebSockets does not support byte I/O"). My inclination is to shift my focus to the HTTP.jl implementation; I get the impression that it is under more active development than WebSockets.jl.

Update: I've continued with my experimentation. I was guided to a previous SO question, unable to write binary data in websocket, that was instructive. I modeled my server implementation after that in the link, yielding:

include("testmessage_pb.jl")

text = "A man spekith"
msg = TestMessage(someText=text)

function server(port)
    @async HTTP.listen(Sockets.localhost, port) do http::HTTP.Stream
        if HTTP.WebSockets.is_upgrade(http.message)
            HTTP.WebSockets.upgrade(http, binary=true) do ws
                while !eof(ws)
                    data = readavailable(ws)
                    IOExtras.startwrite(ws)
                    writeproto(ws, msg)
                    IOExtras.closewrite(ws)
                end
            end
        end
    end
end

Running it, I got an error very much like the previous one I reported: HTTP.WebSockets.WebSocket{HTTP.ConnectionPool.Transaction{Sockets.TCPSocket}} does not support byte I/O

I'm wondering if anyone reading this has any experience with this particular cluster of technologies (julia, protobuf, websockets) and suggestions on how to proceed ("don't try" would count as useful feedback).

julia protocol-buffers websocket
2 Answers

HTTP.jl websockets are working fine. Havn't tried it with protobuf, but hardly there should be any problem with that. It's basically

HTTP.WebSockets.open(data_url) do ws
   x = readavailable(ws)
   # Do protobuf related things, generate response
   write(ws, response)
end

readavailable returns you UInt8[] so you can do whatever you want with it.

user_14941350

I was able to tweak the server implementation that I showed in the question, and it now works. It's a little clunky, and I'll hold off from accepting my own answer in hopes that a nicer approach comes along. In any event, here's the new version:

function server(port)
    @async HTTP.listen(Sockets.localhost, port) do http::HTTP.Stream
        if HTTP.WebSockets.is_upgrade(http.message)
            HTTP.WebSockets.upgrade(http, binary=true) do ws
                while !eof(ws)
                    data = readavailable(ws)
                    iob = PipeBuffer()
                    writeproto(iob, msg)
                    write(ws, take!(iob))
                end
            end
        end
    end
end
user_8883790
Related