What's the protocol for calling Raku code from C code?

Viewed 440

Say I have my event-driven TCP communications library in C.

From my Raku application, I can call a function in the C library using NativeCall.

my $server = create-server("127.0.0.1", 4000);

Now, from my callback in C (say onAccept) I want to call out to a Raku function in my application (say on-accept(connection) where connection will be a pointer to a C struct).

So, how can I do that: call my Raku function on-accept from my C function onAccept ?

ps. I tried posting using a simple title "How to call Raku code from C code", but for whatever reason stackoverflow.com wouldn't let me do it. Because of that I concocted this fancy title.

I was creating a 32-bit DLL. We have to explicitly tell CMake to configure a 64-bit build.

cmake -G "Visual Studio 14 2015 Win64" ..

Anyway, now that the code runs, it's not really what I asked for, because the callback is still in C.

It seems that what I asked for it's not really possible.

I tried to use the approach suggested by Haakon, though I'm afraid I don't understand how it would work.

I'm in Windows, and unfortunately, Raku can't find my dlls, even if I put them in C:\Windows\System32. It finds "msvcrt" (C runtime), but not my dlls.

The dll code (Visual Studio 2015).

#include <stdio.h>

#define EXPORTED __declspec(dllexport)

typedef int (*proto)(const char*);

proto raku_callback;

extern EXPORTED void set_callback(proto);
extern EXPORTED void foo(void);

void set_callback(proto arg)
{
  printf("In set_callback()..\n");
  raku_callback = arg;
}

void foo(void)
{
  printf("In foo()..\n");
  int res = raku_callback("hello");
  printf("Raku return value: %d\n", res);
}

Cmake code for the

CMAKE_MINIMUM_REQUIRED (VERSION 3.1)
add_library (my_c_dll SHARED my_c_dll.c)

Raku code.

use v6.d;

use NativeCall;

sub set_callback(&callback (Str --> int32))
  is native("./my_c_dll"){ * }

sub foo()
  is native("./my_c_dll"){ * }

sub callback(Str $str --> Int) {
  say "Raku callback.. got string: {$str} from C";
  return 32;
}

## sub _getch() returns int32 is native("msvcrt") {*};
## print "-> ";
## say "got ", _getch();

set_callback(&callback);
# foo();

When I run

$ raku test-dll.raku
Cannot locate native library '(null)': error 0xc1
  in method setup at D:\tools\raku\share\perl6\core\sources
    \947BDAB9F96E0E5FCCB383124F923A6BF6F8D76B (NativeCall) line 298
  in block set_callback at D:\tools\raku\share\perl6\core\sources
     \947BDAB9F96E0E5FCCB383124F923A6BF6F8D76B (NativeCall) line 594
  in block <unit> at test-dll.raku line 21

Raku version.

$ raku -v
This is Rakudo version 2020.05.1 built on MoarVM version 2020.05
implementing Raku 6.d.
4 Answers

Another approach could be to save a callback statically in the C library, for example (libmylib.c):

#include <stdio.h>

static int (*raku_callback)(char *arg);

void set_callback(int (*callback)(char * arg)) {
    printf("In set_callback()..\n");
    raku_callback = callback;
}

void foo() {
    printf("In foo()..\n");
    int res = raku_callback("hello");
    printf("Raku return value: %d\n", res);
}

Then from Raku:

use v6;
use NativeCall;

sub set_callback(&callback (Str --> int32)) is native('./libmylib.so') { * }
sub foo() is native('./libmylib.so') { * }

sub callback(Str $str --> Int) {
    say "Raku callback.. got string: {$str} from C";
    return 32;
}

set_callback(&callback);
foo();

Output:

In set_callback()..
In foo()..
Raku callback.. got string: hello from C
Raku return value: 32

Raku is a compiled language; depending on the implementation you've got, it will be compiled to MoarVM, JVM or Javascript. Through compilation, Raku code becomes bytecode in the corresponding virtual machine. So it's never, actually, binary code.

However, Raku code seems to be cleverly organized in a way that an object is actually a pointer to a C endpoint, as proved by Haakon Hagland answer.

WRT to your latest problem, please bear in mind that what you are calling is not a path, but a name that is converted to a navive shared library name and also uses local library path conventions to look for them (it's `PATH' on Windows). So if it's not finding it, add local path to it of simply copy the DLL to one of the searched directories.

First of all, my apologies to @Håkon and @raiph. Sorry for being so obtuse. :)

Håkon's answer does indeed answer my question, although for whatever reason I have failed to see that until now.

Now the code I played with in order to understand Håkon's solution.


// my_c_dll.c
// be sure to create a 64-bit dll

#include <stdio.h>

#define EXPORTED __declspec(dllexport)

typedef int (*proto)(const char*);

proto raku_function;

extern EXPORTED void install_raku_function(proto);
extern EXPORTED void start_c_processing(void);

void install_raku_function(proto arg)
{
  printf("installing raku function\n");
  raku_function = arg;
}

void start_c_processing(void)
{
  printf("* ----> starting C processing..\n");

  for (int i = 0; i < 100; i++)
  {
    printf("* %d calling raku function\n", i);
    int res = raku_function("hello");
    printf("* %d raku function returned: %d\n", i, res);
    Sleep(1000);
  }
}

# test-dll.raku

use v6.d;

use NativeCall;

sub install_raku_function(&raku_function (Str --> int32))
  is native("./my_c_dll.dll") { * }

sub start_c_processing()
  is native("./my_c_dll.dll") { * }

sub my_raku_function(Str $str --> Int)
{
  say "@ raku function called from C with parameter [{$str}]";
  return 32;
}

install_raku_function &my_raku_function;

start { start_c_processing; }

for ^1000 -> $i
{
  say "# $i idling in raku";
  sleep 1;
}                               

$ raku test-dll.raku                   
installing raku function               
# 0 idling in raku                     
* ----> starting C processing..        
* 0 calling raku function              
@ 0 raku function called from C with parameter [hello]
* 0 raku function returned: 32         
# 1 idling in raku                     
* 1 calling raku function              
@ 1 raku function called from C with parameter [hello]
* 1 raku function returned: 32         
# 2 idling in raku                     
* 2 calling raku function              
@ 2 raku function called from C with parameter [hello]
* 2 raku function returned: 32         
# 3 idling in raku                     
* 3 calling raku function              
@ 3 raku function called from C with parameter [hello]
* 3 raku function returned: 32         
# 4 idling in raku                     
* 4 calling raku function              
@ 4 raku function called from C with parameter [hello]
* 4 raku function returned: 32         
# 5 idling in raku                     
* 5 calling raku function              
@ 5 raku function called from C with parameter [hello]
* 5 raku function returned: 32         
^CTerminate batch job (Y/N)?           
^C                                     

What amazes me is that the Raku signature for my_raku_function maps cleanly to the C signature ... isn't Raku wonderful ? :)

TL;DR In reference to this SO, a leading Raku(do) expert on embedding MoarVM and running Raku code from C in a multi-threaded context has said "callbacks are indeed the way to go". I currently think @jjmerelo's answer is incorrect and see no reason why @zentrunix's scenario can't be made to work.

About the two previous answers

Perhaps the approach shown is inadequate for @zentrunix's specific needs, but Håkon Hægland's answer proves that Raku code can be called from C.

Perhaps @jjmerelo's answer means something I don't understand, but it seems to contradict what is plainly proven in Håkon Hægland's.

"callbacks are indeed the way to go"

nine is probably the leading Raku(do) expert on calling Raku from C in a multi-threaded context.

I asked about @zentrunix's question on #raku. nine replied:

callbacks are indeed the way to go. Inline::Perl6 is an example of how to embed MoarVM and run Raku code from C


It's not quite 100% clear to me whether nine was just talking about the solution in @HåkonHægland's answer, or was actively confirming that it should work for @zentrunix's particular scenario as well.

That said, I do know that nine has long ago dealt with embedding Raku in C in a multi-threading scenario.

I think it highly likely that a straight-forward interpretation of nine's comment is that Raku(do) will work fine for @zentrunix's scenario, either using the simple looking approach in @HåkonHægland's answer, or perhaps the more complex looking approach in the Inline::Perl6 module linked above.

Until @zentrunix, or someone else, confirms Raku(do) actually works with @zentrunix's specific scenario, or definitively does not, I will continue to hold a neutral view in relation to it being doable or not. But that neutral view includes thinking that @jjmerelo and @zentrunix have not properly justified their conclusion that Raku(do) can't do as @zentrunix wants.

"[@HåkonHægland's solution is] not really what I asked for, because the callback is still in C"

I've been confused by @zentrunix's comments at every stage thus far of the evolution of this SO. I've tried to clear up my confusion by writing comments on their question and the existing answers, but that didn't help. So I've deleted them and written this nanswer instead.

my callback in C (say onAccept) I want to call out to a Raku [callback]

So, first, a callback calling a callback. Second, and much more importantly, the C code that's calling Raku code is doing so asynchronously, without coordination with Rakudo.

Will that work? I don't know. But I get the impression that nine is saying that it will, perhaps using the code they linked as a template, or perhaps even just using the simple code shown in @HåkonHægland's answer.

The protocol

From @zentrunix's comment on @JJ's answer:

interaction between Raku code and C code would have to serialized ... It seems that what I asked for it's not really possible.

Why not? Wouldn't, for example, Actors work?

Related