How do you call Python code from C code?

user_117015 0
Viewed 82781

I want to extend a large C project with some new functionality, but I really want to write it in Python. Basically, I want to call Python code from C code. However, Python->C wrappers like SWIG allow for the OPPOSITE, that is writing C modules and calling C from Python.

I'm considering an approach involving IPC or RPC (I don't mind having multiple processes); that is, having my pure-Python component run in a separate process (on the same machine) and having my C project communicate with it by writing/reading from a socket (or unix pipe). my python component can read/write to socket to communicate. Is that a reasonable approach? Is there something better? Like some special RPC mechanism?

Thanks for the answer so far - however, i'd like to focus on IPC-based approaches since I want to have my Python program in a separate process as my C program. I don't want to embed a Python interpreter. Thanks!

interop cross-domain c python
9 Answers

I recommend the approaches detailed here. It starts by explaining how to execute strings of Python code, then from there details how to set up a Python environment to interact with your C program, call Python functions from your C code, manipulate Python objects from your C code, etc.

EDIT: If you really want to go the route of IPC, then you'll want to use the struct module or better yet, protlib. Most communication between a Python and C process revolves around passing structs back and forth, either over a socket or through shared memory.

I recommend creating a Command struct with fields and codes to represent commands and their arguments. I can't give much more specific advice without knowing more about what you want to accomplish, but in general I recommend the protlib library, since it's what I use to communicate between C and Python programs (disclaimer: I am the author of protlib).

user_16940

Have you considered just wrapping your python application in a shell script and invoking it from within your C application?

Not the most elegant solution, but it is very simple.

hhafez_423030
user_756520

I haven't used an IPC approach for Python<->C communication but it should work pretty well. I would have the C program do a standard fork-exec and use redirected stdin and stdout in the child process for the communication. A nice text-based communication will make it very easy to develop and test the Python program.

DShawley_417470

If I had decided to go with IPC, I'd probably splurge with XML-RPC -- cross-platform, lets you easily put the Python server project on a different node later if you want, has many excellent implementations (see here for many, including C and Python ones, and here for the simple XML-RPC server that's part the Python standard library -- not as highly scalable as other approaches but probably fine and convenient for your use case).

It may not be a perfect IPC approach for all cases (or even a perfect RPC one, by all means!), but the convenience, flexibility, robustness, and broad range of implementations outweigh a lot of minor defects, in my opinion.

user_958100

Few tips for binding it with Python 3

  1. file() not supported, use open()

ffi.cdef(open('interface.h').read())

  1. PyObject* PyStr_FromString(const char *u)
    Create a PyStr from a UTF-8 encoded null-terminated character buffer.
    Python 2: PyString_FromString
    Python 3: PyUnicode_FromString

Change to: PYVERIFY( pName = PyUnicode_FromString("interface") )

  1. Program name
    wchar_t *name = Py_DecodeLocale(argv[0], NULL);
    Py_SetProgramName(name);
  1. for compiling
   gcc cc.c -o cc -I/usr/include/python3.6m -I/usr/include/x86_64-linux-gnu/python3.6m -lpython3.6m
  1. I butchered dump def .. maybe it will give some ideas
def get_prediction(buffer, buffer_len):
    try:
        data = ffi.buffer(buffer, buffer_len)[:]
        result = ffi.new('char []', data)
      
        print('\n I am doing something here here........',data )

        resultA = ffi.new('char []', b"Failed")  ### New message 

        ##noGCDict[ffi.addressof(resultA)] = resultA 
        return resultA
    except:
        print >>sys.stderr, traceback.format_exc()
        return ffi.NULL
}

Hopefully it will help and save you some time

user_136200980
Related