Packaging a project that uses multiple python versions

user_7014995 0
Viewed 160

I'm trying to package a project that is mainly based on python 3 but uses some python 2 sub-projects (because of some specific dependencies). So the way that the python 3 modules use the python 2 programs is by calling them explicitly using Popen (because we can't just import them).

I've tried using pipenv but it seems that I can't create an environment for python 2 and python 3. I tried creating a python script that just creates wheels from all of the projects (using both of the versions) but it seems kind of hacky and wrong.

So the question is - What is the best way to package a project that uses multiple python versions?

pipenv virtualenvwrapper virtualenv python-3.x python
1 Answers

Problem is not only packaging. You're stuck using the interpreter tools for each Python version independently, and therefore you'll be shuffling around two binaries rather than one. I'm sure there are lots of vendors out there doing this already - Dropbox for example runs several Python based processes communicating with each other on a machine at a given time between the UI and the sync processes.

The second problem is sharing results between interpreters, that's one of the goals of the execnet library, but basically makes a shared memory channel outside of the local process space, and defines a common serialization protocol between each process. Popen is not very reliable if the commands you're calling don't have deterministic results and you want to further manipulate the results after it returns.

This IPC approach can be extended over a network to RPC or HTTP if you wanted to host your applications on external servers or using Docker, for example, which is one of the reasons microservices are popular - you can independently deploy and scale different "bounded contexts"

user_23086830
Related