Compile python modules for arm64 architecture

user_16843804 0
Viewed 573

I've been trying to install some python modules which use C code which is compiled based on the architecture of the host. The final scripts would be deployed on an arm64 architecture, which does not have access to the internet, so installing the modules directly on it is impossible. My machine is an Intel Mac, so it has a different architecture than arm64. I was thinking about creating a docker image with the arm64v8/python docker image and then transferring it into the arm64, but I get the error of:

Using default tag: latest
latest: Pulling from arm64v8/python
no matching manifest for linux/amd64 in the manifest list entries

That means that this solution will not work.

Does anyone have any tips or tricks on how to approach such problem?

arm64 docker python
2 Answers

It's normal that you get this error. Your Mac is an x86_64 machine and the manifest for the arm64v8/python image does not contain your architecture, therefore you can't run it.

However, what you're trying to achieve is possible and you're in the right direction. I think the Docker docs here are a good starting point on how to achieve this.

If the modules you're using are already compiled for arm64v8 and you're just downloading them with pip3, then you can just try and build the image, targeting arm64v8. Try the following command and see if you're able to build the image:

docker buildx build --platform linux/arm64/v8 -t your-arm-image:latest .

If that doesn't work, then the solution would be to build your code (and modules), targeting the arm64v8 architecture. You do this on your own Mac - it's called cross compilation. Look for how it is done for your Python version. After you've done that, you can then build your Docker image (copy your cross-compiled modules to it) and set the target architecture to arm64v8 using the same command from above.

user_161984150

Solution to your problem is Cross-Compilation. Cross-compilation means compile program on machine 2 (arch1) which will be run on machine 2 (arch2), where arch1 != arch2. Good library for cross compilation is clang. Good example of cross compilation usage is android ecosystem, where you have code written in c/c++ and can be run on arm32, arm64, x86 or even x86_64.

Look here for examples: https://www.96boards.org/documentation/guides/crosscompile/commandline.html or https://github.com/tpoechtrager/osxcross

user_16150700
Related