Can not load dylib on MacOS

I'm trying to inject a simple dynamic library on MacOS in different binary files.

This is my simple dynamic library: inject.c

#include <stdio.h>

__attribute__((constructor))
static void myCusomConstructor()
{
    printf("Hello from dylib\n");
}

I compile it with gcc -dynamiclib inject.c -o libinject.dylib. And then when I try to inject it in /bin/ls binary by DYLD_INSERT_LIBRARIES=/Users/user/libinject.dylib /bin/ls I get an error:

dyld: could not load inserted library '/Users/user/libinject.dylib' because no suitable image found. Did find:
    /Users/user/libinject.dylib: incompatible cpu-subtype: 0x00000000 in /Users/user/libinject.dylib
    /Users/user/libinject.dylib: stat() failed with errno=1

zsh: abort      DYLD_INSERT_LIBRARIES=/Users/user/libinject.dylib /bin/ls

So, what I am doing wrong? I run this test on MacBook Pro 2020 with Apple M1 and MacOS Big Sur.

More info about issue

Output for file /bin/ls:

/bin/ls: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/bin/ls (for architecture x86_64):      Mach-O 64-bit executable x86_64
/bin/ls (for architecture arm64e):      Mach-O 64-bit executable arm64e

So, on my machine /bin/ls is a fat binary.

Also, I tried to compile inject.c for different CPU architectures:

arm64: gcc -arch arm64 -dynamiclib inject.c -o libinject-arm64.dylib

Output:

dyld: could not load inserted library '/Users/user/libinject-arm64.dylib' because no suitable image found. Did find:
    /Users/user/libinject-arm64.dylib: incompatible cpu-subtype: 0x00000000 in /Users/user/libinject-arm64.dylib
    /Users/user/libinject-arm64.dylib: stat() failed with errno=1

zsh: abort      DYLD_INSERT_LIBRARIES=/Users/user/libinject-arm64.dylib /bin/ls

arm64e: gcc -arch arm64e -dynamiclib inject.c -o libinject-arm64e.dylib

Output:

dyld: could not load inserted library '/Users/user/libinject-arm64e.dylib' because no suitable image found. Did find:
    /Users/user/libinject-arm64e.dylib: incompatible cpu-subtype: 0x00000002 in /Users/user/libinject-arm64e.dylib
    /Users/user/libinject-arm64e.dylib: stat() failed with errno=1

zsh: abort      DYLD_INSERT_LIBRARIES=/Users/user/libinject-arm64e.dylib /bin/ls

x86_64: gcc -arch x86_64 -dynamiclib inject.c -o libinject-x86.dylib

Output:

dyld: could not load inserted library '/Users/user/libinject-x86.dylib' because no suitable image found. Did find:
    /Users/user/libinject-x86.dylib: mach-o, but wrong architecture
    /Users/user/libinject-x86.dylib: stat() failed with errno=1

zsh: abort      DYLD_INSERT_LIBRARIES=/Users/user/libinject-x86.dylib /bin/ls