Cargo default build target per workspace members

I have three members in my workspace and I want to comiple the first member with musl and other two members with glibc. I have created a .cargo/config file in the project root folder with glibc configs. And created another .cargo/config file in the first member folder with musl configs. However my first member was not compiled with musl. I want to run the first member in an alpine docker container and I am managing docker containers with the second member. How to solve this?