I am using a Nix flake to define a devShell with a python environment and certain dependencies available in it. This environment consists of a mixture of packages from the public PyPI repositories, and several packages internal to our organisation which are also stored as Nix flakes. For various reasons (compatibility with non-Nix users, good dependency resolution) I'm using mach-nix to build the python packages and the environment.

However, I have a problem: one of the external packages (not under my control) is defined as an Nix flake but which uses the normal nixpkgs buildPythonPackage function instead of the mach-nix version. I need to include this package in my mach-nix environment. I cannot just rebuild it from source using mach-nix since the Nix build process includes various customisations that are required to get it to work in a Nix environment.

How can I get this package, build with the normal buildPythonPackage function, into my mach-nix environment?

Things I've tried

There's more detail on the things I've tried at https://github.com/DavHau/mach-nix/issues/453, but they haven't worked so far! The closest I've got is defining an overlay like so (where artiq) is the name of the package I'm trying to install, defined as an input to my flake):

artiq_override = self: super: {
  artiq = artiq.packages.x86_64-linux.artiq;
};

...

devShells.x86_64-linux.artiq = pkgs.mkShell {
  name = "artiq-environment";
  buildInputs = [
    (mach-nix.lib.x86_64-linux.mkPython {
      requirements = builtins.readFile ./requirements.in + ''
        artiq
      '';
      overridesPre = [ artiq_override ];
      providers = {
        artiq = "nixpkgs";
      };
    })
  ];
};

This fails however, with this error:

error: builder for '/nix/store/za9dczrb9937nspg6ikmyq0jpp8nxdy0-mach_nix_file.drv' failed with exit code 1;
       last 10 log lines:
       > Mach-nix version: master
       > Python: 3.9.12
       > Cause: Requirements conflict: artiq
       > The requirements which caused the error:
       >   artiq
       >
       > The given requirements might contain package versions which are not yet part of the dependency DB
       > currently used. The DB can be updated by specifying 'pypiDataRev' when importing mach-nix.
       > For examples see: https://github.com/DavHau/mach-nix/blob/master/examples.md
       >
       For full logs, run 'nix log /nix/store/za9dczrb9937nspg6ikmyq0jpp8nxdy0-mach_nix_file.drv'.
(use '--show-trace' to show detailed location information)

For the bold (to whom I am very grateful) here is an MWE:

{
  inputs = {
    nixpkgs.follows = "artiq/nixpkgs";
    artiq.url = "github:m-labs/artiq";
    mach-nix.url = "mach-nix";
  };

  outputs =
    { self
      , artiq
      , nixpkgs
      , mach-nix
    }:
    let
      pkgs = import nixpkgs { system = "x86_64-linux"; };

      artiq_override = self: super: {
        artiq = artiq.packages.x86_64-linux.artiq;
      };

    in
    rec {
      inherit pkgs;

      devShells.x86_64-linux.default = pkgs.mkShell {
        name = "icl-artiq-environment";
        buildInputs = [
          (mach-nix.lib.x86_64-linux.mkPython {
            requirements = ''
              numpy  # (for example - I actually need more)
              pip
              artiq
            '';
            overridesPre = [ artiq_override ];
            providers = {
              artiq = "nixpkgs";
            };
          })
        ];
      };
    };
}