Conditionally Exclude a DLL from an MSI Installer Based on Configuration

I am able to easily exclude a DLL reference from a WPF project based on the configuration with a .csproj that looks like this:

<ItemGroup Condition="'$(Configuration)' == 'MYCONFIGURATION2'">
  <Reference Include="MYDLL">
    <HintPath>MYDLLPATH\MYDLL.dll</HintPath>
  </Reference>
</ItemGroup>

But when I create the installer using Wix, the DLL is included in the installer for all Configuration projects (ex. both MYCONFIGURATION1 and MYCONFIGURATION2). I understand that for some reason, the build includes in it's output any DLLs that it finds in the source tree. I could delete MYDLL.dll from the source, but then that DLL won't be included for the Configuration that requires it.

My issue is that I want to include MYDLL.dll in all builds that are MYCONFIGURATION2 but want to ensure that DLL is excluded in MYCONFIGURATION. How do I achieve this?