BouncyCastleProvider not picked for Signature object

I have an RSASSA-PSS/SHA256 signature to verify and I am using BouncyCastleProvider and Android system. Please have a look at the following snippet:

Security.removeProvider(BouncyCastleProvider.PROVIDER_NAME);
Security.insertProviderAt(BouncyCastleProviderSingleton.getInstance(), 1);

try {
      RSAPublicKey publicKey = rsaKey.toRSAPublicKey();

      String jcaAlg = "RSASSA-PSS";
      PSSParameterSpec pssSpec = new PSSParameterSpec("SHA-256", "MGF1", MGF1ParameterSpec.SHA256, 32, 1);

      Signature signature = Signature.getInstance(jcaAlg, BouncyCastleProviderSingleton.getInstance());
      signature.setParameter(pssSpec);
      signature.initVerify(publicKey);
      signature.update(obj2.getSigningInput());
      boolean v = signature.verify(obj2.getSignature().decode());

      Signature signature2 = Signature.getInstance(jcaAlg);
      signature2.setParameter(pssSpec);
      signature2.initVerify(publicKey);
      signature2.update(obj2.getSigningInput());
      boolean v2 = signature2.verify(obj2.getSignature().decode());
} catch (Exception ex) {
      throw new EncryptionException("Extracting public RSA key from certificate failed", ex);
}

Basically, I am trying to verify obj2 signature twice. One time, I supply BouncyCastleProvider directly, passing it into Signature.getInstance() method. The other time I am skipping this argument and I trust BouncyCastleProvider will be picked anyway, because I inserted it at the most-prioritized place in Provider list. This probably works as expected, as both signature.getProvider() and signature2.getProvider() return the same instance of BouncyCastleProvider. However, values of v and v2 are different. First verification succeeds (this is the correct behavior, v equals true), and second fails (v equals false). Why would that happen if in the end both Signatures should be using the same provider?

I am using an emulator running API 28.

Thanks in advance for any suggestions.