Windows 11 ERR_SSL_KEY_USAGE_INCOMPATIBLE

Viewed 3015

I support an app that uses Owin to host an HTTP interface on localhost. This has been working for years, but as of Windows 11 the bound certificate shows up as ERR_SSL_KEY_USAGE_INCOMPATIBLE. This is true for both Chrome and Edge. The cert didn't change, and works fine on Windows 10 (tested up to 20H2).

If I generate a new cert, using Windows 11, then the problem goes away. In comparing the old cert to new cert, I can't spot a reason why it wouldn't work.

Key Usage of the non-working cert: Non-Repudiation, Key Encipherment, Data Encipherment, Key Agreement (78) Enhanced: Server Authentication (1.3.6.1.5.5.7.3.1)

Key usage of the working cert: Digital Signature, Key Encipherment (a0) Enhanced: Client Authentication (1.3.6.1.5.5.7.3.2) Server Authentication (1.3.6.1.5.5.7.3.1)


Does anyone know what might be wrong with the old cert causing Windows 11, and only Windows 11, to reject it?



EDIT:
Thank you @Steffen Ullrich! What you said was definitely the push I needed. But would you say it is my answer?

On Windows 10, with the old cert, I used Chrome to find out: The connection to this site is encrypted and authenticated using TLS 1.2, ECDHE_RSA with P-384, and AES_256_GCM.

On Windows 11, with the new cert, I see: The connection to this site is encrypted and authenticated using TLS 1.3, X25519, and AES_256_GCM.

I had already disabled TLS 1.3 via browser settings, but obviously that didn't work. I used the registry to disable TLS 1.3 and now the original cert does work! (Connecting with TLS 1.2 just like Win 10).

Poking around I found this: https://docs.microsoft.com/en-us/windows/win32/secauthn/tls-cipher-suites-in-windows-11

Which shows that the cipher suite called out above is NOT supported by TLS 1.3. I always suspected this was a TLS 1.3 problem, I just couldn't put my finger on it.

So, I'll move forward with my new certs since they are "better" but at least I know why.

2 Answers

My guess is that the key exchanged method changed. Key Encipherment is suitable with RSA key exchange, which is obsolete for a while but might have been used in your code. Digital Signature is needed for DH key exchange instead (like in the modern ECDHE). See Which key usages are required by each key exchange method?.

The old certificate was suitable only for the obsolete RSA key exchange, the new certificate is suitable for both ECDHE and RSA key exchange. I'm less wondering why the old certificate does not work, I wonder more why it did not break earlier since RSA key exchange is obsolete for a while now.

I've solved this creating a new self-certificate and rebuild the same domain with the new self-certificate.

I hope that it'll be helpfull for you.

Related