Problem creating a Salesforce User via SCIMv2 when specifying ContactId

I am using the Salesforce REST APIs to create a Contact and then an associated User. If I allow the creation of a User (via SCIMv2) to create an Account and Contact automatically (as details at link), the creation works. Unfortunately, I need to populate a custom field in the Contact and must create it separately.

The link also states that I can include the AccountId and ContactId to prevent the auto-creation. To do this I include the specified Account and Contact in the SCIMv2 JSON as shown below.

{
  "schemas" : [ "urn:ietf:params:scim:schemas:core:2.0:User", "urn:salesforce:schemas:extension:external:2.0", "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User" ],
  "userName" : "fred.bloggs@example.com",
  "name" : {
    "familyName" : "Bloggs",
    "givenName" : "Fred"
  },
  "emails" : [ {
    "type" : "work",
    "primary" : true,
    "value" : "fred.bloggs@example.com"
  } ],
  "preferredLanguage" : "en_US",
  "locale" : "en_AU",
  "timezone" : "Australia/Hobart",
  "active" : true,
  "entitlements" : [ {
    "value" : "00e5j000000md3VAAQ"
  } ],
  "urn:salesforce:schemas:extension:external:2.0" : {
    "contactId" : "0039D00000JBLfvQAH",
    "accountId" : "0019D00000OkFMYQA3"
  }
}

When I make this request I get an error response from Salesforce:

Response Status:400 - Insert failed. First exception on row 0; first error: DUPLICATES_DETECTED, You're creating a duplicate record. We recommend you use an existing record instead.

Since there is definitely not a User object already in the system for this user, I am assuming it's trying to create the Contact again and failing.

Any help to understand what is happening and what I am doing wrong would be appreciated.