Editing asp.net core identity claims instead of adding them

I'm using external logins for my asp.net core mvc website, and when the login completes I want to store the social info they used to login (such as username, profile image, etc.) as claims in the user's identity...

So I run code like this in my account controller after a successful login:

    var authClaim = user.Claims.FirstOrDefault(c => c.ClaimType == authToken.Name);
    if (authClaim == null)
          externalClaims.Add(new Claim(authToken.Name, authToken.Value));
    else
          authClaim.ClaimValue = authToken.Value;

    if (externalClaims.Any())
          await _userManager.AddClaimsAsync(user, externalClaims);

await _userManager.UpdateAsync(user);

however when I attempt to do this, the claims table is duplicating the claims rather than updating them as I expected, so every time the user logs in I get another collection of records in the table for that user that are identical to the last.

do I have to delete them and re-add them instead on every login? if not, how do I update existing claims without duplicating them?

I'm doing it on every login in case the user has changed their profile or other info (and to make sure I have the most up to date token).

I'm also curious why I add the claims to the main Identity of the user. The User has a property called ExternalClaims, but it is empty and there doesn't appear to be any way to update it. It seems to me this is a better place to put these third-party Claims, but I can't for the life of me find any way to modify it...

either way, I'm sure I'd have the same problem if I use this code, so is the proper course to delete the claim on login and always add it new, or should I be doing something differently?