Assigning Roles to a Discord Member Already in Guild

I'm trying to assign a discord role to a member using OAuth2 on my website. They sign in, I get the access token (which works with other requests), and then send the request below with every header and variable being correct.

factionRoleMapping = { OG: "929218237030354994" };
  const newRoles = [...currentRoles, factionRoleMapping[faction]];
  var rolesInfoFetch = await fetch(
    `https://discord.com/api/v9/guilds/${config.GUILD_ID}/members/${req.body.uid}`,
    {
      method: "PUT",
      headers: { Authorization: req.headers.authorization },
      body: {
        roles: newRoles,
      },
    }
  );

I read the docs and it said that I would get a 204 No Content if the user was already in the discord guild, which I'm guessing should be okay. But I keep getting 401 Unauthorized.

The current permissions are "guilds", "guilds.join", "guilds.members.read", and "identify".

I'm not even sure if this is possible the way I'm doing it, but any help is appreciated!