Package Microsoft.AspNetCore.Authentication.JwtBearer 5.0.0 is not compatible with netcoreapp3.1 yet it's targeting net 5.0

I am migrating to .NET 5.0 from .Net Core 3.1.

I keep getting this error message:

Package Microsoft.AspNetCore.Authentication.JwtBearer 5.0.0 is not compatible with netcoreapp3.1 (.NETCoreApp,Version=v3.1) / win-x86. Package Microsoft.AspNetCore.Authentication.JwtBearer 5.0.0 supports: net5.0 (.NETCoreApp,Version=v5.0)

I understand what it is saying, but I don't know why it is saying it.

My project targets .NET 5.0.

My csproj file looks like this:

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <UserSecretsId>5e2f8086-7a94-402a-bd2a-4160e9236c8f</UserSecretsId>
  </PropertyGroup>


 <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Authentication" Version="2.2.0" />
    <PackageReference Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="5.0.0" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Abstractions" Version="2.2.0" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Formatters.Json" Version="2.2.0" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation" Version="5.0.0" />
    <PackageReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Design" Version="5.0.0" />
  </ItemGroup>

</Project>

I'm unsure why it's telling me about 3.1.

I manually removed the bin and obj folders, tried to update all nuget packages, and clean/rebuild the project, but I get the same issue each time.v

I am on Visual Studio 16.8.2.

EDIT

I should point out, the error is not a compiler error! If I debug in Visual Studio, the website spins up fine. This error message only occurs when I try to publish