Reference ASP .NET Core 5 assemblies from class library

I'm a little bit lost and confused when migrating from .NET Core 2.2. to .NET 5. I've developed as class library which was referencing let's say Microsoft.AspNetCore.Http.Abstractions.

When I create new NET 5 application it implicitly references version 5.0.0 of said assembly (I understand because it uses Microsoft.NET.Sdk.Web, but when class library is created the highest version available from Nuget is 2.2.0.

How can I reference 5.0.0 from class library? Should I reference 2.2.0 (it still downloads it though) and hope that 5.0.0 is backwards compatible? What was the intended usage pattern?

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Http.Abstractions" Version="???"/>
  </ItemGroup>
</Project>

Edit: I found out that is should be done with:

<FrameworkReference Include="Microsoft.AspNetCore.App"/>

bit this will only work if targetting net5, is it possible to reference said assembly from netstandard2.1? If netstandard2.1 can only reference version 2.2.0 is it safe to do it?