Persistent 'No Access-Control-Allow-Origin Header Present' Error Asp.Net Core

I've tried every all sorts of ways to fix this, including trying stack overflow solutions, I'm getting

Access to XMLHttpRequest at '[api location]' from origin '[web app location]' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

when I run it using html. When I run it on swagger, its fine (same domain I guess?), and when I run it on postman, its fine and returns said Access-Control-Allow-Origin

my program.cs code is :

using Azure.Identity;
using [project name].Middleware;

var builder = WebApplication.CreateBuilder(args);

var keyVaultEndpoint = new Uri(Environment.GetEnvironmentVariable("VaultUri"));
builder.Configuration.AddAzureKeyVault(keyVaultEndpoint, new DefaultAzureCredential());

// Add services to the container.

builder.Services.AddControllers();
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseMiddleware<ApiKeyMiddleware>();
app.UseHttpsRedirection();

app.UseAuthorization();

app.MapControllers();

app.Run();

The js and html is

<script>
// WARNING: For POST requests, body is set to null by browsers.

var xhr = new XMLHttpRequest();
xhr.withCredentials = true;

xhr.addEventListener("readystatechange", function() {
  if(this.readyState === 4) {
    console.log(this.responseText);
  }
});

xhr.open("POST", "[api url]");
xhr.setRequestHeader("XApiKey", "[api key]");
xhr.setRequestHeader("Origin", "https://[web app site]");
xhr.setRequestHeader("Access-Control-Allow-Credentials", "true");
xhr.send();
</script>

And the api is

        [HttpPost("[Name]")]
        public IEnumerable<[Model]> [Name]([FromServices] IConfiguration Configuration, [params])
        {
            [sql stuff]
            Response.Headers.Add("Access-Control-Allow-Origin", "[web app]");
            Response.Headers.Add("Access-Control-Allow-Credentials", "true");
            Response.Headers.Add("Access-Control-Allow-Methods", "DELETE, POST, GET, OPTIONS");
            Response.Headers.Add("Access-Control-Allow-Headers", "Origin, Content-Type, Accept, Authorization, X-Requested-With");

            return Enumerable.Range(1, 1).Select(index => new [Model]
            {
                output = "Updated"
            });
        }

When i configure it in program.cs to allow any origin, i'm told access-control-allow-origin can't be a wildcard?

Any help is appreciated. Thank You! :]