AWS DotNet SDK Error: Unable to get IAM security credentials from EC2 Instance Metadata Service

I had the same issue, here is how I fixed it on my development environment

1. I created an AWS profile using the AWS extension for Visual studio
2. Once the profile is set up the credentials are passed using the profile and it worked fine for me

Point to note here, the user profile accessing the key manager should have a valid security group assigned for the Secrets manager.

Try it out let me know, how it went.

I've run into this issue a number of times, but have not been able to resolve it using the above solutions.

What has worked for me is explicitly setting my AWS profile using the AWS_PROFILE environment variable and setting it to the profile I want to use.

Today I ran into this issue again, where even that didn't work. What eventually solved it was setting the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

I dread the day where I run out of alternative ways to supply credentials to AWS.

Same issue and resolved by deleting $HOME/.aws/config and credentials files and recreating with AWS CLI.

In my case I was switching laptops from Windows to a new MBP. I had setup my new environment by copying the .aws directory files and confirmed that AWS CLI worked correctly. Confusingly the dotnet SDK failed with same errors.

Run the following command and follow the prompt using the data provided by AWS:

aws configure

I had the same issue, and resolved it by changing the name of the AWS profile in Visual Studio to default.

Since AWS SDK credentials configuration is causing a lot of headache, I'll throw in some context. First of all, if you are using dotnet core, use the AWSSDK.Extensions.NETCore.Setup package (https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/net-dg-config-netcore.html), which will respect your appsettings.json.

{
  "AWS": {
    "Region": "eu-west-1",
    "Profile": "theprofileyouwantouse"
  }
}

csproj:

  <ItemGroup>
    <PackageReference Include="AWSSDK.Extensions.NETCore.Setup" Version="3.7.1" />
    <PackageReference Include="AWSSDK.SecurityToken" Version="3.7.1.71" />
  </ItemGroup>

Example:

var config = host.Services.GetService<IConfiguration>();
var options = config.GetAWSOptions();
using var client = options.CreateServiceClient<IAmazonSecurityTokenService>();
var result = await client.GetCallerIdentityAsync(new Amazon.SecurityToken.Model.GetCallerIdentityRequest { });

This will try to pick up encrypted credentials in ~/AppData/Local/AWSToolkit and secondly based on your shared config file (~/.aws/config). As of november 2021, it does not utilize aws_access_key_id, aws_secret_access_key, aws_session_token in the version 1 shared credentials file (~/.aws/credentials)*

Next, if the roles you are assuming are AWS SSO, you need the following packages in your csproj file:

    <PackageReference Include="AWSSDK.SSO" Version="3.7.0.94" />
    <PackageReference Include="AWSSDK.SSOOIDC" Version="3.7.0.94" />

*If you happen to have invertedly added your credentials to your shared credentials file (~/.aws/credentials) as [profile myprofile] instead of just [myprofile] the SDK will not behave as you expected, so delete that. If your credentials file is fine, then you don't have to touch it, but keep in mind that the SDK will noe use the cached credentials if any found in that file.

Now, the author does not use the AWSSDK.Extensions.NETCore.Setup package, which means that we are getting a slightly different credentials resolving path. Most importantly: appsettings.json is not respected, this means you must specify the profile you want to use differently, for example by using the AWS_PROFILE environment variable.

Secondly, we are landing directly in the FallbackCredentialsFactory.cs which does this when resolving credentials:

            CredentialsGenerators = new List<CredentialsGenerator>
            {
#if BCL
                () => new AppConfigAWSCredentials(),            // Test explicit keys/profile name first.
#endif
                () => AssumeRoleWithWebIdentityCredentials.FromEnvironmentVariables(),
                // Attempt to load the default profile.  It could be Basic, Session, AssumeRole, or SAML.
                () => GetAWSCredentials(credentialProfileChain),
                () => new EnvironmentVariablesAWSCredentials(), // Look for credentials set in environment vars.
                () => ECSEC2CredentialsWrapper(proxy),      // either get ECS credentials or instance profile credentials
            };

Now the last step in resolving credentials "ECSEC2" has a fallback which returns this:

DefaultInstanceProfileAWSCredentials.Instance

Which leads us to the error which the author sees.

Summary:

1. If you are not using AWSSDK.Extensions.NETCore.Setup, specify the profile using an ENV-variable in launch.json or launchSettings.json if you are going to use the default constructor like the author
2. Rember to add the AWS SSO packages if needed

The question is not exactly my problem, but it's the first hit on google so I figured I'd chip in just in case.

I got the exact above error when issuing

dotnet lambda list-layers

It seems like the dotnet cli uses the AWS_PROFILE variable and does not default to AWS_DEFAULT_PROFILE. In my company, the AWS_DEFAULT_PROFILE is mapped to an identity provider, thus I do not manage different access with different profiles and the default profile is empty. As a workaround, run your command like this

AWS_PROFILE=$AWS_DEFAULT_PROFILE dotnet lambda list-layers

This way the CLI will use the correct credentials.

make sure you have the latest version of EC2config installed https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/UsingConfig_Install.html

Thanks

Just add env variables in control panel AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. The actual value of either is not important. I have set them both to a space (' '). Don't know why it works but it works. It does seem to take longer to log in. It seems that instead of going to the buggy flow, the SDK tries to use the env vars, fails and about after 30 seconds or so logs in as required.

Tested it on two different Win10 PCs with no AWS CLI installed or any AWS profile configured. The issue was recreated 100% and the described w/a fixed it.

I had the same issue and it turned out to be because I had AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN uppercased in my credentials file. Changing the keys to lowercase solved it for me.

In the project defaults.json, verify the profile value. in my case it was empty "profile": "". After setting the profile name, was able to publish

If anyone is using docker-compose and getting this error, I added this to my docker-compose.override.yml file and it was able to read my credentials

volumes:
  - ~/.aws/:/root/.aws:ro

I was deploying to a dot net core web application to an on prem server over IIS and had the same exact issue. No matter what I did the application would not recognize my credentials configured via AWS CLI (aws configure).

I ended up setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with my keys via the Windows environment variables and restarting the server.

The following article was very helpful in understanding the AWS SDK credential loading Client Factory https://www.stevejgordon.co.uk/credential-loading-and-the-aws-sdk-for-dotnet-deep-dive

In my case the config and credentials files were set up correctly in C:/Users//.aws folder so they should have been found by default. However, on a previous project I had set up different credentials (no longer valid) in C:/Users//AppData/Local/AWSToolkit referred to in AWS documentation as the AWS SDK Store. The SDK store is always checked first and then falls back to the default user credentials file. See the following: https://aws.amazon.com/blogs/developer/referencing-credentials-using-profiles/. The simplest solution in my case was simply to delete the files in the AWSToolkit folder. As an alternative I could have set up the SDK Store correctly.

I had the same problem in .NET core 5 with AWS and I solved it by :

what I had :

I had config and credentials files in C:\Users\.aws.

In StartUp.cs after initialized AWS options I added:

#if Debuge
   options.Profile="default";
   options.ProfileLocations="C:\\Users\\.aws\\credentials";
#endif

I had the same issue:

Amazon.Runtime.AmazonServiceException: 'Unable to get IAM security credentials from EC2 Instance Metadata Service.'

I was working with Dot Net Core Microservice, I got this error.

Solution - I removed the AWS credentials path which was mentioned in all the different setting files like appsettings.Debug.json and appsettings.Development.json.

This AWS credentials path should only be mentioned in the appsettings.json file. Remove it from all other files.