How to normalize a private key stored on AWS secrets manager

Another option would be to base64 encode the PEM for storage:

Encode the key:

$ cat private_key 
-----BEGIN RSA PRIVATE KEY-----
asdkmnasefkljzsdkffjsldkgfjlzkmsdflkNOTAREALKEYasddkjnsfdlzxdfvlkmdggo=
-----END RSA PRIVATE KEY-----
$ base64 private_key > encoded_private_key

$ cat encoded_private_key
LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQphc2RrbW5hc2Vma2xqenNka2ZmanNsZGtnZmpsemttc2RmbGtOT1RBUkVBTEtFWWFzZGRram5zZmRsenhkZnZsa21kZ2dvPQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=

Get the key back:

$ base64 -D encoded_private_key
-----BEGIN RSA PRIVATE KEY-----
asdkmnasefkljzsdkffjsldkgfjlzkmsdflkNOTAREALKEYasddkjnsfdlzxdfvlkmdggo=
-----END RSA PRIVATE KEY-----

Edit: Assuming the secret is base64 encoded, this would work:

Encode and push:

aws secretsmanager create-secret --name my-private-key --secret-string `base64 private.pem`

Pull and decode:

aws secretsmanager get-secret-value --secret-id my-private-key --query 'SecretString' --output text |base64 -D > private.pem

Doing the --query --output text thing might make it simpler to parse even if you don't want to base64 encode it as well.

I came up with a solution that leveraged storing a secret in secrets manager as plain text.

1. Store the secret in secrets manager as plain text. They console will have JSON brackets but I removed those.

2. Use the cli to get the secret output as plain text. Now the \n and \s in the text will be converted to the line breaks and spaces they're supposed to be

   aws secretsmanager get-secret-value --secret-id privatekey --query 
   'SecretString' --output text > private.pem
   

The pem file will now be properly formatted

    -----BEGIN RSA PRIVATE KEY-----
    MIIG3DCCBM
    -----END RSA PRIVATE KEY-----

You need to Pipe (|) the output through a few steps

1. To return just the value of the key from the json use jq ".SecretString"
2. To format the public key use cut -b 2- |tr -d '"' |sed -En "s/\\\n/\n/pg"

This will return what you want to to.

Also note that you will want to make private.pem read only. (chmod 400 private.pem)

In summery the full command will look like:

aws secretsmanager get-secret-value --secret-id my-private-key | jq ".SecretString" |cut -b 2- |tr -d '"' |sed -En "s/\\\n/\n/pg" > private.pem

Recently faced similar "issue".

Brief description

• We generated usual public | private {file-name}.pem keys which had format as:

---BEGIN RSA ... KEY---
...
---END RSA ... KEY---

• We stored those public | private {file-name}.pem keys in AWS SSM (parameter store) service using SecureString value type.
• Afterwards we had to fetch those keys using boto3.

Resolution

--> This piece of code will return you base64 ENcoded representation of your key

from boto3 import client
 
parameter_name = '/ssm/parameter/name/to/fetch'
key = client('ssm').get_parameter(Name=parameter_name).get('Parameter', {}).get('Value', '')

--> This piece of code will return you base64 DEcoded representation of your key

from boto3 import client
 
parameter_name = '/ssm/parameter/name/to/fetch'
key = client('ssm').get_parameter(Name=parameter_name, WithDecryption=True).get('Parameter', {}).get('Value', '')

So, make sure to employ additional parameter as WithDecryption=True within client('ssm').get_parameter() which will automatically solve the decoding of the string.

Reference:

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameter

Inspired from solution of Jason Steele.

I know that the Issue is raised for AWS-CLI, I have faced similar issue while retrieving the information in Java.

Solution:

While configuring the public/ private key in AWS console, decode the entire key content with Base64 ( You can also use Notepad++ )

While retrieving the data, decode and get it. It resolves the issue. PFB the java code.

secret = getSecretValueResult.getSecretString(); // gets the entire secret object
Object obj=JSONValue.parse(secret); 
        
        //creating an object of JSONObject class and casting the object into JSONObject type  
        JSONObject jsonObject = (JSONObject) obj;   

        //getting values form the JSONObject and casting that values into corresponding types  
        String vendorPublicKey =  (String) jsonObject.get("vendorPublicKey");
  String decodedKey = vendorPublicKey != null ? new String(Base64.getDecoder().decode(vendorPublicKey)) : "";

decodedKey will have the public/private key in valid format.