Firestore fetch gives an Unauthenticated error in Lambda, works fine locally

Viewed 18

I have a new serverless back-end to a game that is migrating away from Firestore. When the app starts, it fetches player info from the database. If the player doesn't exist in the new database, the backend will check Firestore. Locally, this runs fine. Serverless Framework loads fsClientEmail and fsPrivateKey from a local json file as environment variables when running locally, and from Parameter Store when running in AWS.

model.js

const rowResult = await db.fetchEntity(tableNamePlayer, primaryKey, playerId)
if (rowResult) {
    this.fromMap(rowResult)
    return
}
// not found: check for legacy Firestore record.  If found, move to current storage
console.log('player not found in new db, checking firestore for legacy')
const legacyDoc = await firestore.fetchPlayer(playerId)

firestore.js

const {Firestore} = require('@google-cloud/firestore');

const obfCE = process.env.FIRESTORE_CLIENTEMAIL.substring(0,3) + "*".repeat(process.env.FIRESTORE_CLIENTEMAIL.length-3)
const obfPK = process.env.FIRESTORE_PRIVATEKEY.substring(0,3) + "*".repeat(process.env.FIRESTORE_PRIVATEKEY.length-3)
console.log(`Creating fs object with (${obfCE}), (${obfPK})`)
const firestore = new Firestore({
    projectId: process.env.FIRESTORE_PROJECT,
    credentials: {
        client_email: process.env.FIRESTORE_CLIENTEMAIL,
        private_key: process.env.FIRESTORE_PRIVATEKEY,
    }})

exports.fetchPlayer = async (playerId) => {
    const document = firestore.doc(`players/${playerId}`);

    try {
        let player = await (await document.get()).data()
        return player
    }
    catch (ex) {
        console.log(`fs: fetch player for ${playerId} - Exception thrown: ${ex}`)
    }

    return null
}

Again, runs fine locally, but running in AWS, I get the following in my CloudWatch logs:

2022-09-24T01:13:02.462Z    undefined   INFO    Creating fs object with (rf-****************************************************), (---***********************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************) 
2022-09-24T01:13:03.006Z    8ab7da9e-8108-40a2-9dc2-9520e762a406    INFO    player not found in new db, checking firestore for legacy
2022-09-24T01:13:10.565Z    8ab7da9e-8108-40a2-9dc2-9520e762a406    INFO    fs: fetch player for [snipped - player id] - Exception thrown: Error: 16 UNAUTHENTICATED: Failed to retrieve auth metadata with error: > error:0909006C:PEM routines:get_name:no start line

The first time I ran this and got this error, it was easy to spot that I had a leading double quote (copy-paste error) in the client email. I've triple-checked everything, removed keys and recreated them via both console and cli, added all kinds of checks (like showing those obfuscated keys), but a) I'm not entirely sure what "get_name:no start line" really means, and b) I don't see any other glaring typos.

A quick cli get-parameter on the key (the part which seems to have the most questions about) gives:

{
    "Parameter": {
        "Name": "/rf-api/prod/fsPrivateKey",
        "Type": "SecureString",
        "Value": "-----BEGIN PRIVATE KEY-----\\nMI<snip>MG+\\n-----END PRIVATE KEY-----\\n",
        "Version": 1,
        "LastModifiedDate": "2022-09-20T09:32:38.255000-05:00",
        "ARN": "<snip>",
        "DataType": "text"
    }
}
1 Answers

Not a great answer, but the problem was, in fact, those extra three characters. Instead of taking \n as a newline, it was taking it as a backslash and an n. Pasting in the key via the console, or sending it via cli did not make a difference. The only way I found to get past it was to a) manually edit the parameter entry via console after it had been created, deleting the backslash n characters and just hitting the enter key (doesn't matter how it was created), or b) if you are entering it in the console, you could paste the three lines as three separate pastes and hit enter between. Either way, SSM apparently does not handle \n as a newline.

Related