so one of my cloud functions suddenly crashes on (to my current understanding) random occasions.
The function is used as a postback-URL (POST!) for an API that we receive Data from. The service offering the API logs Errors that show that the Cloud Function responds with:
500 Server Error
Error: Server Error
The server encountered an error and could not complete your request.
Please try again in 30 seconds.
I appended a Screenshot ([1]) as well as a beautified version of the html ([2]) of the whole response to this question.
When this behaviour occurred, I was submitting a maximum of 350 requests to the API at once. The postback-calls are spread over several minutes (because the API takes longer for some input than for others), so it should not be a problem of rates/quotas/limits...
The Firebase Cloud function logs are completely empty regarding this behaviour (this is what confuses me the most). They only show the successful callbacks (same function, same data!). Some people on SO and other forums were pointing to the Google Cloud Logs, but I could not find any more information there than I did on the Firebase Cloud Functions Logs.
Thank you for your time and for any help you can provide!
My package.json:
{
"name": "functions",
"version": "1.0.0",
"scripts": {
"lint": "tslint --project tsconfig.json",
"build": "tsc",
"serve": "npm run build && firebase serve --only functions",
"shell": "npm run build && firebase functions:shell",
"start": "npm run shell",
"deploy": "firebase deploy --only functions",
"logs": "firebase functions:log"
},
"engines": {
"node": "8"
},
"main": "lib/index.js",
"dependencies": {
"esm": "^3.2.25",
"firebase-admin": "^8.6.0",
"firebase-functions": "^3.6.2",
"fs": "0.0.1-security",
"handlebars": "^4.7.6",
"nodemailer": "^6.4.8",
"request": "^2.88.2",
"request-promise": "^4.2.4",
"request-promise-native": "^1.0.7"
},
"devDependencies": {
"@firebase/app-types": "^0.6.0",
"@types/node": "^13.13.10",
"firebase-functions-test": "^0.1.6",
"prettier": "^1.19.1",
"tslint": "^5.12.0",
"typescript": "^3.9.5"
},
"private": true
}
The Cloud Function in a simplified version:
const scrapeCallbackDFS_v2 = async (request, response) => {
console.log('resultsCallbackDFS() called ...')
if (
request.method === 'POST' &&
request.body &&
request.body.status_code === 20000
) {
// extract data from request.url (queryparams)
// ...
// extract data from request.body
// ...
try {
const analysedResults = analyseResults(/* extracted data ... */)
await uploadResult( /* upload to firestore ... */ )
response
.set('Access-Control-Allow-Methods', 'POST, OPTIONS, GET')
.set('Access-Control-Allow-Origin', corsAllowOrigin(request))
.status(200)
.end()
return
} catch (error) {
console.log(userId + ' ' + error)
response
.status(433)
.set('Access-Control-Allow-Methods', 'POST, OPTIONS, GET')
.set('Access-Control-Allow-Origin', corsAllowOrigin(request))
.send(error)
return
}
} else {
response
.status(434)
.set('Access-Control-Allow-Methods', 'POST, OPTIONS, GET')
.set('Access-Control-Allow-Origin', corsAllowOrigin(request))
.send('method:' + request.method)
return
}
}
[1] The logs from the API on the callback-Requests:
[2] beautified Cloud Function Response body to the postback-Request:
"server response":"HTTP\/1.1 100 Continue
<br \/>
<br \/>HTTP\/1.1 500 Internal Server Error
<br \/>X-Cloud-Trace-Context: 95a8809b1fc9d9313ccc534fb0c61636
<br \/>Date: Mon, 08 Jun 2020 17:35:17 GMT
<br \/>Content-Type: text\/html; charset=UTF-8
<br \/>Server: Google Frontend
<br \/>Content-Length: 323
<br \/>Alt-Svc: h3-27=\":443\"; ma=2592000,h3-25=\":443\";
ma=2592000,h3-T050=\":443\"; ma=2592000,h3-Q050=\":443\";
ma=2592000,h3-Q049=\":443\"; ma=2592000,h3-Q048=\":443\";
ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\";
ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"
<br \/>
<br \/>
<br \/>\n
<html>
<head>
<br \/>\n
<meta http-equiv=\ "content-type\" content=\ "text\/html;charset=utf-8\">
<br \/>\n
<title>500 Server Error<\/title>
<br \/>\n
<\/head>
<br \/>\n
<body text=#000000 bgcolor=#ffffff>
<br \/>\n
<h1>Error: Server Error<\/h1>
<br \/>\n
<h2>The server encountered an error and could not complete your request.
<p>Please try again in 30 seconds.<\/h2>
<br \/>\n
<h2><\/h2>
<br \/>\n
<\/body>
<\/html>"