error Command failed with signal "SIGKILL" on fargate

Viewed 297

I have a fargate cluster running a node.js API, running on fargate 1.4.0 I have maybe 8-25 instances running depending on the load. Instances are defined with these parameters using aws CDK:

        cpu: 512,
        assignPublicIp: true,
        memoryLimitMiB: 2048,
        publicLoadBalancer: true,

Like few times a day I get error like this: error Command failed with signal "SIGKILL".

I though I was running out of memory, so I've configured node, to start with less memory like this: NODE_OPTIONS=--max_old_space_size=900 This made it less likely to occur, but I am still getting some SIGKILLs. When looking at the instances at runtime I see they have plenty of memory free on the OS level:

{
  "freemem": "6.95GB",
  "totalmem": "7.79GB",
  "max_old_space_size": 813.1680679321289,
  "processUptime": "46m",
  "osUptime": "49m",
  "rssMemory": "396.89MB"
}

Why is fargate still killing those instances? Is there a way to find out most memory hungry processes just before the SIGKILL?

0 Answers
Related