I am getting the error below:
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 653 in stage 7.0 failed 4 times, most recent failure: Lost task 653.3 in stage 7.0 (TID 27294, ip-10-0-57-16.ec2.internal, executor 34): ExecutorLostFailure (executor 34 exited caused by one of the running tasks) Reason: Container marked as failed: container_1602898457220_0001_01_000370 on host: ip-10-0-57-16.ec2.internal. Exit status: 143. Diagnostics: Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
Killed by external signal
My data set is 80GB Operation i did is create some square, interaction features, so maybe double the number of columns.
I am using 20 m4.16xlarge (64CPU, 256GB, https://aws.amazon.com/ec2/instance-types/) instance spark.yarn.executor.memoryOverhead = '16384'
can i do something to fix this? and why the I got OOM error even my dataset is way smaller than my instance number.