Is it possible to define the cpu and memory limit for individual pipeline in flink?

user_15010774 0
Viewed 29

Is it possible to define the maximum amount of cpu and memory pipeline can use while running it?

I am running below mvn command to submit job to flink:

    sudo mvn clean compile exec:java -Dexec.mainClass=pipeline_main_class -Pflink-runner -Dexec.args=" \
--runner=FlinkRunner \
--project=gcp_project \
--flinkMaster=flink_url:8081 \
--streaming=true \
--input=input_bucket \
--output=output_bucket"
apache-flink
1 Answers

What you can control is how many slots each job uses. But to achieve the level of control you are asking about, the best solution is to use per-job (or per-application) clusters, rather than a shared session cluster.

user_20008230
Related