using indexed jobs in google kubernetes engine

user_930071 0
Viewed 39

I'm trying to run docker image on multiple data using indexed jobs.

It seems that indexed jobs is available since Kubernetes 1.21, but my job on google kubernetes engine doesn't recognize the variable $JOB_COMPLETION_INDEX.

my yaml file looks like this:

spec:
  containers:
  - args:
    - --job_id="$JOB_COMPLETION_INDEX"

If I set job_id=0, it works ok.

Even if I add the following in the top level spec,

spec:
  completions: 3
  parallelism: 3
  completionMode: Indexed

yaml file automatically remove the part (I'm using google cloud web console for editing yaml file).

I tried both 1.22.3-gke.700 and 1.21.5-gke.1302 with Kubernetes alpha features, but nothing worked.

Is indexed job available now in google kubernetes engine?

kubernetes
1 Answers

where did you get args? Use command instead. It will be look something like this:

apiVersion: batch/v1
kind: Job
metadata:
  name: 'sample-job'
spec:
  completions: 3
  parallelism: 3
  completionMode: Indexed
  template:
    spec:
      restartPolicy: Never
      containers:
      - command:
        - 'bash'
        - '-c'
        - 'echo "My partition: ${JOB_COMPLETION_INDEX}"'
        image: 'docker.io/library/bash'
        name: 'sample-load'

It's from official documentation.

Or this example also works:

[...]
    spec:
      subdomain: my-job-svc
      containers:
      - name: task
        image: registry.example.com/processing-image
        command: ["./process",  "--index", "$JOB_COMPLETION_INDEX", "--hosts-pattern", "my-job-{{.id}}.my-job-svc"]
user_168605420
Related