How to pass result of script template as an input parameter to another task in dag in argo workflows

user_13846745 0
Viewed 917

I created a WorkflowTemplate in which I want to pass result of a script template as an input parameter to another task

Here is my WorkflowTemplate

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: dag-wft
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    inputs:
      parameters:
        - name: message
          default: '["tests/hello", "templates/hello", "manifests/hello"]'
    dag:
      tasks:
        - name: prepare-lst
          template: prepare-list-script
          arguments:
            parameters:
              - name: message
                value: "{{inputs.parameters.message}}"

        - name: templates
          depends: prepare-lst
          templateRef:
            name: final-dag-wft
            template: whalesay-final
          arguments:
            parameters:
            - name: fnl_message
              value: "{{item}}"
          withParam: "{{tasks.prepare-lst.outputs.parameters.templates_lst}}"

        - name: manifests
          depends: prepare-lst && (templates.Succeeded || templates.Skipped)
          templateRef:
            name: final-dag-wft
            template: whalesay-final
          arguments:
            parameters:
            - name: fnl_message
              value: "{{item}}"
          withParam: "{{tasks.prepare-lst.outputs.parameters.manifests_lst}}"

  - name: prepare-list-script
    inputs:
      parameters:
        - name: message
    script:
      image: python
      command: [python]
      source: |
        manifests_lst = []
        # templates list preparation
        templates_lst = ['templates/' for template in "{{inputs.parameters.message}}" if 'templates/' in template]
        
        print(templates_lst)

        # manifests list preparation
        for i in "{{inputs.parameters.message}}":
          if 'templates/' not in i:
            manifests_lst.append(i)

        print(manifests_lst)

    outputs:
      parameters:
        - name: templates_lst
        - name: manifests_lst

In the above script template I've added print statement of two variables templates_lst and manifests_lst. I want to pass these two variables result as in input to two other tasks in the dag. The two other tasks are templates and manifests

The way I am accessing the output values is "{{tasks.prepare-lst.outputs.parameters.templates_lst}}" and "{{tasks.prepare-lst.outputs.parameters.manifests_lst}}". It is not working

How we can I do this?

argo-workflows kubernetes
1 Answers

1. Fully define your output parameters

Your output parameter spec is incomplete. You need to specify where the output parameter comes from.

Since you have multiple output parameters, you can't just use standard out ({{tasks.prepare-lst.outputs.parameters.result}}). You have to write two files and derive an output parameter from each.

2. Load the JSON array so it's iterable

If you iterate over the string representation of the array, you'll just get one character at a time.

3. Use an environment variable to pass input to Python

Although it's not strictly necessary, I consider it best practice. If a malicious actor had the ability to set the message parameter, they could inject Python into your workflow. Pass the parameter as an environment variable so the string remains a string.

Changes:

   - name: prepare-list-script
     inputs:
       parameters:
         - name: message
     script:
       image: python
       command: [python]
+      env:
+      - name: MESSAGE
+        value: "{{inputs.parameters.message}}"
       source: |
+        import json
+        import os
+        message = json.loads(os.environ["MESSAGE"])
         manifests_lst = []
         # templates list preparation
-        templates_lst = ['templates/' for template in "{{inputs.parameters.message}}" if 'templates/' in template]
+        templates_lst = ['templates/' for template in message if 'templates/' in template]
         
-        print(templates_lst)
+        with open('/mnt/out/templates_lst.txt', 'w') as outfile:
+          outfile.write(str(json.dumps(templates_lst)))
 
         # manifests list preparation
         for i in "{{inputs.parameters.message}}":
           if 'templates/' not in i:
             manifests_lst.append(i)
 
-        print(manifests_lst)
+        with open('/mnt/out/manifests_lst.txt', 'w') as outfile:
+          outfile.write(str(json.dumps(manifests_lst)))
+      volumeMounts:
+      - name: out
+        mountPath: /mnt/out
+    volumes:
+    - name: out
+      emptyDir: { }
 
     outputs:
       parameters:
         - name: templates_lst
+          valueFrom:
+            path: /mnt/out/templates_lst.txt
         - name: manifests_lst
+          valueFrom:
+            path: /mnt/out/manifests_lst.txt
user_6847760
Related