How do I use Argo Workflows Using Previous Step Outputs As Inputs?

user_3877654 0
Viewed 1041

I am trying to format my workflow per these instructions (https://argoproj.github.io/argo-workflows/workflow-inputs/#using-previous-step-outputs-as-inputs) but cannot seem to get it right. Specifically, I am trying to imitate "Using Previous Step Outputs As Inputs"

I have included my workflow below. In this version, I have added a path to the inputs.artifacts because the error requests one. The error I am now receiving is:

ATA[2022-02-28T14:14:45.933Z] Failed to submit workflow: templates.entrypoint.tasks.print1 templates.print1.inputs.artifacts.result.from not valid in inputs

Can someone please tell me how to correct this workflow so that it works?

---
{
   "apiVersion": "argoproj.io/v1alpha1",
   "kind": "Workflow",
   "metadata": {
      "annotations": {
         "workflows.argoproj.io/description": "Building from the ground up",
         "workflows.argoproj.io/version": ">= 3.1.0"
      },
      "labels": {
         "workflows.argoproj.io/archive-strategy": "false"
      },
      "name": "data-passing",
      "namespace": "sandbox"
   },
   "spec": {
      "artifactRepositoryRef": {
         "configMap": "my-config",
         "key": "data"
      },
      "entrypoint": "entrypoint",
      "nodeSelector": {
         "kubernetes.io/os": "linux"
      },
      "parallelism": 3,
      "securityContext": {
         "fsGroup": 2000,
         "fsGroupChangePolicy": "OnRootMismatch",
         "runAsGroup": 3000,
         "runAsNonRoot": true,
         "runAsUser": 1000
      },
      "templates": [
         {
            "container": {
               "args": [
                  "Hello World"
               ],
               "command": [
                  "cowsay"
               ],
               "image": "docker/whalesay:latest",
               "imagePullPolicy": "IfNotPresent"
            },
            "name": "whalesay",
            "outputs": {
               "artifacts": [
                  {
                     "name": "msg",
                     "path": "/tmp/raw"
                  }
               ]
            },
            "securityContext": {
               "fsGroup": 2000,
               "fsGroupChangePolicy": "OnRootMismatch",
               "runAsGroup": 3000,
               "runAsNonRoot": true,
               "runAsUser": 1000
            }
         },
         {
            "inputs": {
               "artifacts": [
                  {
                     "from": "{{tasks.whalesay.outputs.artifacts.msg}}",
                     "name": "result",
                     "path": "/tmp/raw"
                  }
               ]
            },
            "name": "print1",
            "script": {
               "command": [
                  "python"
               ],
               "image": "python:alpine3.6",
               "imagePullPolicy": "IfNotPresent",
               "source": "cat {{inputs.artifacts.result}}\n"
            },
            "securityContext": {
               "fsGroup": 2000,
               "fsGroupChangePolicy": "OnRootMismatch",
               "runAsGroup": 3000,
               "runAsNonRoot": true,
               "runAsUser": 1000
            }
         },
         {
            "dag": {
               "tasks": [
                  {
                     "name": "whalesay",
                     "template": "whalesay"
                  },
                  {
                     "arguments": {
                        "artifacts": [
                           {
                              "from": "{{tasks.whalesay.outputs.artifacts.msg}}",
                              "name": "result",
                              "path": "/tmp/raw"
                           }
                        ]
                     },
                     "dependencies": [
                        "whalesay"
                     ],
                     "name": "print1",
                     "template": "print1"
                  }
               ]
            },
            "name": "entrypoint"
         }
      ]
   }
}
...
argo-workflows
2 Answers

In the artifact argument of print1, you should only put name and from parameters

E.g:

- name: print1
  arguments:
    artifacts: [{name: results, from: "{{tasks.whalesay.outputs.artifacts.msg}}"}]

and then in your template declaration, you should put name and path in your artifact input, as follows:

- name: input1
  inputs:
    artifacts:
    - name: result
      path: /tmp/raw
...

This works because in the argument of you task (in the dag declaration) you tell the program how you want that input to be called and from where to extract it, and in the template declaration you receive the input from name and tell the program where to place it temporarily. (This is what I understand in my own words)

Another problem I see is in print1 instead of printing to stdout or using sys to run the cat command, you run cat directly, this (I think) is not posible.

You should instead do something like

import sys

sys.stdout.write("{{inputs.artifacts.result}}\n")

or

import os

os.system("cat {{inputs.artifacts.result}}\n")
user_88525310
user_38776540
Related