I am trying to use a kubeflow run parameter as an argument for my pipeline step.
Every time I compile the yaml file however it gets changed from an Integer to a LocalPath.
@dsl.pipeline(name='First Pipeline', description='generates a random set of numbers then performs operations on them returning a json object')
def first_pipeline(generate_n_arg: int = 10):
# Loads the yaml manifest for each component
generate_numbers = kfp.components.load_component_from_file('generate_numbers/generate_numbers.yaml')
multiply_numbers = kfp.components.load_component_from_file('multiply/multiply.yaml')
sum_numbers = kfp.components.load_component_from_file('sum/sum.yaml')
sort_numbers = kfp.components.load_component_from_file('sort/sort.yaml')
output_to_json = kfp.components.load_component_from_file('output_to_json/output_to_json.yaml')
# Run generate_data task
print(generate_n_arg)
generate_numbers_task = generate_numbers(generate_n=generate_n_arg)
my yaml file clearly dictates that it needs to be an integer
description: generate random toy data
inputs:
- {name: generate_n, type: Integer, description: 'an integer for how many numbers to generate'}
outputs:
- {name: out_path, type: LocalPath, description: 'Path where data will be stored.'}
implementation:
container:
image: path/to/image
command: [
python, generate.py,
--generate_n,
{inputPath: generate_n},
--out_path,
{outputPath: out_path},
]
however every time I compile the pipeline it gets changed to a path
command: [python, generate.py, --generate_n, /tmp/inputs/generate_n/data, --out_path,
/tmp/outputs/out_path/data]
even though the pipeline still acknowledges that I input an integer multiple times in the yaml
inputs:
artifacts:
- {name: generate_n_arg, path: /tmp/inputs/generate_n/data}
arguments:
parameters:
- {name: generate_n_arg, value: '10'}
artifacts:
- name: generate_n_arg
raw: {data: '{{workflow.parameters.generate_n_arg}}'}
Does anyone know how I can make it give me the input value instead of an localpath/artifact path?