kubeflow pipeline SDK use dsl.ParallelFor to build a loop

user_18033516 0
Viewed 323

@dsl.pipeline( name='classfier') def classifiertest(): make_classification_com_res = make_classification_com() rng_res = np_random_random_state() uniform_res = rng_uniform(make_classification_com_res.output,rng_res.output) all_datas_res = get_all_datas(x_input=uniform_res.output,y_input=make_classification_com_res.output) forlist= list([0,1,2,3,4,5,6,7,8,9]) with dsl.ParallelFor(forlist) as item_index: for_outter_func(item_index,ds_input=all_datas_res.output)

When I run this pipeline, the following error occurs after clicking the start button of run：

{"error":"Failed to create a new run.: InternalServerError: Failed to store run classfier-9xbrk to table: Error 1366: Incorrect string value: '\xE8\xBF\x99\xE4\xB8\x80...' for column 'WorkflowRuntimeManifest' at row 1","code":13,"message":"Failed to create a new run.: InternalServerError: Failed to store run classfier-9xbrk to table: Error 1366: Incorrect string value: '\xE8\xBF\x99\xE4\xB8\x80...' for column 'WorkflowRuntimeManifest' at row 1","details":[{"@type":"type.googleapis.com/api.Error","error_message":"Internal Server Error","error_details":"Failed to create a new run.: InternalServerError: Failed to store run classfier-9xbrk to table: Error 1366: Incorrect string value: '\xE8\xBF\x99\xE4\xB8\x80...' for column 'WorkflowRuntimeManifest' at row 1"}]}

When I delete these two lines of code, pipeline can successfully commit and run. with dsl.ParallelFor(forlist) as item_index: for_outter_func(item_index,ds_input=all_datas_res.output)

kubeflow pipeline
1 Answers

Delete inline comments.
The following writing style is the cause of problems in kfp.

from kfp.components import create_component_from_func
# This comment is OK.
def inputdata_outputtable(input_arr, output_path):
    import numpy as np
    # This comment is also OK.
    _input_arr = np.array(input_arr) # This comment causes an error.
    np.save(output_path, _input_arr)
user_94269120
Related