How to define component/step using training operators such as TFJob in kubeflow pipeline

user_929769 0
Viewed 151

I know there is a way to use tfjob operator via kubectl, like the example at here (https://www.kubeflow.org/docs/components/training/tftraining/):

kubectl create -f https://raw.githubusercontent.com/kubeflow/training-operator/master/examples/tensorflow/simple.yaml

But I don't know how to incorporate in kubeflow pipeline. A normal component/job is defined via @component decoration or ContainerOp is a Kubernetes Job kind which runs in a Pod, but I don't know how to define a component with special training operator such as TFJob, so that my code runs as

apiVersion: "kubeflow.org/v1"
kind: TFJob

rather than:

apiVersion: "kubeflow.org/v1"
kind: Job

in kubernetes.

P.S.: there is a example here: https://github.com/kubeflow/pipelines/blob/master/components/kubeflow/launcher/sample.py but don't see anywhere specify TFJob

kubeflow-pipelines kubeflow
1 Answers

The example you reference leverages some code that actually creates a TFJob (look at the folder of your example):

The general question you raised is still subject to current discussions. Using tfjob_launcher_op appears to be the currently recommended way. Instead, some people also natively use ResourceOps to simulate your kubectl create call.

user_26250960
Related