Description

For example:

from typing import Optional

from kfp import compiler

from kfp import dsl
from kfp import kubernetes
from kfp.kubernetes import add_toleration_json


@dsl.component()
def some_component():
    print("mounted")


@dsl.pipeline
def my_pipeline(train_tolerations: Optional[list] = None, train_tolerations_dict: Optional[dict] = None):
    task = some_component()
    task.set_caching_options(enable_caching=False)


    add_toleration_json(task, train_tolerations)
    add_toleration_json(task, train_tolerations_dict)

Expected behavior

For kfp-kubernetes options applied on the task, the behavior should be as follows:

1. If you set None the input (like in the example above):
   a. and if the user provides no value, we do not apply that kubernetes resource option on the associated Launcher Pod
   b. if the user does provide a value, this value is used
2. If you provide a default in the pipeline SDK:
   a. if the user provides their input when starting the run (runtime configuration), this takes precedence
   b. if the user does not provide new input, then the default is used

Note that when you do not provide a field in the UI, it is simply not present in runtime configuraiton, for example if both parameters are not provided, you will get a runtime configuration like so:

  "runtime_config": {
    "parameters": {}
  }

And you will see an error like:

F1001 18:24:06.918897      22 main.go:104] KFP driver: driver.Container(pipelineName=my-pipeline, runID=c398b4ea-ef29-4996-af95-df90e2b34b48, taskDisplayName="some-component", taskName="some-component", component="comp-some-component", dagExecutionID=274, componentSpec, KubernetesExecutorConfig) failed: failed to resolve toleration: the resolved input parameter is null: train_tolerations

In such a case we would defer to the 1-a case provided above.

Labels

/area backend
/area sdk

Impacted by this bug? Give it a 👍.