From a321dbb1302e523ba70eaefcc812441ff53ae028 Mon Sep 17 00:00:00 2001 From: Peter Amstutz Date: Fri, 17 May 2024 06:44:26 -0400 Subject: [PATCH 1/5] PR to add "secret" field to input parameters. --- CommandLineTool.yml | 4 ++++ Process.yml | 55 +++++++++++++++++++++++++++++++++++++++++++++ Workflow.yml | 5 +++++ 3 files changed, 64 insertions(+) diff --git a/CommandLineTool.yml b/CommandLineTool.yml index 604c668..ce15754 100644 --- a/CommandLineTool.yml +++ b/CommandLineTool.yml @@ -51,6 +51,10 @@ $graph: ## Changelog for v1.3.0-dev1 + * Added `secret` option on [input parameters](#InputParameter) + to request special handling of secrets such as passwords and + API tokens. + See also the [CWL Workflow Description, v1.3.0-dev1 changelog](Workflow.html#Changelog). For other changes since CWL v1.0, see the [CWL Command Line Tool Description, v1.1 changelog](https://www.commonwl.org/v1.1/CommandLineTool.html#Changelog) diff --git a/Process.yml b/Process.yml index 3a1f3a2..f7b03ac 100644 --- a/Process.yml +++ b/Process.yml @@ -385,6 +385,61 @@ $graph: from the input object, or if the value of the parameter in the input object is `null`. Default values are applied before evaluating expressions (e.g. dependent `valueFrom` fields). + - name: secret + type: ["null", boolean, string] + doc: | + Indicates this input parameter value is sensitive. + Implementations should apply special handling to secret values + to avoid displaying them in logs, including them in output, or + otherwise making them visible or accessible in any way beyond + what is required to make the value of the secret input + parameter available to workflow processes that need it. + + This feature is intended to provide a safer way to handle + credentials such as passwords and API tokens. + + Possible values of the `secret` field can be: + + * null or not provided (default, the input parameter is not secret) + * false (same as null) + * true (parameter is secret) + * a non-empty string (parameter is secret, and may be looked up in platform storage) + + If the value of `secret` is a string, this is a lookup key to + be used to fetch a secret value from the workflow platform + secret store. This assumes a model where a non-sensitive + lookup key is passed to the secret store and a sensitive + string value (the password, API token, etc) is returned. + + The format of this lookup key, as well as management, access + permissions, and authentication for the secret store are + implementation specific and out of scope for this document. + + If the input parameter is a secret, the `type` of the input + parameter must only consist of `string`, `array`, or + `null`. + + If `secret` is a string and the platform supports looking up + credentials, the input parameter is implicitly optional for + the caller, and platform should look up the secret to fill in + the input parameter value when not provided by the caller. + + An explict value provided by the caller always takes + precidence over looking up a value, i.e. checking the secret + store must only happen if the caller did not provide an + explicit value for this secret parameter, or the value is + null. + + If the platform does not support secrets lookup, a string + value of `secret` is treated like boolean true, indicating the + parameter is secret, but must be provided in the input + document (unless marked as optional). + + Failure to look up the secret (for example, due to denial of + access) may yield a value of "null". Execution continues only + if the parameter is optional (i.e. "null" is an accepted + type). However implementations may, at user option, treat + failure to look up a secret as a fatal error. - name: OutputParameter diff --git a/Workflow.yml b/Workflow.yml index 176de9a..5f94063 100644 --- a/Workflow.yml +++ b/Workflow.yml @@ -40,6 +40,11 @@ $graph: CWL group. ## Changelog + + * Added `secret` option on [input parameters](#InputParameter) + to request special handling of secrets such as passwords and + API tokens. + See also the [CWL Command Line Tool Description, v1.3.0-dev1 changelog](CommandLineTool.html#Changelog). For other changes since CWL v1.0, see the [CWL Workflow Description, v1.1 changelog](https://www.commonwl.org/v1.1/Workflow.html#Changelog) From 36ab5e28e00fa7ce50518362e422e9133e4e1166 Mon Sep 17 00:00:00 2001 From: Peter Amstutz Date: Fri, 17 May 2024 06:49:35 -0400 Subject: [PATCH 2/5] Add text about checking correctness for secrets. --- Process.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Process.yml b/Process.yml index f7b03ac..b3fe076 100644 --- a/Process.yml +++ b/Process.yml @@ -441,6 +441,13 @@ $graph: type). However implementations may, at user option, treat failure to look up a secret as a fatal error. + Workflow engines should verify that, when a secret is passed + through multiple levels (e.g. from a parent workflow to a + sub-workflow to a command line tool), it is secret at every + level, for example through a check that secret inputs are only + be passed to workflow steps where the inputs are also marked + as secret, or that secret inputs to a workflow step are + themselves secret at the parent workflow level. - name: OutputParameter type: record From 02abe0c0880184da6c056feb3e4511a196884698 Mon Sep 17 00:00:00 2001 From: Peter Amstutz Date: Fri, 17 May 2024 06:54:10 -0400 Subject: [PATCH 3/5] Fix typos --- Process.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Process.yml b/Process.yml index b3fe076..8780f9c 100644 --- a/Process.yml +++ b/Process.yml @@ -424,8 +424,8 @@ $graph: the caller, and platform should look up the secret to fill in the input parameter value when not provided by the caller. - An explict value provided by the caller always takes - precidence over looking up a value, i.e. checking the secret + An explicit value provided by the caller always takes + precedence over looking up a value, i.e. checking the secret store must only happen if the caller did not provide an explicit value for this secret parameter, or the value is null. From a579af74c3a73832c0e075cd6b9cae8f61a31d56 Mon Sep 17 00:00:00 2001 From: Peter Amstutz Date: Fri, 17 May 2024 12:05:24 -0400 Subject: [PATCH 4/5] New design (3rd one today) with explicit SecretText object type. --- Base.yml | 59 ++++++++++++++++++++++++++++++++++++++++++++++++ Process.yml | 64 +---------------------------------------------------- 2 files changed, 60 insertions(+), 63 deletions(-) diff --git a/Base.yml b/Base.yml index c2ca0d5..1c4825b 100644 --- a/Base.yml +++ b/Base.yml @@ -458,6 +458,64 @@ $graph: jsonldPredicate: _id: "cwl:listing" +- name: SecretText + type: record + doc: | + An input parameter value that is confidential. This feature is + intended for values such as passwords, API tokens, or encryption + keys. + + When a parameter reference refers to a SecretText object, it must + be replaced with the object's value of `plaintext`. + fields: + - name: class + type: + type: enum + name: SecretText_class + symbols: + - cwl:SecretText + jsonldPredicate: + _id: "@type" + _type: "@vocab" + doc: Indicates this object represents a secret. + - name: secretTextId + type: string? + doc: | + This is an identifier to be used to fetch a secret value from + the workflow platform secrets management service. This assumes a + key-value model where the secret identifier is passed to the + secret store and a confidential string value (the password, + API token, encryption key, etc) is returned. + + If `plaintext` is null and `secretTextId` is non-null, the + value of `plaintext` should be set by looking up the value of + `secretTextId` on the workflow platform's secrets management + service. + + The format of the secret identifier, as well as protocol, + service endpoint, key management, access permissions, and + authentication for the secrets management service are + implementation specific and out of scope for this document. + + The `secretTextId` field is optional. If `secretTextId` is + not provided, `plaintext` must be non-null. + + If the platform does not offer secrets management, or there is + a failure when retrieving the value of a secret, and + `plaintext` is null, it is a fatal error. + + - name: plaintext + type: string? + doc: | + Contains the secret value. + + Implementations should apply special handling to confidential + values to avoid displaying them in logs, including them in + output, or otherwise making them visible or accessible in any + way that has potential to leak beyond what is required to make + the value of the confidential input parameter available to + workflow processes that need it. + - name: CWLObjectType type: union @@ -470,6 +528,7 @@ $graph: - string - File - Directory + - SecretText - type: array items: - "null" diff --git a/Process.yml b/Process.yml index 8780f9c..3a1f3a2 100644 --- a/Process.yml +++ b/Process.yml @@ -385,69 +385,7 @@ $graph: from the input object, or if the value of the parameter in the input object is `null`. Default values are applied before evaluating expressions (e.g. dependent `valueFrom` fields). - - name: secret - type: ["null", boolean, string] - doc: | - Indicates this input parameter value is sensitive. - Implementations should apply special handling to secret values - to avoid displaying them in logs, including them in output, or - otherwise making them visible or accessible in any way beyond - what is required to make the value of the secret input - parameter available to workflow processes that need it. - - This feature is intended to provide a safer way to handle - credentials such as passwords and API tokens. - - Possible values of the `secret` field can be: - - * null or not provided (default, the input parameter is not secret) - * false (same as null) - * true (parameter is secret) - * a non-empty string (parameter is secret, and may be looked up in platform storage) - - If the value of `secret` is a string, this is a lookup key to - be used to fetch a secret value from the workflow platform - secret store. This assumes a model where a non-sensitive - lookup key is passed to the secret store and a sensitive - string value (the password, API token, etc) is returned. - - The format of this lookup key, as well as management, access - permissions, and authentication for the secret store are - implementation specific and out of scope for this document. - - If the input parameter is a secret, the `type` of the input - parameter must only consist of `string`, `array`, or - `null`. - - If `secret` is a string and the platform supports looking up - credentials, the input parameter is implicitly optional for - the caller, and platform should look up the secret to fill in - the input parameter value when not provided by the caller. - - An explicit value provided by the caller always takes - precedence over looking up a value, i.e. checking the secret - store must only happen if the caller did not provide an - explicit value for this secret parameter, or the value is - null. - - If the platform does not support secrets lookup, a string - value of `secret` is treated like boolean true, indicating the - parameter is secret, but must be provided in the input - document (unless marked as optional). - - Failure to look up the secret (for example, due to denial of - access) may yield a value of "null". Execution continues only - if the parameter is optional (i.e. "null" is an accepted - type). However implementations may, at user option, treat - failure to look up a secret as a fatal error. - - Workflow engines should verify that, when a secret is passed - through multiple levels (e.g. from a parent workflow to a - sub-workflow to a command line tool), it is secret at every - level, for example through a check that secret inputs are only - be passed to workflow steps where the inputs are also marked - as secret, or that secret inputs to a workflow step are - themselves secret at the parent workflow level. + - name: OutputParameter type: record From 9be8fad1378a7e6bc35e31ec94ad19cac8d3b934 Mon Sep 17 00:00:00 2001 From: Peter Amstutz Date: Fri, 17 May 2024 12:09:34 -0400 Subject: [PATCH 5/5] Update changelog --- CommandLineTool.yml | 7 ++++--- Workflow.yml | 7 ++++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/CommandLineTool.yml b/CommandLineTool.yml index ce15754..dcf94be 100644 --- a/CommandLineTool.yml +++ b/CommandLineTool.yml @@ -51,9 +51,10 @@ $graph: ## Changelog for v1.3.0-dev1 - * Added `secret` option on [input parameters](#InputParameter) - to request special handling of secrets such as passwords and - API tokens. + * Added [SecretText](#SecretText) object type to represent input + types that should be kept confidential by the workflow runner, + such as passwords. SecretText can also be used to refer to + credentials storage in a platform secrets management server. See also the [CWL Workflow Description, v1.3.0-dev1 changelog](Workflow.html#Changelog). For other changes since CWL v1.0, see the diff --git a/Workflow.yml b/Workflow.yml index 5f94063..25abf8e 100644 --- a/Workflow.yml +++ b/Workflow.yml @@ -41,9 +41,10 @@ $graph: ## Changelog - * Added `secret` option on [input parameters](#InputParameter) - to request special handling of secrets such as passwords and - API tokens. + * Added [SecretText](#SecretText) object type to represent input + types that should be kept confidential by the workflow runner, + such as passwords. SecretText can also be used to refer to + credentials storage in a platform secrets management server. See also the [CWL Command Line Tool Description, v1.3.0-dev1 changelog](CommandLineTool.html#Changelog). For other changes since CWL v1.0, see the