Problem
The cohort-method shell accepts ACP recommendations from the free-text analytic-parameters step and converts them into effective_analytic_settings, but there is no dedicated validation layer between the ACP response and downstream script generation.
This means malformed, incomplete, or semantically inconsistent ACP output can still flow into:
analysis-settings/cm_analytic_settings_recommendation.json
effective_analytic_settings
- generated
06_cm_spec.R
- generated
analysisSpecification.json
Current behavior
- ACP recommendations are converted through
shell_settings_from_acp_recommendation(...)
- If ACP returns
source = acp_flow and status = ok, the mapped settings are used downstream
- There is no explicit schema/semantic validation pass that confirms:
- required settings are present
- values are in supported ranges
- combinations are coherent
- unsupported fields are surfaced clearly
Risk
- invalid generated scripts
- silent fallback to partial defaults
- hard-to-debug user experience when ACP output is plausible-looking but not executable
Requested change
Add a validation layer for cohort-method analytic recommendations before they are committed to shell state or used for script generation.
Acceptance criteria
- ACP recommendation objects are validated before conversion/use
- validation distinguishes:
- schema errors
- unsupported fields
- semantically inconsistent settings
- the shell surfaces clear actionable messages when validation fails
- invalid recommendations do not silently produce executable-looking output
- tests cover valid, partially valid, and invalid ACP recommendation payloads
Problem
The cohort-method shell accepts ACP recommendations from the free-text analytic-parameters step and converts them into
effective_analytic_settings, but there is no dedicated validation layer between the ACP response and downstream script generation.This means malformed, incomplete, or semantically inconsistent ACP output can still flow into:
analysis-settings/cm_analytic_settings_recommendation.jsoneffective_analytic_settings06_cm_spec.RanalysisSpecification.jsonCurrent behavior
shell_settings_from_acp_recommendation(...)source = acp_flowandstatus = ok, the mapped settings are used downstreamRisk
Requested change
Add a validation layer for cohort-method analytic recommendations before they are committed to shell state or used for script generation.
Acceptance criteria