Skip to content

Validate ACP cohort-method recommendations before script generation #41

@rkboyce

Description

@rkboyce

Problem

The cohort-method shell accepts ACP recommendations from the free-text analytic-parameters step and converts them into effective_analytic_settings, but there is no dedicated validation layer between the ACP response and downstream script generation.

This means malformed, incomplete, or semantically inconsistent ACP output can still flow into:

  • analysis-settings/cm_analytic_settings_recommendation.json
  • effective_analytic_settings
  • generated 06_cm_spec.R
  • generated analysisSpecification.json

Current behavior

  • ACP recommendations are converted through shell_settings_from_acp_recommendation(...)
  • If ACP returns source = acp_flow and status = ok, the mapped settings are used downstream
  • There is no explicit schema/semantic validation pass that confirms:
    • required settings are present
    • values are in supported ranges
    • combinations are coherent
    • unsupported fields are surfaced clearly

Risk

  • invalid generated scripts
  • silent fallback to partial defaults
  • hard-to-debug user experience when ACP output is plausible-looking but not executable

Requested change

Add a validation layer for cohort-method analytic recommendations before they are committed to shell state or used for script generation.

Acceptance criteria

  • ACP recommendation objects are validated before conversion/use
  • validation distinguishes:
    • schema errors
    • unsupported fields
    • semantically inconsistent settings
  • the shell surfaces clear actionable messages when validation fails
  • invalid recommendations do not silently produce executable-looking output
  • tests cover valid, partially valid, and invalid ACP recommendation payloads

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions