Feature: validator checks and updates parameters according to nlayer

[dayantur]

Problem

This PR proposes a solution for the problem discussed in #712 . In fact, different parameters arrays have different dimension requirements:

• height: nlayer + 1 elements
• veg_frac, veg_scale, building_frac, building_scale: nlayer elements
• roofs, walls (in both vertical_layers and initial_states): nlayer elements with complex nested structures

No validation existed to detect these mismatches at configuration time, and currently (if nlayer user < nlayer standard config), the validator "fills up" the remaining dimensions with dummy values.

Main Changes

• Updated orchestrator.py with detect_nlayer_from_user_yaml() to extract nlayer value from user configuration
• Updated phase_a.py with validate_nlayer_dimensions() (core validation logic for all array types), create_null_roof_wall_template() (replicates complex nested roof/wall structures with null values), annotate_missing_parameters() and create_analysis_report().

Testing

Added comprehensive tests in test/data_model/test_validation.py.

Solution

Implemented Phase A validation that:

If nlayer user < nlayer sample config:

1. Detects nlayer from user configuration automatically
2. Validates dimensions of all vertical layer arrays against nlayer requirements
3. Adds null placeholders to help users:
   - Simple arrays: pads with None values
   - Complex structures (roofs/walls): creates complete null template structures matching reference element
4. Fails validation and generates clear, actionable reports asking users to replace nulls with actual values
5. Writes updated YAML with padded arrays so users can fill in values

If nlayer user > nlayer sample config:

1. Detects nlayer from user configuration automatically
2. Validates dimensions of all vertical layer arrays against nlayer requirements
3. Fails validation and generates clear, actionable reports asking users to remove values to match required array dimension

Report Format

Clear messages distinguish between array types and levels:

  - Found (3) vertical layer array dimension mismatch error(s):
  -- veg_frac: has 3 element(s), expected 4. Added nulls.
     Suggested fix: Replace 1 null value(s) to match nlayer=4
  -- roofs at level vertical_layers: has 3 element(s), expected 2
     Suggested fix: Remove 1 alb substructure to match nlayer=2
  -- walls at level initial_states: has 1 element(s), expected 2. Added nulls.
     Suggested fix: Replace null values.

[github-actions]

🤖 I've automatically formatted the code in this PR using:

• Python: ruff v0.8.6
• Fortran: fprettify v0.3.7

Please pull the latest changes before making further edits.

[github-actions]

🔍 Schema Preview Deployed

Preview URLs:

• 🆕 Preview schemas: https://umep-dev.github.io/SUEWS/preview/pr-731/schemas/suews-config/
• 📋 Preview landing: https://umep-dev.github.io/SUEWS/preview/pr-731/

Production URLs (unchanged):

• ✅ Production schemas: https://umep-dev.github.io/SUEWS/schemas/suews-config/
• 🏠 Production landing: https://umep-dev.github.io/SUEWS/

---

⚠️ Important: Preview schemas are in a subdirectory and do not affect production. The preview pages include warning banners to prevent accidental use in production configs.

[github-actions]

🤖 I've automatically formatted the code in this PR using:

• Python: ruff v0.8.6
• Fortran: fprettify v0.3.7

Please pull the latest changes before making further edits.

[github-actions]

🤖 I've automatically formatted the code in this PR using:

• Python: ruff v0.8.6
• Fortran: fprettify v0.3.7

Please pull the latest changes before making further edits.

[github-actions]

🤖 I've automatically formatted the code in this PR using:

• Python: ruff v0.8.6
• Fortran: fprettify v0.3.7

Please pull the latest changes before making further edits.

[github-actions]

🤖 I've automatically formatted the code in this PR using:

• Python: ruff v0.8.6
• Fortran: fprettify v0.3.7

Please pull the latest changes before making further edits.

[dayantur]

Hi @sunt05 - here a proposed solution for what we discussed in #712 .
Please have a look also at Details in the first comment/summary of this PR (see above).
This should (more elegantly) fix the issue - but I am obviously happy to discuss further/change whatever you think can be improved :D

[sunt05]

Question about :

This script appears to be unused in the codebase:

• Not imported or referenced anywhere in the code
• The generated files ( - ) don't exist in the repo
• Only exists in

Was this a development/testing utility that was accidentally committed? If so, I'd suggest removing it from the PR to keep the repository clean.

If the script is intended for future use or documentation purposes, consider:

1. Moving it to or directory
2. Adding a README explaining its purpose
3. Or converting it to a pytest fixture in

What was the intended use case for this script?

[dayantur]

Question about generate_nlayer_configs.py:

This script appears to be unused in the codebase:

• Not imported or referenced anywhere in the code
• The generated files (sample_config_4.yml - sample_config_7.yml) do not exist in the repo
• Only sample_config.yml exists in src/supy/sample_data/

Was this a development/testing utility that was accidentally committed? If so, I would suggest removing it from the PR to keep the repository clean.

If the script is intended for future use or documentation purposes, consider:

1. Moving it to scripts/ or tools/ directory
2. Adding a README explaining its purpose
3. Or converting it to a pytest fixture in test/data_model/conftest.py

What was the intended use case for this script?

Sorry @sunt05 - that should be removed!

[sunt05]

Thanks @dayantur ! Please see my comment above about the script at the root level.

[github-actions]

🤖 I've automatically formatted the code in this PR using:

• Python: ruff v0.8.6
• Fortran: fprettify v0.3.7

Please pull the latest changes before making further edits.

[sunt05]

Review Summary

Excellent work on implementing automatic nlayer dimension validation! This PR addresses issue #712 comprehensively with a well-designed solution. All 5 new tests pass successfully.

Strengths

1. Comprehensive validation logic: Correctly handles all array types (simple arrays, nlayer+1 arrays, and complex nested structures)
2. Smart template generation: The create_null_roof_wall_template() function intelligently replicates complex nested structures for roofs/walls
3. Clear error reporting: The report format clearly distinguishes between array types and provides actionable fix suggestions
4. Excellent test coverage: 5 comprehensive tests covering all scenarios (simple arrays, multiple errors, complex structures, and no-error cases)
5. Safe defaults: detect_nlayer_from_user_yaml() gracefully handles errors and defaults to 3

Minor Observations

1. Code formatting change: get_ver_git.py has a formatting change (black/ruff auto-format) that's unrelated to the feature - this is fine but worth noting
2. Silent error handling: detect_nlayer_from_user_yaml() silently defaults to 3 on all errors. This is probably fine, but consider if logging might help debugging in some cases
3. RefValue handling: The code correctly handles both RefValue format ({"value": ...}) and direct values - well done on this complexity

Suggestions for Future Enhancement (not blocking)

1. Consider adding a test that validates the actual error report format/content (checking the report string generation)
2. The null template creation for roofs/walls is impressive - might be worth documenting this approach in code comments for future maintainers

Verdict

✅ Approve - This PR is ready to merge. The implementation is solid, well-tested, and addresses the problem described in #712 effectively.

[github-actions]

🤖 I've automatically formatted the code in this PR using:

• Python: ruff v0.8.6
• Fortran: fprettify v0.3.7

Please pull the latest changes before making further edits.

[dayantur]

hi @sunt05 :) This is just to let you know that after your (positive!) review, I found a tiny bug this morning - I already fixed that and now everything seems to work correctly.

[sunt05]

Thanks @dayantur - looks there is some conflict with master please fix then we can merge! Good job!