Generate simulation, model, and package classes

[deltamarnix]

This pull request introduces a code generation framework for creating Python classes and modules based on MODFLOW 6 definition files (DFNs). The changes include new functionality for generating classes, templates for code generation, and utilities for processing DFNs. Below is a summary of the most important changes, grouped by theme.

IMPORTANT: The moved files you see are not the generated files. I just moved them for easier comparison. They need to go and newly generated files need to be in place.

Core Code Generation Functionality

• Introduced generate_classes in codegen/generate_classes.py, a new entry point for generating Python classes from MODFLOW 6 DFNs. This includes a command-line interface (cli_main) and options for specifying DFN paths, GitHub repositories, and verbosity.
• Added make.py with functions for generating Python modules (make_all, make_targets, make_init) using Jinja2 templates and filters.

Filters and Utilities for DFN Processing

• Created codegen/filters.py with utility functions for processing DFNs, such as _get_vars, description, variables, and type_str. These functions assist in extracting metadata and generating Python-compatible names and types.
• Updated codegen/__init__.py to expose the generate_classes function for external use.

Jinja2 Templates for Code Generation

• Added templates in codegen/templates/ for generating Python modules:
  • __init__.py.jinja for module initialization.
  • model.py.jinja and package.py.jinja for generating model and package classes, respectively. [1] [2]
  • macros.py.jinja for reusable Jinja2 macros, such as docstrings and xattree_field.
  • recarray.py.jinja for generating record array classes.

TODO

There are still some things to do, because there are quite some special cases that need to be addressed. Especially DIS and TDIS are tricky.

• I need special cases for DIS and TDIS where blockname = dimension, then it is a dim() and it contains a scope=parent
• I need to add dim() instead of field for blockname = dimension
• Add on_setattr=update_maxbound for blockname=period
• Fix the typing of NDArrays
• Add this function? attrs_post_init
• Output in a model?
• Add separate attributes for every package instead of one packages attribute for model.
• Figure out later how we are going to do keystrings
• I need to simplify the templates by creating base templates, because simulation, model, and package look very much alike.
• devtools needs fixing for shapes around (maxbound), instead it should be two dimenions for nnodes and nper.

[Copilot]

Pull Request Overview

This pull request introduces a comprehensive code generation framework for creating Python classes from MODFLOW 6 definition files (DFNs). The main purpose is to automatically generate simulation, model, and package classes based on MODFLOW 6 specifications, moving the manually written classes to a new modflow subdirectory structure.

Key changes include:

• New code generation system with CLI interface and Jinja2 templates for generating Python classes
• Restructured import paths moving existing classes from flopy4.mf6 to flopy4.mf6.modflow
• Template-based generation system supporting simulation, model, and package class types

Reviewed Changes

Copilot reviewed 23 out of 35 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
codegen/generate_classes.py	Main entry point for code generation with CLI interface
codegen/make.py	Core generation logic using Jinja2 templates
codegen/filters.py	Utility functions for processing DFNs and type conversions
codegen/templates/*.jinja	Jinja2 templates for generating different class types
test/*.py	Updated import paths to use new modflow module structure
docs/examples/*.py	Updated import paths for documentation examples
flopy4/mf6/modflow/	Restructured module files with updated imports

Comments suppressed due to low confidence (1)

flopy4/mf6/modflow/gwf/gwf.py:28

• [nitpick] The function name '_convert_grid' uses a leading underscore which typically indicates a private function, but this appears to be a rename from 'convert_grid'. If this function is still used elsewhere, the leading underscore may be inappropriate.

def _convert_grid(value):

[wpbonelli]

nice- just transcribing some things we chatted on yesterday, and responding to a few of your notes

Add separate attributes for every package instead of one packages attribute for model.

this is related to the parent/child binding problem the writer also needs to solve. detecting when a field is a subcomponent and handling it differently. We should try and unify the detection logic somewhere so codegen and writer can both use it.

Figure out later how we are going to do keystrings

we should be able to translate keystring recarray variables into 1d arrays in the generated classes in the same way as we do regular/tabular recarray variables. Keystrings just indicate that you don't know exactly what item type to expect from list-based format: you might get any of these records on each line instead of just one. But ultimately it's still a bunch of arrays.

If we can disentangle structural specification from input format, then maybe this translation could become unnecessary, as we said yesterday.. DFNs right now do a little bit (but not all of either) of both: tell you what components connect to what and what fields are in each, and how the input files should look. Meanwhile, both mf6 and flopy need to deal with 1d arrays internally, while the DFN spec we are generating them from encodes them in list-based input structure which is just a convenient way to assign sparse data to large arrays. this feels like unclearly defined scope.

devtools needs fixing for shapes around (maxbound), instead it should be two dimenions for nnodes and nper.

a propos to the above, a nice first step might be doing the list-to-array translation proactively in the TOML DFNs instead of waiting to do it here. such that DFNs define package- or period-data (i.e. space- or time-aligned) arrays with the real shape (aligned to the grid and tdis). maxbound is an artificial dimension, really an mf6 input format detail.

a question then is how/where to define this mapping between an input-format-agnostic structural representation and the specific mf6 list-based input format. a new section inside each DFN? "sidecar" file alongside the DFN? entirely separate specification? is it even reasonable to try and separate these concerns, or is the structure of an mf6 simulation inextricable from its input format? the fact that we are considering different input formats (e.g. netcdf) suggests not.