Built-in way to change model seed for each iteration in batch run?

[dylan-munson]

Hi, I have a model that I am trying to run with a starting seed in the batch runner. I only just realized (though it probably should have been obvious to me earlier) that if you just specify one seed as a model parameter, the same seed will be used for all iterations in the batch run. Is there a built-in/simple way to get the seed to change for each iteration (so that say, running 100 iterations doesn't really just collapse down to running 1)?

[mlrx-potindji]

I noticed the same issue a few weeks ago. Since I didn’t have time to wait for an answer, I came up with a workaround: I ran the model iteratively over the maximum number of iterations without setting a seed. This way, each batch of runs produced slightly different results. While this can affect reproducibility, it works fine as a quick solution when only first-glance results are needed.

I hope it helps.

# Number of independent simulation runs
max_iteration = 100  

# Number of steps (time periods) per simulation
max_steps = 100       

# --- First simulation run (iteration 0) ---

# Initialize the model
model = NoWardModel()

# Run the model for the defined number of steps
for _ in range(max_steps):
    model.step()

# Collect model-level data from Mesa's DataCollector
model_data = model.datacollector.get_model_vars_dataframe()

# Add extra columns for iteration number and step counter
model_data['iteration'] = 0  # first run = iteration 0
model_data['Step'] = list(range(1, max_steps + 1))

# Create the main results DataFrame to hold all runs
df = model_data.copy()


# --- Repeat for the remaining runs (iterations 1 → max_iteration-1) ---
for it in range(1, max_iteration):

    # Initialize a fresh model (independent run)
    model = NoWardModel()

    # Run it for max_steps steps
    for _ in range(max_steps):
        model.step()

    # Collect model-level data for this run
    model_data = model.datacollector.get_model_vars_dataframe()

    # Add iteration number and step index
    model_data['iteration'] = it
    model_data['Step'] = list(range(1, max_steps + 1))

    # Append results from this run to the main DataFrame
    df = pd.concat([df, model_data], axis=0, ignore_index=True)

[mlrx-potindji]

Great insights from the ema_workbench. Thanks

[EwoutH]

Thanks for reaching out. I think the best way to currently approach this is instead of passing a single seed, pass an array of different seeds:

# Generate unique seeds for each iteration
n_iterations = 10
base_seed = 42
seeds = [base_seed + i for i in range(n_iterations)]

parameters = {
    'seed': seeds,
    # other parameters...
}

batch_run(
    MyModel,
    parameters,
    iterations=1,  # Set to 1 since seeds are now in parameters
    number_processes=4
)

[EwoutH]

I think the issue could be addressed in the Mesa library by modifying the _model_run_func to handle seeds more intelligently, if we want to:

def _model_run_func(
    model_cls: type[Model],
    run: tuple[int, int, dict[str, Any]],
    max_steps: int,
    data_collection_period: int,
) -> list[dict[str, Any]]:
    run_id, iteration, kwargs = run
    
    # Handle seed uniqueness across iterations
    if 'seed' in kwargs and iteration > 0:
        # Only modify if seed is a single value (not already an iterable)
        seed_value = kwargs['seed']
        if isinstance(seed_value, (int, float)) and not isinstance(seed_value, bool):
            kwargs = kwargs.copy()
            kwargs['seed'] = int(seed_value) + iteration
    
    model = model_cls(**kwargs)
    # ... rest of function

However, we have to discuss if:

• A) the current behavior is intended, and changing it would be a breaking change
• B) the current behavior is unintended, and changing it would be a bugfix

[tpike3]

I would argue B, bugfix

[dylan-munson]

Thanks @EwoutH , this does indeed seem to work as a stopgap until this can be addressed more fully in a future version of Mesa. I also agree that this seems more like a bug.