From d00cf4fc3cc1dc87887578cb1c8f68e3809585e4 Mon Sep 17 00:00:00 2001 From: GabrielLoiseau Date: Wed, 29 Oct 2025 19:14:30 +0100 Subject: [PATCH 1/3] Add GEPA gepa_kwargs documentation --- dspy/teleprompt/gepa/gepa.py | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/dspy/teleprompt/gepa/gepa.py b/dspy/teleprompt/gepa/gepa.py index 87cbbf80a5..8b8509551a 100644 --- a/dspy/teleprompt/gepa/gepa.py +++ b/dspy/teleprompt/gepa/gepa.py @@ -243,7 +243,7 @@ def metric( Note: When both instruction_proposer and reflection_lm are set, the instruction_proposer is called in the reflection_lm context. However, reflection_lm is optional when using a custom instruction_proposer. Custom instruction proposers can invoke their own LLMs if needed. - component_selector: Custom component selector implementing the ReflectionComponentSelector protocol, + component_selector: Custom component selector implementing the [ReflectionComponentSelector](https://github.com/gepa-ai/gepa/blob/main/src/gepa/proposer/reflective_mutation/base.py) protocol, or a string specifying a built-in selector strategy. Controls which components (predictors) are selected for optimization at each iteration. Defaults to 'round_robin' strategy which cycles through components one at a time. Available string options: 'round_robin' (cycles through components sequentially), @@ -274,7 +274,38 @@ def metric( called with and without the pred_name. This flag (defaults to True) determines whether a warning is raised if a mismatch in module-level and predictor-level score is detected. seed: The random seed to use for reproducibility. Default is 0. - gepa_kwargs: (Optional) provide additional kwargs to be passed to [gepa.optimize](https://github.com/gepa-ai/gepa/blob/main/src/gepa/api.py) method + gepa_kwargs: (Optional) Additional keyword arguments to pass directly to [gepa.optimize](https://github.com/gepa-ai/gepa/blob/main/src/gepa/api.py). + Useful for accessing advanced GEPA features not directly exposed through DSPy's GEPA interface. + + Available parameters: + - batch_sampler: Strategy for selecting training examples. Can be a [BatchSampler](https://github.com/gepa-ai/gepa/blob/main/src/gepa/strategies/batch_sampler.py) instance or a string + ('epoch_shuffled'). Defaults to 'epoch_shuffled'. Only valid when reflection_minibatch_size is None. + - reflection_prompt_template: Custom prompt template for reflection. Must contain placeholders + `` and ``. Ignored if adapter provides its own + `propose_new_texts` method. + - merge_val_overlap_floor: Minimum number of shared validation ids required between parents before + attempting a merge subsample. Only relevant when using `val_evaluation_policy` other than 'full_eval'. + Default is 5. + - stop_callbacks: Optional stopper(s) that return True when optimization should stop. Can be a single + [StopperProtocol](https://github.com/gepa-ai/gepa/blob/main/src/gepa/utils/stop_condition.py) or a list of StopperProtocol instances. + Examples: [FileStopper](https://github.com/gepa-ai/gepa/blob/main/src/gepa/utils/stop_condition.py), + [TimeoutStopCondition](https://github.com/gepa-ai/gepa/blob/main/src/gepa/utils/stop_condition.py), + [SignalStopper](https://github.com/gepa-ai/gepa/blob/main/src/gepa/utils/stop_condition.py), + [NoImprovementStopper](https://github.com/gepa-ai/gepa/blob/main/src/gepa/utils/stop_condition.py), + or custom stopping logic. Note: This overrides the default + max_metric_calls stopping condition. + - mlflow_tracking_uri: The tracking URI to use for MLflow (when use_mlflow=True). + - mlflow_experiment_name: The experiment name to use for MLflow (when use_mlflow=True). + - display_progress_bar: Show a tqdm progress bar over metric calls. Default is True. Set to False to disable. + - use_cloudpickle: Use cloudpickle instead of pickle for serialization. Can be helpful when the + serialized state contains dynamically generated DSPy signatures. Default is False. + - val_evaluation_policy: Strategy controlling which validation ids to score each iteration. Can be + 'full_eval' (evaluate every id each time) or an [EvaluationPolicy](https://github.com/gepa-ai/gepa/blob/main/src/gepa/strategies/eval_policy.py) instance. Default is 'full_eval'. + - raise_on_exception: Whether to propagate proposer/evaluator exceptions instead of stopping gracefully. + Default is True. Set to False for graceful error handling. + + Note: Parameters already handled by DSPy's GEPA class will be overridden by the direct parameters + and should not be passed through gepa_kwargs. Note: Budget Configuration: Exactly one of `auto`, `max_full_evals`, or `max_metric_calls` must be provided. From 85dcbef02a0b369dca9b5dfae78873dcff6873db Mon Sep 17 00:00:00 2001 From: GabrielLoiseau Date: Fri, 31 Oct 2025 10:06:16 +0100 Subject: [PATCH 2/3] Remove unnecessary arguments --- dspy/teleprompt/gepa/gepa.py | 8 -------- 1 file changed, 8 deletions(-) diff --git a/dspy/teleprompt/gepa/gepa.py b/dspy/teleprompt/gepa/gepa.py index 8b8509551a..8ef1591b51 100644 --- a/dspy/teleprompt/gepa/gepa.py +++ b/dspy/teleprompt/gepa/gepa.py @@ -280,9 +280,6 @@ def metric( Available parameters: - batch_sampler: Strategy for selecting training examples. Can be a [BatchSampler](https://github.com/gepa-ai/gepa/blob/main/src/gepa/strategies/batch_sampler.py) instance or a string ('epoch_shuffled'). Defaults to 'epoch_shuffled'. Only valid when reflection_minibatch_size is None. - - reflection_prompt_template: Custom prompt template for reflection. Must contain placeholders - `` and ``. Ignored if adapter provides its own - `propose_new_texts` method. - merge_val_overlap_floor: Minimum number of shared validation ids required between parents before attempting a merge subsample. Only relevant when using `val_evaluation_policy` other than 'full_eval'. Default is 5. @@ -294,15 +291,10 @@ def metric( [NoImprovementStopper](https://github.com/gepa-ai/gepa/blob/main/src/gepa/utils/stop_condition.py), or custom stopping logic. Note: This overrides the default max_metric_calls stopping condition. - - mlflow_tracking_uri: The tracking URI to use for MLflow (when use_mlflow=True). - - mlflow_experiment_name: The experiment name to use for MLflow (when use_mlflow=True). - - display_progress_bar: Show a tqdm progress bar over metric calls. Default is True. Set to False to disable. - use_cloudpickle: Use cloudpickle instead of pickle for serialization. Can be helpful when the serialized state contains dynamically generated DSPy signatures. Default is False. - val_evaluation_policy: Strategy controlling which validation ids to score each iteration. Can be 'full_eval' (evaluate every id each time) or an [EvaluationPolicy](https://github.com/gepa-ai/gepa/blob/main/src/gepa/strategies/eval_policy.py) instance. Default is 'full_eval'. - - raise_on_exception: Whether to propagate proposer/evaluator exceptions instead of stopping gracefully. - Default is True. Set to False for graceful error handling. Note: Parameters already handled by DSPy's GEPA class will be overridden by the direct parameters and should not be passed through gepa_kwargs. From b665b28e6e07c17934e9010993690823ecad3abb Mon Sep 17 00:00:00 2001 From: Gabriel Loiseau <64709298+GabrielLoiseau@users.noreply.github.com> Date: Tue, 4 Nov 2025 14:54:23 +0100 Subject: [PATCH 3/3] Add MLflow args --- dspy/teleprompt/gepa/gepa.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/dspy/teleprompt/gepa/gepa.py b/dspy/teleprompt/gepa/gepa.py index 8ef1591b51..f32483bd6c 100644 --- a/dspy/teleprompt/gepa/gepa.py +++ b/dspy/teleprompt/gepa/gepa.py @@ -295,6 +295,10 @@ def metric( serialized state contains dynamically generated DSPy signatures. Default is False. - val_evaluation_policy: Strategy controlling which validation ids to score each iteration. Can be 'full_eval' (evaluate every id each time) or an [EvaluationPolicy](https://github.com/gepa-ai/gepa/blob/main/src/gepa/strategies/eval_policy.py) instance. Default is 'full_eval'. + - use_mlflow: If True, enables MLflow integration to log optimization progress. + MLflow can be used alongside Weights & Biases (WandB). + - mlflow_tracking_uri: The tracking URI to use for MLflow (when use_mlflow=True). + - mlflow_experiment_name: The experiment name to use for MLflow (when use_mlflow=True). Note: Parameters already handled by DSPy's GEPA class will be overridden by the direct parameters and should not be passed through gepa_kwargs.