multi-objectinge workflows

Author: ddy6

README.md

@@ -7,9 +7,9 @@ Optimum trial targeting for expensive black-box evaluations improves sample effi
 
 Looptimum is a file-backed loop for optimum parameter targeting when each
 trial is costly (time, compute, money, or operational risk).
-You provide a parameter space and a scalar objective; Looptimum suggests the
+You provide a parameter space and objective schema; Looptimum suggests the
 next trial, records decisions, and resumes cleanly after interruptions.
-Current stable release: `v0.3.3`.
+Current stable release: `v0.3.5`.
 For expensive black-box objectives, Looptimum starts with bounded exploration
 and then shifts to surrogate-guided suggestion ranking to reduce wasted trials.
 Its key differentiator is operational: a file-backed, resumable workflow that
@@ -25,8 +25,9 @@ For a spec-style contract summary, use
 - Private contact: [contact@looptimum.com](mailto:contact@looptimum.com)
 - Start here: [`PILOT.md`](PILOT.md), [`intake.md`](intake.md),
   [`docs/pilot-checklist.md`](docs/pilot-checklist.md)
-- Best initial fit: bounded numeric parameters, one scalar objective, and
-  expensive evaluations in client-controlled environments
+- Best initial fit: bounded parameter spaces, one scalar objective or explicit
+  scalarization rule, and expensive evaluations in client-controlled
+  environments
 - Scope and delivery are tailored to the project; contact for scope
 
 ## Common Triggers
@@ -41,7 +42,7 @@ For a spec-style contract summary, use
 
 Looptimum replaces ad hoc sweep loops with a small, explicit workflow:
 
-1. Define parameter bounds and objective direction.
+1. Define parameter bounds, objective schema, and optional constraints.
 2. `suggest` one trial.
 3. Run that trial in your environment.
 4. `ingest` the result and repeat.
@@ -200,7 +201,8 @@ expanded stub in
 - Each evaluation is expensive enough that sample efficiency matters.
 - Your evaluator runs as external jobs and you want a thin outer loop above
   training/evaluation infrastructure.
-- You can define one scalar objective (`minimize` or `maximize`).
+- You can define one scalar objective or an explicit scalarization /
+  lexicographic rule for multiple objectives.
 - You have a bounded parameter set (commonly small-to-moderate dimensional).
 - You want resumable, file-backed operation in local/offline/restricted
   environments.
@@ -217,9 +219,11 @@ expanded stub in
 
 ### Inputs
 
-- Parameter space definition (`float` and `int` currently supported in public
-  templates).
-- Objective schema (name + direction).
+- Parameter space definition (`float`, `int`, `bool`, and `categorical` in
+  public templates; numeric params can also declare `scale`, and params may use
+  `when` for conditional activation).
+- Objective schema (required `primary_objective`, optional
+  `secondary_objectives`, optional `scalarization` policy).
 - Trial budget and seed/config settings.
 
 ### `suggest` Output
@@ -236,8 +240,8 @@ Each suggestion includes:
 - `trial_id` (must match a pending trial)
 - `params` (must match suggested params exactly)
 - `objectives`:
-  - `status: ok` -> primary objective must be numeric and finite
-  - non-`ok` status -> primary objective must be `null`
+  - `status: ok` -> all configured objective values must be numeric and finite
+  - non-`ok` status -> all configured objective values must be `null`
 - `status`: `ok`, `failed`, `killed`, `timeout`
 
 ### `ingest` Optional Fields
@@ -259,8 +263,11 @@ Each suggestion includes:
 
 Best ranking rule:
 
-- `best` is computed only from `status: "ok"` observations and their primary
-  objective values.
+- `best` is computed only from `status: "ok"` observations.
+- Single-objective campaigns rank by the primary objective value.
+- Multi-objective campaigns rank by the configured scalarization or
+  lexicographic policy while preserving raw objective vectors in status,
+  manifests, and reports.
 - `penalty_objective` is never used to rank `best`.
 
 ### Local State Files
@@ -271,7 +278,9 @@ Best ranking rule:
 - `state/acquisition_log.jsonl`: append-only decision trace.
 - `state/event_log.jsonl`: append-only lifecycle/operations trace.
 - `state/trials/trial_<id>/manifest.json`: per-trial audit manifest.
-- `state/report.json` and `state/report.md`: explicit report outputs from `report`.
+- `state/report.json` and `state/report.md`: explicit report outputs from
+  `report`, including objective-config and Pareto summaries for multi-objective
+  campaigns.
 
 ### Compatibility Notes
 
@@ -280,10 +289,6 @@ Best ranking rule:
   `terminal_reason`.
 - For non-`ok` outcomes with no reason provided, ingest synthesizes
   `terminal_reason` as `status=<status>`.
-- Legacy `v0.2.x` non-`ok` payloads with numeric primary objective are
-  accepted in `v0.3.x`, normalized to
-  `objective: null` + `penalty_objective`, and emit a deprecation warning.
-- Sentinel primary-objective compatibility is planned for removal in `v0.4.0`.
 - `v0.2.x` state without `schema_version` (or with `0.2.x`) upgrades in-memory
   to `0.3.0` and persists on next mutating command.
 - Earlier `v0.3.x` state versions load transparently in `v0.3.x`.
@@ -295,7 +300,7 @@ Best ranking rule:
   compatibility.
 - Breaking changes are allowed only on `0.x` major-line increments (for
   example `0.3 -> 0.4`) and require explicit compatibility notes.
-- Current patch tag in this line: `v0.3.3` (see `CHANGELOG.md`).
+- Current patch tag in this line: `v0.3.5` (see `CHANGELOG.md`).
 - Full policy: [`docs/stability-guarantees.md`](docs/stability-guarantees.md).
 
 ### Duplicate Ingest Behavior
@@ -335,6 +340,8 @@ The `examples/` folder shows integration patterns, not benchmark leaderboards.
 - `examples/toy-objectives/02_subprocess_cli/`: subprocess/CLI wrapper pattern
 - `examples/toy_objectives/03_tiny_quadratic_loop/`: dedicated tiny end-to-end
   objective (`suggest -> evaluate -> ingest -> status`, typically under one minute)
+- `docs/examples/multi_objective/`: generated multi-objective report/state pack
+  with weighted-sum and lexicographic objective-schema examples
 
 Run the tiny end-to-end objective from repo root:
 

client_harness_template/tests/test_phase6_assets.py

@@ -10,6 +10,7 @@
 CI_WORKFLOW = REPO_ROOT / ".github" / "workflows" / "ci.yml"
 PYPROJECT = REPO_ROOT / "pyproject.toml"
 TYPE_SAFETY_DOC = REPO_ROOT / "docs" / "type-safety.md"
+MULTI_OBJECTIVE_EXAMPLE = REPO_ROOT / "docs" / "examples" / "multi_objective"
 
 
 def test_golden_acquisition_log_has_expected_shape_and_timestamps() -> None:
@@ -76,3 +77,35 @@ def test_mypy_scope_and_type_safety_doc_are_present() -> None:
     doc_text = TYPE_SAFETY_DOC.read_text(encoding="utf-8")
     assert "Type-checking tool: `mypy`." in doc_text
     assert "Initial blocking CI gate scope" in doc_text
+
+
+def test_multi_objective_example_pack_has_expected_artifacts() -> None:
+    assert MULTI_OBJECTIVE_EXAMPLE.exists(), (
+        f"missing multi-objective example pack: {MULTI_OBJECTIVE_EXAMPLE}"
+    )
+
+    readme_path = MULTI_OBJECTIVE_EXAMPLE / "README.md"
+    weighted_schema = MULTI_OBJECTIVE_EXAMPLE / "objective_schema.json"
+    lexicographic_schema = MULTI_OBJECTIVE_EXAMPLE / "objective_schema_lexicographic.json"
+    status_path = MULTI_OBJECTIVE_EXAMPLE / "status_after_ingest.json"
+    report_path = MULTI_OBJECTIVE_EXAMPLE / "state" / "report.json"
+    manifest_path = MULTI_OBJECTIVE_EXAMPLE / "state" / "trials" / "trial_1" / "manifest.json"
+
+    for path in (
+        readme_path,
+        weighted_schema,
+        lexicographic_schema,
+        status_path,
+        report_path,
+        manifest_path,
+    ):
+        assert path.exists(), f"missing multi-objective example artifact: {path}"
+
+    report_payload = json.loads(report_path.read_text(encoding="utf-8"))
+    assert report_payload["objective_config"]["objective_names"] == ["loss", "throughput"]
+    assert report_payload["pareto_front"]["trial_ids"] == [1, 2]
+    assert report_payload["best"]["objective_vector"] == {"loss": 0.3, "throughput": 2.0}
+
+    manifest_payload = json.loads(manifest_path.read_text(encoding="utf-8"))
+    assert manifest_payload["scalarization_policy"] == "weighted_sum"
+    assert manifest_payload["objective_vector"] == {"loss": 0.3, "throughput": 2.0}

docs/README.md

@@ -25,11 +25,11 @@ Included files:
   state persistence, contamination controls, safe parallelism, and robust-best policy
 - `stability-guarantees.md`: public compatibility guarantees, deprecation
   policy, and breaking-change rules for `v0.3.x` (current patch:
-  `v0.3.3`)
+  `v0.3.5`)
 - `feedback-loop.md`: post-release feedback intake, issue-triage workflow,
   and backlog synchronization rules
 - `search-space.md`: supported parameter types, constraints framing, and
-  multi-objective boundaries
+  multi-objective handling
 - `constraints.md`: hard-constraint contract semantics, troubleshooting, and
   example authoring guidance
 - `decision-trace.md`: acquisition log schema and decision metadata guidance
@@ -46,6 +46,7 @@ Reference artifacts:
 
 - `../PILOT.md`
 - `examples/README.md`
+- `examples/multi_objective/README.md`
 - `examples/state_snapshots/README.md`
 - `examples/decision_trace/README.md`
 - `examples/constraints/README.md`

docs/ci-knob-tuning.md

@@ -113,7 +113,7 @@ Default policy: top-k + median-of-repeats.
 
 Recommended procedure:
 
-1. rank candidate configs by primary objective
+1. rank candidate configs by the configured objective policy
 2. select top-k (for example `k=3` or `k=5`)
 3. run each candidate for `n` repeats (for example `n=3`)
 4. choose winner by median objective

docs/examples/README.md

@@ -10,6 +10,8 @@ Integration pattern note:
 
 Included:
 
+- `multi_objective/`: generated weighted-sum / lexicographic example pack with
+  `status`, `report`, and trial-manifest outputs
 - `state_snapshots/`: sample state/log/CSV snapshots captured from a temp run
   of `templates/bo_client_demo`
 - snapshots include both `status: "ok"` and non-`ok` ingest examples

docs/examples/multi_objective/README.md

@@ -0,0 +1,32 @@
+# Multi-Objective Example Pack
+
+Generated reference artifacts for a small multi-objective run captured from
+`templates/bo_client_demo`.
+
+This pack shows the current public multi-objective contract surface:
+
+- `objective_schema.json`: weighted-sum example with `primary_objective`,
+  `secondary_objectives`, and `scalarization`
+- `objective_schema_lexicographic.json`: alternative lexicographic objective
+  contract using the same raw objective names
+- `suggestion_1.json` / `suggestion_2.json`: canonical suggestion payloads
+- `result_1.json` / `result_2.json`: ingest payloads with complete
+  `objectives` maps
+- `status_after_ingest.json`: `best` with scalarized ranking metadata and raw
+  `objective_vector`
+- `state/report.json` / `state/report.md`: report outputs with
+  `objective_config`, scalarized ranking data, and `pareto_front`
+- `state/trials/trial_<id>/manifest.json`: trial manifests with
+  `objective_vector` and `scalarization_policy`
+
+The weighted-sum capture uses:
+
+- primary objective: `loss` (`minimize`)
+- secondary objective: `throughput` (`maximize`)
+- scalarization policy: `weighted_sum`
+
+The generated `report.json` shows both observed trials on the Pareto front, and
+`status_after_ingest.json` shows `best.objective_name = "scalarized"` with the
+raw objective vector preserved.
+
+These files are documentation examples, not benchmark claims.

docs/examples/multi_objective/bo_config.json

@@ -0,0 +1,37 @@
+{
+  "seed": 17,
+  "max_trials": 40,
+  "initial_random_trials": 6,
+  "candidate_pool_size": 600,
+  "surrogate": {
+    "type": "rbf_proxy",
+    "length_scale": 0.22
+  },
+  "acquisition": {
+    "type": "ucb",
+    "kappa": 1.8,
+    "xi": 0.01
+  },
+  "feature_flags": {
+    "enable_botorch_gp": false,
+    "fallback_to_proxy_if_unavailable": true,
+    "enable_service_api_preview": false,
+    "enable_dashboard_preview": false,
+    "enable_auth_preview": false
+  },
+  "batch_size": 1,
+  "stopping": {
+    "min_improvement": 0.0001,
+    "patience_trials": 10
+  },
+  "paths": {
+    "state_file": "state/bo_state.json",
+    "observations_csv": "state/observations.csv",
+    "acquisition_log_file": "state/acquisition_log.jsonl",
+    "constraints_schema_file": "../_shared/schemas/constraints.schema.json",
+    "ingest_schema_file": "../_shared/schemas/ingest_payload.schema.json",
+    "objective_schema_schema_file": "../_shared/schemas/objective_schema.schema.json",
+    "search_space_schema_file": "../_shared/schemas/search_space.schema.json",
+    "suggestion_schema_file": "../_shared/schemas/suggestion_payload.schema.json"
+  }
+}

docs/examples/multi_objective/objective_schema.json

@@ -0,0 +1,19 @@
+{
+  "primary_objective": {
+    "name": "loss",
+    "direction": "minimize"
+  },
+  "secondary_objectives": [
+    {
+      "name": "throughput",
+      "direction": "maximize"
+    }
+  ],
+  "scalarization": {
+    "policy": "weighted_sum",
+    "weights": {
+      "loss": 1.0,
+      "throughput": 1.0
+    }
+  }
+}

docs/examples/multi_objective/objective_schema_lexicographic.json

@@ -0,0 +1,15 @@
+{
+  "primary_objective": {
+    "name": "loss",
+    "direction": "minimize"
+  },
+  "secondary_objectives": [
+    {
+      "name": "throughput",
+      "direction": "maximize"
+    }
+  ],
+  "scalarization": {
+    "policy": "lexicographic"
+  }
+}

docs/examples/multi_objective/parameter_space.json

@@ -0,0 +1,16 @@
+{
+  "parameters": [
+    {
+      "name": "x1",
+      "type": "float",
+      "bounds": [0.0, 1.0],
+      "description": "First controllable factor"
+    },
+    {
+      "name": "x2",
+      "type": "float",
+      "bounds": [0.0, 1.0],
+      "description": "Second controllable factor"
+    }
+  ]
+}