Skip to content

Add Q-matrix preprocessing permutation#33

Open
bernalde wants to merge 3 commits into
mainfrom
fix/issue-12-q-matrix-permutation
Open

Add Q-matrix preprocessing permutation#33
bernalde wants to merge 3 commits into
mainfrom
fix/issue-12-q-matrix-permutation

Conversation

@bernalde

@bernalde bernalde commented May 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Add deterministic Q-matrix preprocessing that uses a reverse Cuthill-McKee-style ordering to place coupled variables closer in the MPS tensor order.
  • Preserve caller-facing variable order by unpermuting Solution samples and probability checks.
  • Wire the existing preprocess optimizer attribute through the QUBODrivers/JuMP solve path.
  • Merge current main into this branch and drop the stale duplicate one-variable QUBO handling that already landed via minimize/maximize crash on single-variable (1x1) QUBO inputs #49.
  • Expand qmatrix_permutation documentation for cutoff, return semantics, and examples.

Tests run

  • git diff --check origin/main...HEAD
    • Result: passed.
  • julia +1.12 --project=. -e 'using Pkg; Pkg.test()'
    • Result: passed.
  • julia +1.12 --project=docs/ docs/make.jl
    • First attempt failed before loading project code because the docs environment was not instantiated.
  • julia +1.12 --project=docs/ -e 'using Pkg; Pkg.instantiate()'
    • Result: passed.
  • julia +1.12 --project=docs/ docs/make.jl
    • Result: passed. Documenter skipped deployment because this was not running in CI.

Closes #12

@bernalde bernalde left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking issues found. I inspected the PR diff, surrounding solver/Solution/JuMP code, tests, docs, README, and CI workflows. The implementation keeps caller-facing bit order stable while using the permuted tensor order internally, and the one-variable fallback is consistent with the existing API surface.
Tests run locally:

  • julia +1.12 --project=. -e 'using Pkg; Pkg.test()' passed.
  • julia +1.12 --project=docs/ docs/make.jl passed.
  • julia +1.12 --project=test -e 'push!(LOAD_PATH, pwd()); using Test, TenSolver, JuMP, LinearAlgebra; ...' targeted preprocessing/JuMP/one-variable smoke passed.

@bernalde bernalde requested a review from iagoleal May 15, 2026 20:43

@bernalde bernalde left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Senior maintainer review. Note: I am the PR author, so GitHub only permits a COMMENT event from me; a formal approval or change-request must come from another maintainer. No blocking issues found.

Diff verification: merge-base ca27880; git diff <merge-base>..HEAD --stat = 6 files, +228 / -5, matching the PR's reported totals.

Issue intent (#12): the PR implements a real heuristic, not a placeholder — a weighted Reverse Cuthill-McKee ordering (src/solver.jl) that reduces coupling bandwidth so the QUBO better matches a 1D MPS chain. Verified behaviorally: on a scrambled path graph it reduces bandwidth 3 -> 1. This is a reasonable interpretation of #12.

Correctness (the critical check) — apply/invert is correct:

  • Apply: preprocess_qubo returns Q[perm, perm], l[perm]; site k = original variable perm[k].
  • Invert on sampling: original_order does x[perm] = bs (src/solution.jl), mapping site-order samples back to original variable order.
  • Invert on lookup: coeff does bs = bs[psi.permutation], the correct inverse direction.
  • Energy uses the original Q/l on the already-un-permuted sample, so returned energy is in the original problem space.
  • End-to-end check: minimize(Q, l; preprocess=true) returned the correct energy and a sample in original variable order. No un-inverted-permutation bug.

Default behavior unchanged: preprocess defaults to false in both the JuMP attribute and the minimize kwarg; existing results are not altered unless a user opts in. maximize forwards the kwarg too.

Nonblocking

  1. qmatrix_permutation is exported in docs/src/api.md @docs but its docstring does not document the cutoff argument, the return-value semantics (entry k is the original index for site k), or include an example. Expand with an Arguments/Returns section and a small jldoctest.
  2. Cutoff default inconsistency: standalone qmatrix_permutation defaults cutoff = 0 (src/solver.jl:102) but preprocess_qubo passes the solver's cutoff = 1e-8. Harmless, but add a one-line comment noting the two callers' differing thresholds are intentional.

Questions
3. Completeness of Closes #12: RCM is a single static heuristic; #12 frames the ordering problem as NP and references specific literature (the issue comment points to the tridiagonal-QUBO-in-P result, arXiv:2309.10509). RCM is a fine first implementation but does not try alternatives or claim optimality. Consider Refs #12 and keeping the issue open for follow-up, or accept RCM as sufficient — reviewer's call.
4. The PR bundles an exact one-variable solve path (src/solver.jl, length(sites)==1 branch) that is logically independent of permutation. It is correct and covered by the new "One variable" test, but is the same single-variable fix appearing across several parallel PRs (#34, #41) — worth de-duplicating across the open PRs and calling out in the description.
5. The PR description claims Pkg.test() failed on a manifest/OpenSSL_jll mismatch requiring julia +1.12. On Julia 1.10.11 in a clean worktree, Pkg.instantiate() and the full Pkg.test() both succeeded — that caveat appears to be a local-environment artifact and can be dropped.

Tests run (Julia 1.10.11, clean worktree on the PR head):

  • julia --project=. -e 'using Pkg; Pkg.instantiate(); Pkg.test()' — full suite passed (Testing TenSolver tests passed): QUBO Correctness 78/78 incl. new "One variable", "Permutation reduces coupling bandwidth", "Samples are returned in original variable order"; JuMP preprocess attribute 2/2; QUBODrivers.jl 130; Aqua.jl 11; VRP 2; HDF5 16; Doctests 1.
  • Independent round-trip: permutation validity, bandwidth 3 -> 1, preprocess=true vs false agree on energy.

Merge readiness: Ready to merge after minor polish. The core risk — a permutation not inverted on output — is not present; apply and invert are correct and verified. Default behavior is unchanged (opt-in), and the full suite passes. Address the two docstring/cutoff nonblocking items and decide Closes vs Refs #12. The formal verdict must come from another maintainer.

Comment thread src/solver.jl
Return a deterministic permutation that places coupled QUBO variables closer
together in the one-dimensional MPS ordering.
"""
function qmatrix_permutation(Q::AbstractMatrix; cutoff = 0)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nonblocking: qmatrix_permutation is exported in the docs @docs block but its docstring omits the cutoff argument, the return-value semantics (entry k is the original index for site k), and an example. Also note the default here is cutoff = 0 while preprocess_qubo calls it with cutoff = 1e-8 — worth a comment that the differing thresholds are intentional.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 2c9cf21. The qmatrix_permutation docstring now documents cutoff, return semantics, an example, and the intentional 0 vs 1e-8 cutoff difference.

@bernalde bernalde left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Senior maintainer review of PR #33 (Add Q-matrix preprocessing permutation). Diff totals confirmed against the merge-base via gh: +228 / -5 across 6 files, matching the PR's reported totals. The head was reviewed statically from the exact head SHA (554fe7a); full head file contents were retrieved via the GitHub contents API.

Procedural note: I am the PR author identity, so GitHub will not let me APPROVE or REQUEST_CHANGES on my own PR. This is posted as event=COMMENT. The formal verdict must come from another maintainer.

Environment limitation (disclosed for transparency): in this review sandbox, git and julia invocations were denied, so I could not check out the head working tree or run the test suite myself. I therefore could not independently reproduce the PR's claimed Pkg.test() / docs-build passes, and could not re-run the new testsets. All findings below are from static analysis of the diff plus the full head sources; the test-execution claims in the PR description remain unverified by me.

--- Blocking ---

  1. The PR conflicts with main and re-implements one-variable QUBO handling that main already provides. gh pr view 33 reports mergeable=CONFLICTING, mergeStateStatus=DIRTY. The PR branch (commits dated 2026-05-15) was cut before #49 (fix: solve single-variable QUBO inputs exactly instead of crashing, merged 2026-06-11, commit b67de66) landed on main. main's _minimize already has if length(sites) == 1; return _single_variable_minimize(...), whereas this PR adds a different inline if length(sites) == 1 ... end block in the same region (src/solver.jl head lines 324-343). These two implementations collide. Why it matters: the PR cannot merge as-is, and the bundled one-variable code is now redundant work that duplicates and diverges from the merged solution. Fix: rebase onto current main, drop this PR's inline one-variable block and the test/qubo.jl "One variable" testset (or reconcile them with _single_variable_minimize), and keep only the permutation feature, which is the actual subject of issue #12. Refs #33, and main commit b67de66.

  2. Closes #12 overstates and mixes scope. Issue #12 asks only for Q-matrix permutation toward a more 1D (tridiagonal) site ordering. This PR delivers that, but also bundles unrelated one-variable solve handling (item 1). A faithful resolution of #12 is the permutation work alone. Fix: scope the PR to the permutation feature and keep Closes #12; remove the one-variable changes (already solved on main). If for some reason one-variable code must stay, it should be Refs, not Closes, and reconciled with the existing path. Refs #12.

  3. Tests for the changed behavior were not verifiable here, and the suite cannot be trusted from the PR description alone. I could not run julia +1.12 --project=. -e 'using Pkg; Pkg.instantiate(); Pkg.test()' (julia execution denied in this environment). Because of the merge conflict in item 1, the suite as it would land on main does not even exist as a clean tree yet. Another maintainer must run the full suite on a clean rebased checkout before merge and confirm both the new permutation testsets and the existing single-variable tests pass together. This is blocking only in the sense that changed behavior must be shown green on a mergeable tree; it is not a claim that the code is wrong.

--- Nonblocking ---

  1. qmatrix_adjacency iterates all O(n^2) index pairs of Q even when Q is sparse (the JuMP/QUBODrivers path passes a sparse matrix from QUBOTools.qubo(...; :sparse)). For a preprocessing pass this is acceptable, but it defeats sparsity for large problems. Consider iterating stored nonzeros for the sparse case. src/solver.jl qmatrix_adjacency.

  2. qmatrix_permutation default is cutoff = 0, while preprocess_qubo forwards the solver cutoff (default 1e-8). The two defaults differ; this is fine in practice since the solver always passes its own cutoff, but the public docstring should state that edges with abs(Q[i,j]+Q[j,i]) <= cutoff are dropped, so users calling qmatrix_permutation directly understand the thresholding. src/solver.jl qmatrix_permutation docstring.

--- Questions ---

  1. In the inline one-variable block, the degenerate case (both bit values optimal) uses the "full" state label and bond_dim is logged as 1. Is sample over the "full" Qudit state guaranteed to return only 0/1 product bitstrings with the intended uniform probability, and is [x] in psi (via coeff) correct for both branches? I could not run this. Given item 1, the question may be moot if this block is removed in favor of _single_variable_minimize; if it is kept, please confirm behavior with a test that exercises the degenerate (["full"]) path. src/solver.jl head lines 324-343.

  2. The permutation convention (site s holds original variable permutation[s]; original_order does x[permutation] = bs; coeff does bs[permutation]) is internally consistent under static reading. Has this round-trip been validated against a non-identity permutation where the optimum is asymmetric, beyond the included 3-variable symmetric test, to guard against an inverse-permutation mistake? src/solution.jl original_order / coeff.

--- Summary ---

  1. Blocking: (a) PR conflicts with main and duplicates/diverges from the already-merged one-variable path (#49); (b) Closes #12 bundles unrelated one-variable scope; (c) changed behavior not shown green on a clean, mergeable tree.
  2. Nonblocking: dense O(n^2) adjacency scan over sparse Q; document cutoff thresholding in qmatrix_permutation.
  3. Questions: correctness of the degenerate "full" one-variable branch; permutation round-trip coverage for asymmetric/non-identity cases.
  4. Tests run and outcomes: none executed by me; git and julia were denied in this review environment, so the head tree could not be checked out and the suite could not be run. The PR-reported passes are unverified. Diff totals were verified via gh (+228/-5, 6 files) and the head sources were read via the contents API.
  5. Merge-readiness: not ready. The PR is in a CONFLICTING state and must be rebased onto current main, with the one-variable code removed/reconciled and the suite shown green, before it can be considered. The permutation feature itself appears sound on static reading.

I would not merge this until the blocking issues above are addressed.

Comment thread src/solver.jl
# Quantization
sites = ITensorMPS.siteinds(first, H; plev=0)

if length(sites) == 1

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking. This inline one-variable block collides with main, which already returns _single_variable_minimize(...) from the same if length(sites) == 1 point (added by #49, commit b67de66, merged 2026-06-11). gh pr view 33 reports CONFLICTING/DIRTY. Rebase onto current main and drop this block (and the corresponding test/qubo.jl "One variable" testset), keeping only the permutation feature that issue #12 asks for. If kept, reconcile it with _single_variable_minimize rather than duplicating it.

@bernalde bernalde Jun 14, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 2c9cf21. The branch was brought current with main, and the duplicate one-variable block plus weaker testset were dropped in favor of the existing _single_variable_minimize path and tests.

Comment thread src/solver.jl Outdated
labels = length(states) == 2 ? ["full"] : string.(states)
psi = MPS(sites, labels)
elapsed_time = time() - initial_time
dist = Solution{T}(psi, T[optimal], Int[1], Float64[elapsed_time], permutation)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question. The degenerate branch uses the "full" state label when both bit values are optimal. Please confirm that sampling and the coeff/in check behave correctly over the "full" Qudit state (uniform over {0,1}), ideally with a test exercising this path. Note this may be moot if the block is removed per the blocking comment above.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not applicable after 2c9cf21. The questioned branch-local one-variable block was removed; the existing main single-variable degeneracy test remains and passed in Pkg.test().

Comment thread src/solver.jl
function minimize(Q :: AbstractMatrix{T} , l :: Union{AbstractVector{T}, Nothing} = nothing , c :: T = zero(T); cutoff=1e-8, kwargs...) where T
H = tensorize(Q, isnothing(l) ? diag(Q) : diag(Q) + l; cutoff)
function minimize(Q :: AbstractMatrix{T} , l :: Union{AbstractVector{T}, Nothing} = nothing , c :: T = zero(T); cutoff=1e-8, preprocess::Bool=false, kwargs...) where T
Qp, lp, permutation = preprocess ? preprocess_qubo(Q, l, cutoff) : (Q, l, nothing)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Permutation wiring looks consistent on static reading: obj keeps the original Q/l, samples are unpermuted before obj is applied, and H is built from the permuted Qp/lp. Nonblocking: qmatrix_adjacency scans all O(n^2) pairs even though the QUBODrivers path passes a sparse Q; consider iterating stored nonzeros for large sparse inputs.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 2c9cf21. qmatrix_adjacency now builds candidate edges from nonzero matrix entries instead of scanning all pairs, and test/qubo.jl covers asymmetric/cutoff behavior.

@bernalde

Copy link
Copy Markdown
Member Author

Review comments addressed in 2c9cf218f134b6be718763fda47a31fc353ffd68.

Commits pushed:

Main changes:

  • Dropped this branch's duplicate one-variable solver block and weaker "One variable" testset, keeping the implementation and stronger tests already on main.
  • Expanded qmatrix_permutation docs to cover cutoff, return semantics, an example, and why the public default differs from the solve-path cutoff.
  • Changed qmatrix_adjacency to build candidate edges from nonzero matrix entries instead of scanning every pair.
  • Added cutoff/asymmetric-edge coverage for qmatrix_permutation.
  • Updated the PR body so it no longer claims this branch adds the one-variable solve path.

Tests run:

  • git diff --check origin/main...HEAD passed.
  • julia +1.12 --project=. -e 'using Pkg; Pkg.test()' passed.
  • julia +1.12 --project=docs/ docs/make.jl initially failed because the docs environment was not instantiated.
  • julia +1.12 --project=docs/ -e 'using Pkg; Pkg.instantiate()' passed.
  • julia +1.12 --project=docs/ docs/make.jl passed; Documenter skipped deployment outside CI.

Comments intentionally not addressed:

  • None of the Blocking comments were declined.
  • The question about the removed one-variable block is now not applicable to this PR. The main branch's single-variable degeneracy test remains in the suite and passed.

Remaining risks or follow-up:

  • Post-push CI is still running.
  • I did not benchmark the sparse preprocessing path; the change is limited to avoiding an all-pairs scan when nonzero positions are available.

@codecov

codecov Bot commented Jun 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.75000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.13%. Comparing base (975ef7d) to head (2c9cf21).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/solution.jl 90.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #33      +/-   ##
==========================================
+ Coverage   96.89%   98.13%   +1.24%     
==========================================
  Files           4        4              
  Lines         193      268      +75     
==========================================
+ Hits          187      263      +76     
+ Misses          6        5       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preprocessing: Implement Q-matrix permutation

1 participant