Skip to content

Fix broken ASV benchmarks#15587

Merged
gadial merged 1 commit intoQiskit:mainfrom
alexanderivrii:fix-asv
Jan 21, 2026
Merged

Fix broken ASV benchmarks#15587
gadial merged 1 commit intoQiskit:mainfrom
alexanderivrii:fix-asv

Conversation

@alexanderivrii
Copy link
Copy Markdown
Member

Summary

Some of our existing ASV benchmarks were broken. This commit fixes these (though alternatively I could have simply deleted some of the benchmarks as well).

Details and Comments

For TestCircuitManipulate: #13919 deleted the huge qasm file qv_N100_12345.qasm and modified the circuit_construction benchmark to construct the relevant quantum circuit instead of loading it from qasm. This PR applies the same fix to the manipulate benchmark as well.

For PassBenchmarks: the ApplyLayout pass is a transformation pass, and in particular changes the DAG from virtual to physical (which also changes the register names). If we want to save the DAG before the ApplyLayout pass (so that we can run the ApplyLayout to measure its performance), we should deepcopy.

For RoutedPassBenchmarks: self.routed_dag was not defined. Note that GateDirection/CheckGateDirection raise an error if the DAG's connectivity does not adhere to the coupling map. At some point in the past, the test included StochasticSwap to route the circuit (and define self.routed_dag in the first place), however the line got removed with the removal of stochastic swap. As we still need to route the circuit for this benchmark, I have reincluded the removed line with SabreSwap instead.

@alexanderivrii alexanderivrii requested a review from a team as a code owner January 20, 2026 07:43
@alexanderivrii alexanderivrii added type: qa Issues and PRs that relate to testing and code quality Changelog: None Do not include in the GitHub Release changelog. labels Jan 20, 2026
@qiskit-bot
Copy link
Copy Markdown
Collaborator

One or more of the following people are relevant to this code:

  • @Qiskit/terra-core

@alexanderivrii alexanderivrii added this to the 2.4.0 milestone Jan 20, 2026
@coveralls
Copy link
Copy Markdown

Pull Request Test Coverage Report for Build 21163412204

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 10 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.009%) to 87.926%

Files with Coverage Reduction New Missed Lines %
crates/qasm2/src/lex.rs 4 91.52%
crates/circuit/src/parameter/symbol_expr.rs 6 72.96%
Totals Coverage Status
Change from base Build 21151417354: 0.009%
Covered Lines: 100104
Relevant Lines: 113850

💛 - Coveralls

@gadial gadial self-assigned this Jan 20, 2026
Copy link
Copy Markdown
Contributor

@gadial gadial left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. My only concern is that I'm not sure how deterministic is the transpilation stage (not across different runs but across future qiskit versions) but since this was already done for the QASM benchmark it makes sense to follow suit.

@alexanderivrii
Copy link
Copy Markdown
Member Author

My only concern is that I'm not sure how deterministic is the transpilation stage (not across different runs but across future qiskit versions) but since this was already done for the QASM benchmark it makes sense to follow suit.

Thanks Gadi. This is a good point, and in fact I just commented on something very related in a different PR: see this comment, where the quality of the output circuit depends on the transpiler seed. My take on this is that we should be aware of this randomness and investigate outliers to see if they are not hiding a real bug.

@gadial gadial added this pull request to the merge queue Jan 21, 2026
Merged via the queue into Qiskit:main with commit 185e2b3 Jan 21, 2026
25 checks passed
@github-project-automation github-project-automation bot moved this from Ready to Done in Qiskit 2.4 Jan 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Changelog: None Do not include in the GitHub Release changelog. type: qa Issues and PRs that relate to testing and code quality

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants