-
Notifications
You must be signed in to change notification settings - Fork 2
Translation validator #518
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ailrst
wants to merge
187
commits into
main
Choose a base branch
from
tv-redo
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
commit c087241 Merge: 70b7b49 ac7f875 Author: Alistair Michael <[email protected]> Date: Mon Jun 16 16:54:15 2025 +1000 Merge remote-tracking branch 'origin/main' into tv-minus-tv commit ac7f875 Author: Kait <[email protected]> Date: Mon Jun 16 16:20:59 2025 +1000 Ctags setup to enable go-to-definition and listing symbols (#459) [Ctags](https://en.wikipedia.org/wiki/Ctags) is an ancient piece of software that uses regular expressions to implement go-to-definition for virtually any programming language. This can be somewhat unreliable, but it is universally-available and very robust. This is very useful for me, personally, because I don't use the scala LSP. This PR adds `./mill ctags` to generate the ctags file in the root of the repository. Once generated, this can be read by any editor which supports ctags. I've written more in [editor-setup](https://github.com/UQ-PAC/BASIL/blob/5e8d1bb97fb4ff0211a2ed061a5c6e31bb3f3c61/docs/development/editor-setup.md#ctags), and there is also an example setup. Using it looks like this:  commit 0968142 Author: Kait <[email protected]> Date: Mon Jun 16 16:03:38 2025 +1000 ci: use scalatest runner directly (again) (#447) this brings back the old scalatest.sh which directly invokes scalatest's runner. now that there's a case fallback if MILL_WORKSPACE_ROOT is unset, this seems to work fine and leads to the best behaviour: tests are run all in one thread, allowing for the best runtime performance and correctly-sorted output. for reference: - commit before mill 12.10 and scalatest.sh removal 55587f0 - my unfortunate suggestion to use `./mill test` directly #421 (comment) - hacky band-aid by re-sorting output. this worked but it was slow. #429 i'm sorry for the trouble lol commit bbcdacb Author: am <[email protected]> Date: Mon Jun 16 15:14:57 2025 +1000 Make --relf optional (#448) Make --relf optional for tasks not requiring symbols. The symbol table is only really required for the spec input and DSA heuristics and the relf parser tends to be by far the most fragile part of the frontend. When relf is not provided look for `--main-procedure-name` (default 'main') in the procedure list to set as Program.mainProcedure. This is previously all we do to detect the entrypoint, except with the relf symbol table to find a procedure address to search for. I think eventually we should be able to retrieve all the same information from the gtirb `Module.symbols` but that's a bigger task. --------- Co-authored-by: rina <[email protected]> commit 920b9c1 Author: Kait <[email protected]> Date: Mon Jun 16 12:15:12 2025 +1000 allow TempIf in the middle of blocks, given unique branching behaviour (#457) allows TempIf to occur in the middle of blocks, but only if both sides of the tempif branch or both sides do not branch. this allows handling of, for example, `casal w0, w4, [x3]` 0x88e0fc64 which looks like this: ``` AtomicStart.0 {{ }} ( ) ; constant bits ( 32 ) Exp16__6 = Mem.read.0 {{ 4 }} ( __array _R [ 3 ],4,9 ) ; if eq_bits.0 {{ 32 }} ( Exp16__6,__array _R [ 0 ] [ 0 +: 32 ] ) then { Mem.set.0 {{ 4 }} ( __array _R [ 3 ],4,9,__array _R [ 4 ] [ 0 +: 32 ] ) ; } AtomicEnd.0 {{ }} ( ) ; __array _R [ 0 ] = ZeroExtend.0 {{ 32,64 }} ( Exp16__6,64 ) ; ``` i wish i could write a unit test for this, but alas... Fixes: #454 commit 96648b2 Author: Kait <[email protected]> Date: Mon Jun 16 11:18:44 2025 +1000 fix --load-directory when used with filenames with multiple dots (#455) assuming you had files atomic.-O0.gts atomic.-O0.relf atomic.-O0.spec in a directory, previously, this would not work: ./mill run --load-directory-gtirb atomic/atomic/atomic.-O0.gts the logic would incorrectly replace /two/ levels of file extension and look for files named atomic.gts and atomic.relf. this fixes the logic to only strip off one file extension. a test case is this zip from Aaron: https://github.com/user-attachments/files/20717350/atomic.zip commit 1d165f1 Author: Kait <[email protected]> Date: Fri Jun 13 15:12:56 2025 +1000 fix bug when using --load-directory with a filename in the current dir (#456) previously, when running ./mill run --load-directory-gtirb asdf.gts to load files from the current directory, you would see an error: Exception in thread "main" java.lang.NullPointerException: Cannot invoke "java.nio.file.Path.getFileName()" because the return value of "java.nio.file.Path.getParent()" is null at Main$.loadDirectory(Main.scala:45) at Main$.main(Main.scala:283) at Main.main(Main.scala) this pr fixes using --load-directory without any directory prefix, by assuming the current working directory. previously, this could also be worked around by using ./mill run --load-directory-gtirb ./asdf.gts commit ca1bbf0 Author: am <[email protected]> Date: Fri Jun 13 13:30:05 2025 +1000 assume&prove registers are preserved across calls (#430) * assume&prove registers are preserved across calls --simplify now adds an assertion requiring 1. callee-saved registers r19-r28 and FP (R29) are preserved (required by ABI) 2. Return-address, LR (R30) are preserved (assuming well-behaved stack and return structure) and removes the corresponding registers from the out-param list. * enable by default only when dsa enabled commit f0db936 Author: am <[email protected]> Date: Fri Jun 13 13:28:22 2025 +1000 use hashmap to improve dsl loading perf by 2000% (#450) Caches the `label -> block` and `name -> procedure` mapping to avoid linear search. The tests just reconstruct this cache every time as they usually modify the procedure and incrementally resolve. commit 9f0ef5e Author: Kait <[email protected]> Date: Fri Jun 13 11:18:11 2025 +1000 docs: disable on-click link handler to improve perf and reliability (#453) scala has a long precedent of thinking that it's new implementations are better than historically-proven technologies. here, we see this hubris in the simple act of clicking a hyperlink in the docs page. go to a page in the (new!) online basil api docs, [this one for example](https://uq-pac.github.io/BASIL/api/basil/ir/dsl/EventuallyIndirectCall.html). find a link which would take you somewhere else in the same documentation, for example `Variable`. click the link. observe how nothing happens for a second, then the page changes (if you're lucky). you are witnessing Scala's custom Javascript dispatch a request to the next page and then manually stitch it into the current webpage. and that's _if_ it works. i have also observed the on-click handler throwing a Javascript exception and fail to perform *any* navigation. this is a miserable experience for the person reading documentation. at time of writing, going [to this page](https://uq-pac.github.io/BASIL/api/basil/translating/BAPLoader$.html#checkCondition-25c) and clicking on ExpContext will do nothing. this works with a ctrl+click, which presumably bypasses the custom handler. this PR corrects this self-inflicted issue by replacing the query for all links in a page, `document.querySelectorAll("a")`, with an empty list. this prevents any on-click handlers from being registered to links. the links now work noticeably faster and with no chance of failing mysteriously. one small drawback is that the state of the tree view in the left sidebar is not persisted across link clicks. maybe this can be fixed if i care to properly fix their javascript. in the meantime, this is a small price to pay to restore an internet browser feature which has existed since netscape navigator. commit d9f60a0 Author: Kait <[email protected]> Date: Thu Jun 12 22:11:31 2025 +1000 rewrite Twine to improve performance of ToScala by 560% (#451) * Twine: migrate to bespoke AST for twines commit 2a94278 Author: Kait <[email protected]> Date: Thu Jun 12 21:16:28 2025 +1000 docs: fix cross-linking to bnfc javadocs (#452) previously, in pages like https://uq-pac.github.io/BASIL/api/basil/ir/parsing/BasilEarlyBNFCVisitor.html, the links to bnfc-generated classes like Visitor would not work. this fixes those links. commit 2ca272f Author: Kait <[email protected]> Date: Wed Jun 11 14:23:48 2025 +1000 Add Basil IR parser using BNFC-generated visitor (#441) this allows the loading of Basil IR files and replaces the old EvalScala feature. try it out! ```bash ./mill runMain ir.parsing.Run basic-keep-pc-after-analysis.il ``` more docs: https://github.com/UQ-PAC/BASIL/blob/54df2a05e3dfa474d4e8bb717c17058177e7d6c6/src/main/scala/ir/parsing/package.scala#L4-L70 **todo** - [x] **important:** the grammar used is a little bit outdated. for example, it expects bveq/inteq instead of the newly implemented polymorphic eq. - [ ] it also isn't updated with the sigils in #435. - [x] fix `var procName = "none proc name"` hack to avoid mutable variable. - [x] add more docs - [x] **fix issues marked in commit comments** - [x] warn on var (var - [x] parse error handling - [x] migrate to jflex/cup and write mill module #442 - [x] test cases lol - [ ] parsing of ir files with every operator - [ ] error cases and messages - [x] round trip of serialiser - [x] fix the docs after significantly complicating the visitor structure - [x] connect to basil pipeline through a --input argument --------- Co-authored-by: Alistair Michael <[email protected]> commit 70b7b49 Author: alistair <[email protected]> Date: Mon Jun 9 15:58:57 2025 +1000 wip: include globals for params I think the best way forward here is to make the param tv tweak the the monadic to match target and source so that the axiom/ackermann can stay the same. This would mean using the source as the point of comparison, and making dropping the _in / _out for the target global registers. Hence the functions inherently match up. This is sufficient as for every procedure the modifications have to be sufficient to prove the inv, so if the param list is incorrect you'll lose inductivity etc. Otherwise, bookkeeping is required to match parameters between source and target via a mapping on the original paramters produced by param analysis. commit 7936052 Merge: 3414227 b9c8b20 Author: l-kent <[email protected]> Date: Fri Jun 6 10:52:34 2025 +1000 Merge pull request #446 from UQ-PAC/fix-computeDomain computeDomain non-determinism commit a034bb4 Author: Alistair Michael <[email protected]> Date: Thu Jun 5 10:05:27 2025 +1000 qf_bv commit 20eb551 Author: Alistair Michael <[email protected]> Date: Wed Jun 4 15:01:56 2025 +1000 totalise asserts commit b9c8b20 Author: l-kent <[email protected]> Date: Wed Jun 4 13:05:02 2025 +1000 hopefully fix non-determinism in computeDomain commit daf090e Merge: e0689fd 308b65e Author: Alistair Michael <[email protected]> Date: Wed May 28 12:22:57 2025 +1000 Merge remote-tracking branch 'origin/main' into trans commit e0689fd Merge: 05f20b7 818ebb0 Author: Alistair Michael <[email protected]> Date: Thu May 22 14:51:44 2025 +1000 Merge remote-tracking branch 'origin/tv-reduce-passify' into trans commit 05f20b7 Merge: 6f0722a b45683f Author: Alistair Michael <[email protected]> Date: Thu May 22 14:45:16 2025 +1000 Merge remote-tracking branch 'origin/main' into trans commit 6f0722a Author: Alistair Michael <[email protected]> Date: Thu May 22 14:23:30 2025 +1000 logging commit 818ebb0 Author: Alistair Michael <[email protected]> Date: Thu May 22 14:21:04 2025 +1000 rm onlyone commit 7a52744 Author: Alistair Michael <[email protected]> Date: Wed May 21 19:15:01 2025 +1000 fixes commit 6d6e94f Author: Alistair Michael <[email protected]> Date: Tue May 20 16:45:04 2025 +1000 OnCrash commit 48b48af Author: Alistair Michael <[email protected]> Date: Tue May 20 16:40:06 2025 +1000 fix issues in smt encoding commit ddc2f61 Merge: 0d5ed38 2282242 Author: Alistair Michael <[email protected]> Date: Thu May 15 11:07:28 2025 +1000 Merge branch 'simple-copyprop' into trans commit 0d5ed38 Author: Alistair Michael <[email protected]> Date: Thu May 15 11:07:25 2025 +1000 extend extract on main simp pass commit 9806917 Author: Alistair Michael <[email protected]> Date: Wed May 14 17:36:47 2025 +1000 fix dead var elim commit 01fba93 Merge: 92736fb 59f1f2b Author: Alistair Michael <[email protected]> Date: Wed May 14 16:11:22 2025 +1000 Merge remote-tracking branch 'origin' into trans commit 92736fb Merge: 569393e eebe343 Author: Alistair Michael <[email protected]> Date: Wed May 14 16:01:11 2025 +1000 Merge remote-tracking branch 'origin' into trans commit 569393e Author: Alistair Michael <[email protected]> Date: Wed May 14 16:00:55 2025 +1000 simplify extends/slices using known bits commit b8498ac Merge: 1640c90 6670cdf Author: Alistair Michael <[email protected]> Date: Tue May 13 14:41:52 2025 +1000 Merge branch 'main' into trans commit 3b4d31f Author: Alistair Michael <[email protected]> Date: Tue May 6 17:16:01 2025 +1000 smt encoding work commit 5365a94 Author: Alistair Michael <[email protected]> Date: Tue May 6 11:17:18 2025 +1000 partial flow sensitivity commit e83f8af Author: Alistair Michael <[email protected]> Date: Tue May 6 15:08:25 2025 +1000 fixes commit 2282242 Author: Alistair Michael <[email protected]> Date: Tue May 6 11:17:18 2025 +1000 partial flow sensitivity commit a0750ad Author: alistair <[email protected]> Date: Sun May 4 20:41:23 2025 +1000 partial SSA for product program dag Not quite working; invariant needs renaming for each cut individually, using the cut point's renaimg set. This needs more structure retained in the invariant. commit 1640c90 Author: Alistair Michael <[email protected]> Date: Fri May 2 17:02:35 2025 +1000 cleanup commit 300c32c Author: Alistair Michael <[email protected]> Date: Fri May 2 16:51:00 2025 +1000 make tv a separate pass commit f89d065 Author: Alistair Michael <[email protected]> Date: Fri May 2 16:00:41 2025 +1000 support simulassign, fix warnings commit bd56587 Author: Alistair Michael <[email protected]> Date: Fri May 2 15:09:41 2025 +1000 update gamamdomain for simulassign commit e49fd31 Merge: e93941e eedf2ad Author: Alistair Michael <[email protected]> Date: Fri May 2 10:38:51 2025 +1000 Merge branch 'simple-copyprop' into trans commit e93941e Merge: d006b6b 151ef75 Author: Alistair Michael <[email protected]> Date: Fri May 2 10:38:08 2025 +1000 Merge branch 'main' into trans commit eedf2ad Author: Alistair Michael <[email protected]> Date: Thu May 1 16:20:40 2025 +1000 disable dsa by default for simplify commit 62f656f Author: Alistair Michael <[email protected]> Date: Thu May 1 16:20:31 2025 +1000 add prop for base+offset commit 17626a1 Author: Alistair Michael <[email protected]> Date: Thu May 1 14:49:09 2025 +1000 implement standard copyprop commit d006b6b Author: Alistair Michael <[email protected]> Date: Tue Apr 29 17:19:54 2025 +1000 try only split source commit 234a52b Author: Alistair Michael <[email protected]> Date: Tue Apr 29 14:38:46 2025 +1000 fix rd commit 55656e7 Author: alistair <[email protected]> Date: Fri Apr 18 21:44:59 2025 +1000 ackermann fix traversal commit 7eab90f Author: alistair <[email protected]> Date: Fri Apr 18 21:02:53 2025 +1000 ackermann fixes commit 7541c57 Author: Alistair Michael <[email protected]> Date: Thu Apr 24 13:01:31 2025 +1000 fix ack commit 8135309 Author: Alistair Michael <[email protected]> Date: Thu Apr 24 10:47:58 2025 +1000 remvoe dominator use commit 5fbdd42 Author: alistair <[email protected]> Date: Fri Apr 18 16:00:57 2025 +1000 ackermannisation attempt commit ab126e5 Author: Alistair Michael <[email protected]> Date: Thu Apr 17 10:43:10 2025 +1000 tv calls simulassign commit 7463e3f Author: Alistair Michael <[email protected]> Date: Wed Apr 16 17:34:51 2025 +1000 tweaks commit f3562b3 Author: Alistair Michael <[email protected]> Date: Wed Apr 16 16:05:33 2025 +1000 fix splitting commit 6beb1c5 Author: alistair <[email protected]> Date: Wed Apr 16 01:16:26 2025 +1000 fix commit e342521 Author: alistair <[email protected]> Date: Wed Apr 16 01:02:04 2025 +1000 idk commit ecfc6d2 Author: alistair <[email protected]> Date: Tue Apr 15 23:57:50 2025 +1000 fixes commit 53dd843 Author: alistair <[email protected]> Date: Tue Apr 15 23:14:41 2025 +1000 flat boolean expr commit 78516f7 Author: Alistair Michael <[email protected]> Date: Tue Apr 15 18:26:14 2025 +1000 auto splits commit e3b87b5 Author: Alistair Michael <[email protected]> Date: Tue Apr 15 17:04:47 2025 +1000 wpconj encoding commit 384240b Author: Alistair Michael <[email protected]> Date: Wed Apr 9 13:32:08 2025 +1000 x commit 65b0cf1 Author: Alistair Michael <[email protected]> Date: Tue Apr 8 16:49:24 2025 +1000 wip value analysis commit 685ebeb Author: Alistair Michael <[email protected]> Date: Fri Apr 4 16:54:54 2025 +1000 restore conds example commit 54d0b11 Author: Alistair Michael <[email protected]> Date: Fri Apr 4 16:49:07 2025 +1000 incl axioms in query commit 727d3f0 Author: Alistair Michael <[email protected]> Date: Fri Apr 4 16:44:43 2025 +1000 simulassign commit da855f3 Author: Alistair Michael <[email protected]> Date: Wed Apr 2 17:18:54 2025 +1000 dsa validation commit f0b1a23 Author: Alistair Michael <[email protected]> Date: Tue Apr 1 16:55:44 2025 +1000 custom trace sort commit 158a464 Author: Alistair Michael <[email protected]> Date: Mon Mar 31 16:14:57 2025 +1000 cleanup commit c6dfdf8 Author: Alistair Michael <[email protected]> Date: Mon Mar 31 14:25:32 2025 +1000 extract z3 query from boogie and run separately non-incremental commit 9a9148c Author: Alistair Michael <[email protected]> Date: Mon Mar 31 11:21:27 2025 +1000 fix loop splitting and inv gen commit 6119f8a Author: alistair <[email protected]> Date: Sat Mar 29 18:05:10 2025 +1000 split invariant by cut commit 4482a32 Author: alistair <[email protected]> Date: Sat Mar 29 12:18:00 2025 +1000 dead store in example commit 6e4a579 Author: alistair <[email protected]> Date: Sat Mar 29 12:13:17 2025 +1000 improve memory encoding commit a166e44 Author: alistair <[email protected]> Date: Sat Mar 29 01:28:30 2025 +1000 translation validation test commit dfebbc4 Author: Alistair Michael <[email protected]> Date: Fri Mar 28 16:59:26 2025 +1000 convert proc to transitino system
this lets us exclude read-uninit variables which are unreachable.
be aware of limitations of the RemoveUnreachableBlocks method.
/**
* Very coarse transform pass to remove obviously unreachable
* blocks.
*
* This only removes in the most simple cases.
* Does NOT remove:
* - mutually-recursive unreachable blocks,
* - blocks only reachable from unreachable blocks.
* - any blocks in the presence of indirect calls.
*/
object RemoveUnreachableBlocks {
these generate dsa variables up to PC_1600 and this would exceed the old 1000 fuel limit.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This draft pull request adds a translation validated simplifier pipeline enabled by the
--simplify-tvflag. This has mostly rewritten from the last iteration but probably still needs work.See:
ir/transforms/validate/SimplifyPipeline.scalafor the supported transforms.This incarnation directly generates QF_BV SMT queries for validataion, as testing showed this is substantially more performant. Similarly we use CVC5 over Z3 as this works better on these queries.
The smt interface is provided by javasmt, and we now support basic model extraction, in the case of validation failure you get input like:
In addition to the above, this implementation is improved over the previous iteration by supporting basic interprocedural reasoning (a basic interprocedural analysis lifts call side effects to procedure-local effects): this is sufficient to validate the Parameter inference transform.
To Do
extract smt query from JavaSMT to avoid duplicating query buildit doesn't really provide much to support this--simplify-tvpasses on cntlm-nodukfix solver timeout; seems to ignore request shutdown (if its even firing), cvc5 should be able to take a timeout flag throughits down the stackfurtherOptionscvc5.tlimit.assert(sat(invariant)),assert(sat(not(invariant))--tv-assertKnown Design Issues
The SSA transform is extremely slow because it tracks all defined variables and unions the map on every joinSideEffectStatementis a somewhat tortured abstraction (extends NOP statement): really the DAG CFA needs to be its own graph IR with edges of only assignment and monadic side effect statements, but I don't want to write a new graph implementation and iteratorWe haven't implemented the case analysis functionality of the previous implementation -- seems to not be super neccessary due to more effective smt representationbasic form of this has been implemented