-
Notifications
You must be signed in to change notification settings - Fork 648
Remove ACVP test vectors from repository #2303
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
baentsch
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From just looking at the code, this seems OK to me. Thanks @xuganyu96 !
I chose to not use fixtures
Sounds sensible to me.
Currently the JSON files are fetched from the website instead of GitHub API. Should this be a concern?
My only concern would have been that that website may be unreachable when CI runs -- but given this is also on GH and not some US gov server (that may be subject to shutdown (in various meanings of the word :), I'd say it's highly unlikely for this to be a problem. My take for now: Let's see how this performs and whether we need to improve later (eg, using GH caching).
0795a83 to
9de3977
Compare
|
I also share @baentsch concerns about NIST server accessibility. It adds another variable to trouble shoot in case of a test failure. |
The test vectors are hosted on GitHub, not NIST server. I am open to discuss backup solutions, but for now I consider the cost of maintaining local copies of test vector to significantly outweigh the risk of test vectors becoming unavailable. |
9de3977 to
ad89810
Compare
Signed-off-by: Ganyu (Bruce) Xu <[email protected]>
Signed-off-by: Ganyu (Bruce) Xu <[email protected]>
…al because many files are downloaded twice Signed-off-by: Ganyu (Bruce) Xu <[email protected]>
Signed-off-by: Ganyu (Bruce) Xu <[email protected]>
Signed-off-by: Ganyu (Bruce) Xu <[email protected]>
Signed-off-by: Ganyu (Bruce) Xu <[email protected]>
ad89810 to
db482a6
Compare
The test vector JSON files are currently hosted on GitHub, not NIST website. It is true that GitHub might go down, but I think that this risk is insignificant, especially since if GitHub goes down, GitHub Actions will likely go down, in which case we are running into much bigger problems than a few ACVP test vectors being offline. |
Thanks, Bruce. I understand now. I have no further issues with this. |
This PR closes #2279
ACVP test vectors will now be fetched from
https://raw.githubusercontent.com/usnistgov/ACVP-Server/refs/tags/v1.1.0.40/gen-val/json-files/at test time using Python'srequestslibrary. Previously stored JSON files andfetch_values.shscripts were removed. I also ran a Python code formatter.Some considerations for reviewers:
pytest.fixtureso that a single instance of a resource is not fetched repeatedly across tests. However, from past experience I found fixtures to be a pain to debug, and all but one JSON file is used exactly once, so I chose to not use fixtures