Skip to content

tests: refactor tagged hash verification #1725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 16 additions & 25 deletions src/modules/ellswift/tests_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -405,31 +405,22 @@ void run_ellswift_tests(void) {

/* Test hash initializers. */
{
secp256k1_sha256 sha, sha_optimized;
static const unsigned char encode_tag[] = {'s', 'e', 'c', 'p', '2', '5', '6', 'k', '1', '_', 'e', 'l', 'l', 's', 'w', 'i', 'f', 't', '_', 'e', 'n', 'c', 'o', 'd', 'e'};
static const unsigned char create_tag[] = {'s', 'e', 'c', 'p', '2', '5', '6', 'k', '1', '_', 'e', 'l', 'l', 's', 'w', 'i', 'f', 't', '_', 'c', 'r', 'e', 'a', 't', 'e'};
static const unsigned char bip324_tag[] = {'b', 'i', 'p', '3', '2', '4', '_', 'e', 'l', 'l', 's', 'w', 'i', 'f', 't', '_', 'x', 'o', 'n', 'l', 'y', '_', 'e', 'c', 'd', 'h'};

/* Check that hash initialized by
* secp256k1_ellswift_sha256_init_encode has the expected
* state. */
secp256k1_sha256_initialize_tagged(&sha, encode_tag, sizeof(encode_tag));
secp256k1_ellswift_sha256_init_encode(&sha_optimized);
test_sha256_eq(&sha, &sha_optimized);

/* Check that hash initialized by
* secp256k1_ellswift_sha256_init_create has the expected
* state. */
secp256k1_sha256_initialize_tagged(&sha, create_tag, sizeof(create_tag));
secp256k1_ellswift_sha256_init_create(&sha_optimized);
test_sha256_eq(&sha, &sha_optimized);

/* Check that hash initialized by
* secp256k1_ellswift_sha256_init_bip324 has the expected
* state. */
secp256k1_sha256_initialize_tagged(&sha, bip324_tag, sizeof(bip324_tag));
secp256k1_ellswift_sha256_init_bip324(&sha_optimized);
test_sha256_eq(&sha, &sha_optimized);
secp256k1_sha256 sha_optimized;
{
unsigned char tag[] = "secp256k1_ellswift_encode";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8a17983:

fa67b67 can be relevant.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find! The commit message states "However, it requires exactly specifying the array size, which can be
cumbersome," but I don't think this is true.

Using the test program:

// repro.c
#include <stdio.h>

int main() {
    char str[] = "hello world";  // This should trigger the warning
    printf("%s\n", str);
    return 0;
}

I am able to compile with gcc14:

nix-shell --expr 'with import <nixpkgs> {}; mkShell.override { stdenv = overrideCC stdenv gcc14; }'
gcc -v
gcc -Wall -Wextra -Wpedantic -Werror repro.c -o out

and able to compile with gcc15:

nix-shell --expr 'with import <nixpkgs> {}; mkShell.override { stdenv = overrideCC stdenv gcc15; }'
gcc -v
gcc -Wall -Wextra -Wpedantic -Werror repro.c -o out

However, if I specify the array size, I can reproduce the error:

// repro.c
#include <stdio.h>

int main() {
    char str[11] = "hello world";  // This should trigger the warning
    printf("%s\n", str);
    return 0;
}

No error with:

nix-shell --expr 'with import <nixpkgs> {}; mkShell.override { stdenv = overrideCC stdenv gcc14; }'
gcc -Wall -Wextra -Wpedantic -Werror repro.c -o out

And an error with:

nix-shell --expr 'with import <nixpkgs> {}; mkShell.override { stdenv = overrideCC stdenv gcc15; }'
gcc -Wall -Wextra -Wpedantic -Werror repro.c -o out

repro.c: In function ‘main’:
repro.c:4:20: error: initializer-string for array of ‘char’ truncates NUL terminator but destination lacks ‘nonstring’ attribute (12 chars into 11 available) [-Werror=unterminated-string-initialization]
    4 |     char str[11] = "hello world";  // This should trigger the warning
      |                    ^~~~~~~~~~~~~
cc1: all warnings being treated as errors

Based on the above, I'd recommend we prefer the approach in this PR of not specifying the array size and perhaps document it as the preferred convention going forward? I find being able to specify the tag as a string to be much more reviewable than specifying the tag as an array of characters.

That being said, also happy to go the other way and update the musig tests to match the other modules if thats the preferred convention, as I think the main benefit is to have all of the modules follow the same convention.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To convince myself, I also verified with a few versions of clang, e.g.,:

nix-shell --expr 'with import <nixpkgs> {}; mkShell.override { stdenv = llvmPackages_16.stdenv; }'
clang -Wall -Wextra -Wpedantic -Werror -Wmost repro.c

Copy link
Contributor

@real-or-random real-or-random Aug 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@josibake The NUL byte resulting from char str[] = "hello world" does not hurt per se, but there are two minor issues with this: First, it's conceptually the wrong thing: If we want a char array, the simplest thing to do is to define a char array instead of a NUL-terminated string. Second and probably more relevant, it changes sizeof(str) to be 12 instead of 11. (See https://godbolt.org/z/da6PExKTh for demonstration. godbolt.org is the easiest way to test toy examples on many compilers.) We could, of course, accept this and always use sizeof(str) - 1, but it's easy to miss this.

edit: Sorry, I now saw that you're aware of the - 1 thing. And I agree, the ability to grep for the string is a good argument for the NUL-terminated string. If you ask me, I prefer to forego the grepability and define the right kind of object and have sizeof correct. But there's no definitive answer in the end.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@real-or-random thanks for the context! That explains the sizeof(str) - 1 for the musig examples. So it seems the choices are:

  1. Do something conceptually wrong for something that is slightly easier to review
  2. Do the conceptually correct thing for something that is slightly harder to review

"Slightly harder/easier" is a bit hand-wavy, but the fact that we used to specify the tags as strings (and the recently added musig also adopted this convention vs staying consistent with the existing modules) indicates option 1 is the more natural option. However, it likely needs an explainer, especially for why we are using sizeof(tag) - 1. On the flipside, I'm guessing option 2 feels more natural for reviewers who review/write a majority of the time in C?

Regardless of which convention is chosen, I do think its worth documenting in CONTRIBUTING.md. I'll add a commit for that once reviewers have weighed in on which convention they prefer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe godbolt.org/z/eKbT6sha4?

That still generates a warning if I add -Wextra.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe godbolt.org/z/eKbT6sha4?

That still generates a warning if I add -Wextra.

Right.

https://godbolt.org/z/n5rf5Y7cP

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, interesting, I wasn't aware of nonstring. That's another neat way.

Though when I think about it, I still prefer {'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd'}. Code is read much more often than it's written, so it makes sense to optimize reader (or reviewer) burden, and {'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd'} is immediately clear to a reviewer familiar with C. It's just a bit hard on the eyes, but there will be no need to look up macros or GNU extension attributes, etc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though when I think about it, I still prefer {'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd'}. Code is read much more often than it's written, so it makes sense to optimize reader (or reviewer) burden, and {'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd'} is immediately clear to a reviewer familiar with C. It's just a bit hard on the eyes, but there will be no need to look up macros or GNU extension attributes, etc.

Agreed. That's why I raised this point in the first place.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like 2 votes for keeping it as is, vs one vote to change it 😅 I'll update this PR tomorrow to instead convert the musig module to the existing convention, and add a note documenting the convention.

secp256k1_ellswift_sha256_init_encode(&sha_optimized);
sha256_tag_test_internal(&sha_optimized, (unsigned char*)tag, sizeof(tag) - 1);
}
{
unsigned char tag[] = "secp256k1_ellswift_create";
secp256k1_ellswift_sha256_init_create(&sha_optimized);
sha256_tag_test_internal(&sha_optimized, (unsigned char*)tag, sizeof(tag) - 1);
}
{
unsigned char tag[] = "bip324_ellswift_xonly_ecdh";
secp256k1_ellswift_sha256_init_bip324(&sha_optimized);
sha256_tag_test_internal(&sha_optimized, (unsigned char*)tag, sizeof(tag) - 1);
}
}
}

Expand Down
6 changes: 0 additions & 6 deletions src/modules/musig/tests_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -548,12 +548,6 @@ static void musig_nonce_test(void) {
}
}

static void sha256_tag_test_internal(secp256k1_sha256 *sha_tagged, unsigned char *tag, size_t taglen) {
secp256k1_sha256 sha;
secp256k1_sha256_initialize_tagged(&sha, tag, taglen);
test_sha256_eq(&sha, sha_tagged);
}

/* Checks that the initialized tagged hashes have the expected
* state. */
static void sha256_tag_test(void) {
Expand Down
33 changes: 13 additions & 20 deletions src/modules/schnorrsig/tests_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,8 @@ static void nonce_function_bip340_bitflip(unsigned char **args, size_t n_flip, s
}

static void run_nonce_function_bip340_tests(void) {
unsigned char tag[] = {'B', 'I', 'P', '0', '3', '4', '0', '/', 'n', 'o', 'n', 'c', 'e'};
unsigned char aux_tag[] = {'B', 'I', 'P', '0', '3', '4', '0', '/', 'a', 'u', 'x'};
unsigned char algo[] = {'B', 'I', 'P', '0', '3', '4', '0', '/', 'n', 'o', 'n', 'c', 'e'};
size_t algolen = sizeof(algo);
secp256k1_sha256 sha;
secp256k1_sha256 sha_optimized;
unsigned char nonce[32], nonce_z[32];
unsigned char msg[32];
Expand All @@ -36,19 +33,17 @@ static void run_nonce_function_bip340_tests(void) {
unsigned char *args[5];
int i;

/* Check that hash initialized by
* secp256k1_nonce_function_bip340_sha256_tagged has the expected
* state. */
secp256k1_sha256_initialize_tagged(&sha, tag, sizeof(tag));
secp256k1_nonce_function_bip340_sha256_tagged(&sha_optimized);
test_sha256_eq(&sha, &sha_optimized);

/* Check that hash initialized by
* secp256k1_nonce_function_bip340_sha256_tagged_aux has the expected
* state. */
secp256k1_sha256_initialize_tagged(&sha, aux_tag, sizeof(aux_tag));
secp256k1_nonce_function_bip340_sha256_tagged_aux(&sha_optimized);
test_sha256_eq(&sha, &sha_optimized);
/* Check that the initialized tagged hashes have the expected state */
{
unsigned char tag[] = "BIP0340/nonce";
secp256k1_nonce_function_bip340_sha256_tagged(&sha_optimized);
sha256_tag_test_internal(&sha_optimized, tag, sizeof(tag) - 1);
}
{
unsigned char tag[] = "BIP0340/aux";
secp256k1_nonce_function_bip340_sha256_tagged_aux(&sha_optimized);
sha256_tag_test_internal(&sha_optimized, tag, sizeof(tag) - 1);
}

testrand256(msg);
testrand256(key);
Expand Down Expand Up @@ -158,13 +153,11 @@ static void test_schnorrsig_api(void) {
/* Checks that hash initialized by secp256k1_schnorrsig_sha256_tagged has the
* expected state. */
static void test_schnorrsig_sha256_tagged(void) {
unsigned char tag[] = {'B', 'I', 'P', '0', '3', '4', '0', '/', 'c', 'h', 'a', 'l', 'l', 'e', 'n', 'g', 'e'};
secp256k1_sha256 sha;
unsigned char tag[] = "BIP0340/challenge";
secp256k1_sha256 sha_optimized;

secp256k1_sha256_initialize_tagged(&sha, (unsigned char *) tag, sizeof(tag));
secp256k1_schnorrsig_sha256_tagged(&sha_optimized);
test_sha256_eq(&sha, &sha_optimized);
sha256_tag_test_internal(&sha_optimized, tag, sizeof(tag) - 1);
}

/* Helper function for schnorrsig_bip_vectors
Expand Down
7 changes: 7 additions & 0 deletions src/tests.c
Original file line number Diff line number Diff line change
Expand Up @@ -609,6 +609,13 @@ static void test_sha256_eq(const secp256k1_sha256 *sha1, const secp256k1_sha256
CHECK(sha1->bytes == sha2->bytes);
CHECK(secp256k1_memcmp_var(sha1->s, sha2->s, sizeof(sha1->s)) == 0);
}
/* Convenience function for using test_sha256_eq to verify the correctness of a
* tagged hash midstate. This function is used by some module tests. */
static void sha256_tag_test_internal(secp256k1_sha256 *sha_tagged, unsigned char *tag, size_t taglen) {
secp256k1_sha256 sha;
secp256k1_sha256_initialize_tagged(&sha, tag, taglen);
test_sha256_eq(&sha, sha_tagged);
}

static void run_hmac_sha256_tests(void) {
static const char *keys[6] = {
Expand Down