Add build option for randomized ML-DSA signing

[M-AlNoaimi]

This PR adds an option to enable randomized signing for ML-DSA, as requested in open-quantum-safe/liboqs-python#128.

I've added a OQS_ENABLE_SIG_ML_DSA_RANDOMIZED_SIGNING build flag (defaulting to OFF) and a conditional test to verify that signatures are non-deterministic when the flag is enabled.

A small update to the CONFIGURE.md was done as well to note the flag.

[baentsch]

Thanks for the PR @M-AlNoaimi -- I'd be glad to get feedback to my single comments before approving this. And then I have 2 basic questions: 1) How and where does this change the actual logic in ML-DSA? 2) How shall users of the library recognize whether this config option has been set? Doing it this way seems like inviting a maintenance nightmare ("User: ML-DSA doesn't work! Maintainer: Which build flag did you set? User: How should I know? I only have a binary.")

[baentsch]

Oops -- seeing just now that @ashman-p approved this PR. In that case I assume I am unaware of some things and am willing to retract my review: In order for me to do so, would you please address my questions, Norm such as to not have my possible lack of understanding cause undue work to the PR's author?

[M-AlNoaimi]

Thanks for the PR @M-AlNoaimi -- I'd be glad to get feedback to my single comments before approving this. And then I have 2 basic questions: 1) How and where does this change the actual logic in ML-DSA?

These changes only introduces a build time option and a test, AFAIK the signing logic is only affected if the new option is enabled, in which case the ML-DSA impl will use randomized signing.

How shall users of the library recognize whether this config option has been set? Doing it this way seems like inviting a maintenance nightmare ("User: ML-DSA doesn't work! Maintainer: Which build flag did you set? User: How should I know? I only have a binary.")

I guess right now the only way they'd be able to tell was at build time, but I don't think it should be much of an issue if the default behaviour is set to OFF, and the new option keeps it that way as suggested on liboqs-python #128 So I think if it's off by default, current usage would stay the same.

[bhess]

Thanks for the PR!

1. How and where does this change the actual logic in ML-DSA?

I have the same question. From what I can tell, the OQS_ENABLE_SIG_ML_DSA_RANDOMIZED_SIGNING macro isn’t currently used in the ML-DSA implementation to control the switch between randomized and deterministic signing.

As far as I understand, the ML-DSA implementation already defaults to randomized signing. Is there any evidence that this wasn't the case?
The behavior is controlled by the following parts of the code:

liboqs/src/sig/ml_dsa/pqcrystals-dilithium-standard_ml-dsa-65_ref/config.h

Line 5 in 52169a1

#define DILITHIUM_RANDOMIZED_SIGNING

and

liboqs/src/sig/ml_dsa/pqcrystals-dilithium-standard_ml-dsa-65_ref/sign.c

Lines 229 to 234 in 52169a1

	#ifdef DILITHIUM_RANDOMIZED_SIGNING
	randombytes(rnd, RNDBYTES);
	#else
	for(i=0;i<RNDBYTES;i++)
	rnd[i] = 0;
	#endif

[baentsch]

So I think if it's off by default, current usage would stay the same.

Correct. But this PR makes the option available (thanks for the latest commit making it actually available!), so this can be used to change the behaviour of the library and there should be a way to query this also in a binary. Otherwise this invites a maintenance nightmare.

[bhess]

Thanks for adding the control logic in the last commit, @M-AlNoaimi. I left one inline comment.

[baentsch]

This approach feels "brittle" (but I may be paranoid here): For example, would it be possible to build everything with "-DOQS_ENABLE_SIG_ML_DSA_RANDOMIZED_SIGNING" (passed to the compiler during a re-build)? If so, this function will fail to return the right value. And/or would it be sensible to have cmake variable and C define have different names instead? And then there's still the possibility that header file and binary under test don't match up, e.g., because of some user-specific LD_LIBRARY_PATH setup...
I still feel it would be good for users to have a way (e.g., static variable) to query which way (randomized or deterministic) the library under test/use has been built; this test function could use the same facility to pick which way to test (and test the user-level "build-type" facility at the same time).