Add JSON Test Vectors #90

siv2r · 2025-03-30T13:06:06Z

Addresses issue #35.

This PR introduces functionality for generating and validating JSON test vectors for ChillDKG’s public APIs.

Summary:

Implements Design 3, with assertions for intermediate messages.
Modules in python/vectors_generator/ generate one JSON vector file per API, saving them in the python/vectors/.
Modifies chilldkg_ref:
1. To ensure reproducible test vectors, this PR temporarily uses a zeroed aux_rand value for schnorr_sign
2. Adds the GE.from_bytes_with_infinity method.
Adds tests within tests.py to load the generated JSON vectors and verify API outputs for correctness.

- Add GE.from_bytes_with_infinity method - In encpedpop, unpack simplpedpop's error message from tuple - Temporarily use zeroed 32-byte array for aux_rand arg of schnorr_sign See: BlockstreamResearch#86

outputs one JSON file per API in `python/vectors` folder

jonasnick

Nice, thank you! This looks like "design 3" as I had it in mind.

I was surprised, shouldn't this be based on #88? I think the test vectors should contain the messages in their serialized form.

jonasnick

Posting a status update, still reviewing...

jonasnick · 2025-04-25T16:42:16Z

python/vector_generator/coordinator.py

+            "cmsg1": cmsg1_asdict(cmsg1),
+            "pmsgs2": [pmsg2_asdict(m) for m in invalid_pmsgs2],
+            "error": error,
+            "comment": "invalid certificate length",


wrong comment

jonasnick · 2025-04-25T16:43:11Z

python/vector_generator/participant.py

+            "pmsg1": pmsg1_asdict(pmsgs1[0]),
+            "cmsg1": cmsg1_asdict(invalid_cmsg1),
+            "error": error,
+            "comment": "coms_to_secret list in cmsg1 has an invalid value at index 0",


wrong comment

jonasnick · 2025-04-25T16:43:30Z

python/vector_generator/participant.py

+            "pmsg1": pmsg1_asdict(pmsgs1[0]),
+            "cmsg1": cmsg1_asdict(invalid_cmsg1),
+            "error": error,
+            "comment": "coms_to_secret list in cmsg1 has an invalid value at index 0",


wrong comment

jonasnick · 2025-04-25T16:44:42Z

python/vector_generator/others.py

+def generate_hostpubkey_vectors():
+    vectors = {"valid_test_cases": [], "error_test_cases": []}
+
+    # --- Valid test case 1 ---


nit: can we start numbering with 0? The test cases are stored in a list, where the conventional first index is 0.

jonasnick · 2025-04-25T16:45:19Z

python/vector_generator/others.py

+    rand_encshare = bytes.fromhex(
+        "FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141"
+    )


This rand_encshare doesn't look rand(om). Should be invalid_encshare?

Ah yes, thanks.

jonasnick · 2025-04-25T16:45:35Z

python/vector_generator/others.py

+            "hostseckey": None,
+            "recovery_data": bytes_to_hex(invalid_crec),
+            "error": error,
+            "comment": "invalid threshold",


wrong comment

jonasnick · 2025-04-25T16:49:16Z

python/vector_generator/others.py

+    # --- Error test case 4: invalid threshold ---
+    invalid_crec = b"\x00" * 4 + crec[4:]


Suggested change

# --- Error test case 4: invalid threshold ---

invalid_crec = b"\x00" * 4 + crec[4:]

# --- Error test case 4: invalid threshold ---

invalid_crec = b"\x00" * 4 + crec[4 + 33 * t:]

Otherwise, recover will fail due to a wrong length of the recovery data, not because the threshold is wrong in params_validate (which presumably was the intention).

And we need to define t

t = 2 params = chilldkg.SessionParams(hostpubkeys, t)

at the top of the function.

jonasnick · 2025-04-25T16:50:31Z

python/vector_generator/others.py

+            "hostseckey": None,
+            "recovery_data": bytes_to_hex(invalid_crec),
+            "error": error,
+            "comment": "last pubnonce in the pubnonces list is invalid",


This comment is wrong or at least misleading. The pubnonce is not invalid by itself, but recover fails because this pubnonce hasn't been signed. Which one did you want to test?

jonasnick · 2025-04-25T16:53:21Z

python/vector_generator/others.py

+            "comment": "last pubnonce in the pubnonces list is invalid",
+        }
+    )
+    # --- Error test case 6: last signature in the certificate is invalid ---


Duplicate test case number

jonasnick · 2025-04-25T16:54:32Z

python/vector_generator/participant.py

+            "comment": "coms_to_secret list in cmsg1 has an invalid value at index 0",
+        }
+    )
+    # --- Error Test Case 4: pop list in cmsg1 has an invalid value at index 1 ---


Duplicate test case number

siv2r · 2025-04-26T11:52:19Z

Thanks for the review. I'll address your comments soon.

I was surprised, shouldn't this be based on #88? I think the test vectors should contain the messages in their serialized form.

I assumed we were favoring design 2 over design 1. That is why the JSON representation of messages shows internal fields (for better readability) rather than a single byte array. The definitions of design 1 and design 2 are in this comment as:

We have two main designs for our custom types in the JSON:

Design 1: Represent it as a single large hex string (a fully serialized byte array).

Design 2: Use a nested JSON structure that shows all internal fields.

I can switch the message JSON to a single hex-encoded byte array if you prefer.

jonasnick · 2025-04-29T11:50:56Z

I had thought design 1 vs. design 2 was mostly about representing the state objects. As for the messages that are passed between the participants, I believe it's better to include them in the test vectors in their serialized form. This way, the byte encoding is tested as well, which is an important part of the spec.

jonasnick · 2025-05-01T13:37:19Z

python/vector_generator/participant.py

+            "comment": "participant 1 sent an invalid secshare for participant 0",
+        }
+    )
+    # --- Error Test Case 2: Coordinator tampers with participant 0's encrypted secshare ---


Suggested change

# --- Error Test Case 2: Coordinator tampers with participant 0's encrypted secshare ---

# --- Error Test Case 2: Coordinator tampered with participant 0's encrypted secshare ---

we use past tense everywhere else

jonasnick · 2025-05-01T13:37:36Z

python/vector_generator/participant.py

+            "cmsg1": cmsg1_asdict(invalid_cmsg1),
+            "cinv_msg": cinv_msg_asdict(cinv_msgs[0]),
+            "error": error,
+            "comment": "coordinator tampers with participant 0's encrypted secshare",


Suggested change

"comment": "coordinator tampers with participant 0's encrypted secshare",

"comment": "coordinator tampered with participant 0's encrypted secshare",

jonasnick · 2025-05-01T13:38:08Z

python/vector_generator/participant.py

+        }
+    )
+
+    # --- Error test case 1: Wrong hostseckey length ---


what about wrong randomness length?

jonasnick · 2025-05-01T13:39:07Z

python/vector_generator/participant.py

+        }
+    )
+
+    # --- Error Test Case 1: Wrong host secret key length ---


why is this commented out?

jonasnick · 2025-05-01T13:41:08Z

python/vector_generator/participant.py

+    # })
+    # --- Error Test Case 2: pubnonces list in cmsg1 has an invalid value at index 0 ---
+    invalid_cmsg1 = copy.deepcopy(cmsg1)
+    invalid_cmsg1.enc_cmsg.pubnonces[0] = b"\xeb" * 32  # random pubnonce


Is this a valid pubnonce? I think if possible for every GE we should have a test where the GE is wrong and a test where the GE is infinity. For every serialized GE, we should also have a test where the byte array does not correspond to a GE. Similarly for scalars. This also applies to other test vectors in other files.

jonasnick · 2025-05-01T13:43:52Z

python/vector_generator/participant.py

+    # --- Error Test Case 4: pop list in cmsg1 has an invalid value at index 1 ---
+    invalid_cmsg1 = copy.deepcopy(cmsg1)
+    invalid_cmsg1.enc_cmsg.simpl_cmsg.pops[1] = bytes.fromhex(
+        "09C289578B96E6283AB13E4741FB489FC147FB1A5F446A314BA73C052131EFB04B83247A0BCEDF5205202AD64188B24B0BC5B51A17AEB218BD98DBE000C843B9"


Is this a valid signature but for the wrong public key? Please document.

jonasnick · 2025-05-01T13:47:31Z

python/vector_generator/participant.py

+            "comment": "host secret key doesn't match any hostpubkey",
+        }
+    )
+    # --- Error test case 3: Invalid threshold ---


Can we order the test cases such that they roughly correspond to the location in the code where the assertions are raised (also in other files)? This would help with review. For example, the threshold is checked in params_validate before the check for matching hostpubkeys (currently error test case 2).

jonasnick · 2025-05-01T14:06:15Z

python/vector_generator/coordinator.py

+import chilldkg_ref.chilldkg as chilldkg
+
+
+def generate_coordinator_step1_vectors():


I feel like we need to test more here (which is also related to message serialization). For example, what if the VSS commitment contains invalid values or has the wrong length.

jonasnick · 2025-05-01T15:10:25Z

python/tests.py


+def test_hostpubkey_gen_vectors():
+    input_file = Path("vectors/hostpubkey_gen_vectors.json")
+    with open(input_file) as f:


When run from the parent directory via python python/tests.py this gives me an error:

FileNotFoundError: [Errno 2] No such file or directory: 'vectors/hostpubkey_gen_vectors.json'

python python/tests.py works fine on master

jonasnick · 2025-05-01T15:13:31Z

python/vector_generator/coordinator.py

+    invalid_pmsgs1[1].enc_pmsg.enc_shares.pop()
+    error = expect_exception(
+        lambda: coordinator_step1(invalid_pmsgs1, params),
+        chilldkg.FaultyParticipantOrCoordinatorError,  # REVIEW: why is this working? shouldn't it raise an error saying encpedpop.FaultyParticipantOrCoordinatorError was raised instead?


Why would it raise "encpedpop.FaultyParticipantOrCoordinatorError"? FaultyParticipantOrCoordinatorError is defined in util.py and it's part of the chilldkg module.

siv2r added 3 commits March 30, 2025 14:01

python: modify chilldkg_ref for test vectors gen

68b00ac

- Add GE.from_bytes_with_infinity method - In encpedpop, unpack simplpedpop's error message from tuple - Temporarily use zeroed 32-byte array for aux_rand arg of schnorr_sign See: BlockstreamResearch#86

python: add test vector generator

1c4b124

outputs one JSON file per API in `python/vectors` folder

python: test generated vectors against the APIs

0280035

jonasnick reviewed Apr 24, 2025

View reviewed changes

jonasnick reviewed Apr 25, 2025

View reviewed changes

jonasnick reviewed May 1, 2025

View reviewed changes

		# --- Error test case 4: invalid threshold ---
		invalid_crec = b"\x00" * 4 + crec[4:]

	# --- Error Test Case 2: Coordinator tampers with participant 0's encrypted secshare ---
	# --- Error Test Case 2: Coordinator tampered with participant 0's encrypted secshare ---

	"comment": "coordinator tampers with participant 0's encrypted secshare",
	"comment": "coordinator tampered with participant 0's encrypted secshare",

		import chilldkg_ref.chilldkg as chilldkg


		def generate_coordinator_step1_vectors():

Add JSON Test Vectors #90

Are you sure you want to change the base?

Add JSON Test Vectors #90

Uh oh!

Conversation

siv2r commented Mar 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonasnick left a comment

Choose a reason for hiding this comment

Uh oh!

jonasnick left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

siv2r commented Apr 26, 2025

Uh oh!

jonasnick commented Apr 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

siv2r commented Mar 30, 2025 •

edited

Loading