feat: add keysets v2 client side implementation

Keysets v2: align with NUT-13 HMAC derivation (4-byte counter, single HMAC split); implement wallet-side short ID manager and token helpers; apply default final_expiry on rotation; keep mint agnostic of s_id

alignment

NUT-02/13 alignment:
- Keys API: include final_expiry in /v1/keys and /v1/keys/{id}
- Derivation: keyset ID v2 per spec (unit + optional final_expiry; SHA256; v=01)
- NUT-13 v2 secrets: switch to HMAC-SHA256 with 8-byte counter and type byte (00/01)
- Tests: update wallet v2 tests to HMAC-SHA256; derive seed from BIP-39 mnemonic; fix TokenV4 short ID expansion test to use KeysetManager directly; update API expectations for final_expiry

Remove mint_use_keysets_v2 flag; always generate v2 keyset IDs for new keysets. Keep optional default final_expiry via settings. Update keyset rotation/activation to use default expiry unconditionally. Update docs.

wallet: log debug when resolving short keyset id to full id (cache hit).

wallet: integrate v2 short keyset IDs directly in TokenV4 creation; deprecate WalletTokensV2 helper; tests: remove tokens_v2 dependency, assert short-ID serialization; secrets: NUT-13 v2 docstring to HMAC-SHA256; proofs: fix helper method refactor regression; cleanup

remove deprecated cashu/wallet/tokens_v2.py; short-ID handling now in WalletProofs._make_tokenv4

Summary of changes on keysets-v2
- Removed deprecated helper:
  - cashu/wallet/tokens_v2.py deleted.
- Integrated short-ID handling into wallet TokenV4 creation:
  - cashu/wallet/proofs.py: _make_tokenv4 now converts v2 full keyset IDs to short IDs during token creation via KeysetManager; ensured _get_proofs_mint_unit remains correct.
- Updated tests to remove tokens_v2 dependency:
  - tests/wallet/test_wallet_keysets_v2.py: test_token_serialization_with_short_ids now uses WalletProofs to assert short-ID serialization.
- Docstring correction:
  - cashu/wallet/secrets.py: NUT-13 v2 derivation is HMAC-SHA256 (not SHA512).

Remote
- Branch keysets-v2 is now ahead by 2 commits and pushed to origin.

format

move tests to v2 keyset id

fixed test_mint_api

more fixes

bump nutshell version to 0.18 => derive ID v1/v2 based on nutshell version of the keyset: if not specified the nutshell version is 0.18 and the keyset ID is v2

more fixes

Fix keyset version tests: update assertions for v1/v2 ID expectations

- Change test_keyset_0_15_0 and test_keyset_0_15_0_encrypted to expect V1_KEYSET_ID (version 0.15.0 uses v1 IDs)
- Update test_keyset_short_id to properly assert V1_KEYSET_ID for legacy keysets
- Simplify test_keyset_v2_deterministic to use version 0.18.0 which uses v2 IDs
- Remove redundant test_keyset_v1_v2_compatibility test (covered by other tests)
- Update test section header to 'KEYSET IDs NUT-02 TEST VECTORS'

Add comprehensive keyset ID test vectors and version behavior tests

New tests added:
1. test_keyset_versions_produce_correct_id_format: Tests that different
   nutshell versions (0.11, 0.14, 0.15, 0.17, 0.18, 0.19) produce the
   correct ID format (base64 for <0.15, v1 hex for 0.15-0.17, v2 hex for >=0.18)

2. test_keyset_id_v1_test_vectors: Implements NUT-02 test vectors for v1
   keyset ID derivation with two test vectors (small 4-key keyset and
   large 64-key keyset)

3. test_keyset_id_v2_test_vectors: Implements NUT-02 test vectors for v2
   keyset ID derivation with three test vectors testing different
   combinations of keyset sizes, units, and final_expiry values

These tests ensure compatibility with the NUT specifications and verify
that the keyset ID derivation algorithms produce stable, interoperable
results across different implementations.

Test vectors source: https://github.com/cashubtc/nuts/pull/182/files

Fix test_keyset_version_detection for v1 vs v2 ID distinction

Corrected the test to properly distinguish between v1 and v2 keyset IDs:
- V1 IDs (version 0.15-0.17) start with '00' and are NOT detected as v2
- V2 IDs (version 0.18+) start with '01' and ARE detected as v2

This test was incorrectly expecting a v1 keyset (version 0.15.0) to be
identified as v2.

more fixes

Add v2 short keyset ID expansion support for token redemption

Fix for TokenV3 and TokenV4 redemption with v2 short keyset IDs:

- Added _expand_short_keyset_ids() method to WalletProofs class
  - Expands v2 short IDs (16 chars, '01' + 7 bytes) to full IDs (66 chars)
  - Uses KeysetManager to map short->full IDs from loaded keysets
  - Handles expansion failures gracefully with warning logging

- Updated redeem_TokenV3() and redeem_TokenV4() in helpers.py
  - Calls _expand_short_keyset_ids() after load_mint() to ensure
    all keyset IDs in proofs are expanded before redemption

- Added comprehensive test: test_proof_short_id_expansion()
  - Verifies short ID expansion works correctly
  - Tests that proofs with v2 short IDs get expanded to full IDs

- Fixed existing tests in test_wallet_keysets_v2.py
  - Removed incorrect 'await' calls on synchronous KeysetManager methods
  - Fixed test_token_v4_short_keyset_expansion to use proper v2 short ID
  - Fixed test_nut13_spec_compliance to use actual V2_KEYSET_ID constant

- Updated test_wallet_cli.py token to use v2 short keyset IDs

This ensures wallets can properly handle TokenV3/V4 tokens containing
v2 short keyset IDs, which are used to reduce token size.

feat: improve v2 keyset ID handling and fee calculation

- Add short-to-full keyset ID expansion in wallet helpers
- Enhance KeysetManager with better documentation and type hints
- Fix fee calculation to handle short keyset IDs gracefully
- Add keyset ID expansion before proof redemption in CLI
- Update tests to reflect new keyset ID handling behavior
- Add fallback logic for missing keysets in fee calculations

format

fixes

base64 keysets detection

Author: lollerfirst

cashu/core/base.py

@@ -24,6 +24,7 @@
     derive_keys_deprecated_pre_0_15,
     derive_keyset_id,
     derive_keyset_id_deprecated,
+    derive_keyset_id_v2,
     derive_pubkeys,
 )
 from .crypto.secp import PrivateKey, PublicKey
@@ -805,6 +806,7 @@ class MintKeyset:
     version: Optional[str] = None
     amounts: List[int]
     balance: int
+    final_expiry: Optional[int] = None  # NEW: Final expiry timestamp for keyset v2
 
     duplicate_keyset_id: Optional[str] = None  # BACKWARDS COMPATIBILITY < 0.15.0
 
@@ -826,6 +828,7 @@ def __init__(
         id: str = "",
         balance: int = 0,
         fees_paid: int = 0,
+        final_expiry: Optional[int] = None,
     ):
         DEFAULT_SEED = "supersecretprivatekey"
         if seed == DEFAULT_SEED:
@@ -861,6 +864,7 @@ def __init__(
         self.balance = balance
         self.fees_paid = fees_paid
         self.input_fee_ppk = input_fee_ppk or 0
+        self.final_expiry = final_expiry
 
         if self.input_fee_ppk < 0:
             raise Exception("Input fee must be non-negative.")
@@ -915,6 +919,7 @@ def from_row(cls, row: Row):
             amounts=json.loads(row["amounts"]),
             balance=row["balance"],
             fees_paid=row["fees_paid"],
+            final_expiry=row["final_expiry"],
         )
 
     @property
@@ -958,12 +963,33 @@ def generate_keys(self):
             )
             self.public_keys = derive_pubkeys(self.private_keys, self.amounts)  # type: ignore
             self.id = id_in_db or derive_keyset_id_deprecated(self.public_keys)  # type: ignore
+        elif self.version_tuple < (0, 18):
+            self.private_keys = derive_keys(
+                self.seed, self.derivation_path, self.amounts
+            )
+            self.public_keys = derive_pubkeys(self.private_keys, self.amounts)  # type: ignore
+            
+            if id_in_db:
+                # If loading from DB, preserve existing ID
+                self.id = id_in_db
+            else:
+                assert self.public_keys is not None
+                self.id = derive_keyset_id(self.public_keys)
+                logger.info(f"Generated keyset v1 ID: {self.id}")
         else:
             self.private_keys = derive_keys(
                 self.seed, self.derivation_path, self.amounts
             )
             self.public_keys = derive_pubkeys(self.private_keys, self.amounts)  # type: ignore
-            self.id = id_in_db or derive_keyset_id(self.public_keys)  # type: ignore
+            
+            # KEYSETS V2: Use new keyset ID derivation
+            if id_in_db:
+                # If loading from DB, preserve existing ID
+                self.id = id_in_db
+            else:
+                assert self.public_keys is not None
+                self.id = derive_keyset_id_v2(self.public_keys, self.unit, self.final_expiry)
+                logger.info(f"Generated keyset v2 ID: {self.id}")
 
 
 # ------- TOKEN -------

cashu/core/crypto/keys.py

@@ -1,12 +1,15 @@
 import base64
 import hashlib
 import random
-from typing import Dict, List
+from typing import TYPE_CHECKING, Dict, List, Optional
 
 from bip32 import BIP32
 
 from .secp import PrivateKey, PublicKey
 
+if TYPE_CHECKING:
+    from ..base import Unit
+
 
 def derive_keys(mnemonic: str, derivation_path: str, amounts: List[int]):
     """
@@ -54,13 +57,121 @@ def derive_pubkeys(keys: Dict[int, PrivateKey], amounts: List[int]):
 
 
 def derive_keyset_id(keys: Dict[int, PublicKey]):
-    """Deterministic derivation keyset_id from set of public keys."""
+    """Deterministic derivation keyset_id from set of public keys (version 00)."""
     # sort public keys by amount
     sorted_keys = dict(sorted(keys.items()))
     pubkeys_concat = b"".join([p.serialize() for _, p in sorted_keys.items()])
     return f"00{hashlib.sha256(pubkeys_concat).hexdigest()[:14]}"
 
 
+def derive_keyset_id_v2(
+    keys: Dict[int, PublicKey], 
+    unit: "Unit", 
+    final_expiry: Optional[int] = None
+) -> str:
+    """
+    Deterministic derivation keyset_id v2 from set of public keys (version 01).
+    
+    Args:
+        keys: Dictionary mapping amounts to public keys
+        unit: The unit of the keyset (e.g., Unit.sat, Unit.usd)
+        final_expiry: Optional unix epoch timestamp for keyset expiration
+        
+    Returns:
+        Full 33-byte keyset ID (version byte + 32-byte hash) as hex string
+    """
+    # sort public keys by amount in ascending order
+    sorted_keys = dict(sorted(keys.items()))
+    
+    # concatenate all public keys to one byte array with fixed separator between keys
+    keyset_id_bytes = b"".join([p.serialize() for p in sorted_keys.values()])
+    
+    # add the lowercase unit string to the byte array (no separator necessary since we hash)
+    keyset_id_bytes += f"unit:{unit.name}".encode("utf-8")
+    
+    # only include final_expiry if provided (per spec discussion)
+    if final_expiry is not None:
+        keyset_id_bytes += f"final_expiry:{final_expiry}".encode("utf-8")
+    
+    # SHA256 hash the concatenated byte array
+    hash_digest = hashlib.sha256(keyset_id_bytes).hexdigest()
+    
+    # prefix with version byte 01
+    return f"01{hash_digest}"
+
+
+def derive_keyset_short_id(keyset_id: str) -> str:
+    """
+    Derive the short keyset ID (8 bytes) from a full keyset ID.
+    
+    Args:
+        keyset_id: Full keyset ID (either version 00 or 01)
+        
+    Returns:
+        Short keyset ID (version byte + first 7 bytes of hash)
+    """
+    # For version 00, keep existing behavior (already short)
+    if keyset_id.startswith("00"):
+        return keyset_id
+    
+    # For version 01, return first 16 chars (8 bytes in hex)
+    if keyset_id.startswith("01"):
+        return keyset_id[:16]
+    
+    raise ValueError(f"Unsupported keyset version in ID: {keyset_id}")
+
+
+def is_base64_keyset_id(keyset_id: str) -> bool:
+    """
+    Check if a keyset ID is a legacy base64 format (pre-0.15.0).
+    
+    Base64 keyset IDs:
+    - Don't start with "00" or "01" version prefix
+    - Are typically 12 characters long
+    - Are valid base64 strings
+    
+    Args:
+        keyset_id: The keyset ID to check
+        
+    Returns:
+        True if the keyset ID is base64 format, False otherwise
+    """
+    # If it starts with a known version prefix, it's not base64
+    if keyset_id.startswith("00") or keyset_id.startswith("01"):
+        return False
+    
+    # Try to decode as base64 to confirm
+    try:
+        base64.b64decode(keyset_id)
+        return True
+    except Exception:
+        return False
+
+
+def get_keyset_id_version(keyset_id: str) -> str:
+    """
+    Extract the version from a keyset ID.
+    
+    Returns:
+        - "00" for version 0 (hex keyset IDs with 00 prefix)
+        - "01" for version 1 (v2 keyset IDs with 01 prefix)
+        - "base64" for legacy base64 keyset IDs (pre-0.15.0)
+    """
+    if len(keyset_id) < 2:
+        raise ValueError("Invalid keyset ID: too short")
+    
+    # Check if it's a legacy base64 keyset ID
+    if is_base64_keyset_id(keyset_id):
+        return "base64"
+    
+    return keyset_id[:2]
+
+
+def is_keyset_id_v2(keyset_id: str) -> bool:
+    """Check if a keyset ID is version 2 (starts with '01')."""
+    return keyset_id.startswith("01")
+
+
 def derive_keyset_id_deprecated(keys: Dict[int, PublicKey]):
     """DEPRECATED 0.15.0: Deterministic derivation keyset_id from set of public keys.
     DEPRECATION: This method produces base64 keyset ids. Use `derive_keyset_id` instead.

cashu/core/models.py

@@ -104,6 +104,7 @@ class KeysResponseKeyset(BaseModel):
     id: str
     unit: str
     keys: Dict[int, str]
+    final_expiry: Optional[int] = None 
 
 
 class KeysResponse(BaseModel):
@@ -115,6 +116,7 @@ class KeysetsResponseKeyset(BaseModel):
     unit: str
     active: bool
     input_fee_ppk: Optional[int] = None
+    final_expiry: Optional[int] = None
 
 
 class KeysetsResponse(BaseModel):

cashu/core/settings.py

@@ -8,7 +8,7 @@
 
 env = Env()
 
-VERSION = "0.17.0"
+VERSION = "0.18.0"
 
 
 def find_env_file():
@@ -91,7 +91,6 @@ class MintWatchdogSettings(MintSettings):
 class MintDeprecationFlags(MintSettings):
     mint_inactivate_base64_keysets: bool = Field(default=False)
 
-
 class MintBackends(MintSettings):
     mint_lightning_backend: str = Field(default="")  # deprecated
     mint_backend_bolt11_sat: str = Field(default="")

cashu/mint/crud.py

@@ -688,8 +688,8 @@ async def store_keyset(
         await (conn or db).execute(
             f"""
             INSERT INTO {db.table_with_schema('keysets')}
-            (id, seed, encrypted_seed, seed_encryption_method, derivation_path, valid_from, valid_to, first_seen, active, version, unit, input_fee_ppk, amounts, balance)
-            VALUES (:id, :seed, :encrypted_seed, :seed_encryption_method, :derivation_path, :valid_from, :valid_to, :first_seen, :active, :version, :unit, :input_fee_ppk, :amounts, :balance)
+            (id, seed, encrypted_seed, seed_encryption_method, derivation_path, valid_from, valid_to, first_seen, active, version, unit, input_fee_ppk, amounts, balance, final_expiry)
+            VALUES (:id, :seed, :encrypted_seed, :seed_encryption_method, :derivation_path, :valid_from, :valid_to, :first_seen, :active, :version, :unit, :input_fee_ppk, :amounts, :balance, :final_expiry)
             """,
             {
                 "id": keyset.id,
@@ -710,6 +710,7 @@ async def store_keyset(
                 "input_fee_ppk": keyset.input_fee_ppk,
                 "amounts": json.dumps(keyset.amounts),
                 "balance": keyset.balance,
+                "final_expiry": keyset.final_expiry,  # NEW: Store final expiry
             },
         )
 
@@ -822,7 +823,7 @@ async def update_keyset(
         await (conn or db).execute(
             f"""
             UPDATE {db.table_with_schema('keysets')}
-            SET seed = :seed, encrypted_seed = :encrypted_seed, seed_encryption_method = :seed_encryption_method, derivation_path = :derivation_path, valid_from = :valid_from, valid_to = :valid_to, first_seen = :first_seen, active = :active, version = :version, unit = :unit, input_fee_ppk = :input_fee_ppk
+            SET seed = :seed, encrypted_seed = :encrypted_seed, seed_encryption_method = :seed_encryption_method, derivation_path = :derivation_path, valid_from = :valid_from, valid_to = :valid_to, first_seen = :first_seen, active = :active, version = :version, unit = :unit, input_fee_ppk = :input_fee_ppk, final_expiry = :final_expiry
             WHERE id = :id
             """,
             {
@@ -843,6 +844,7 @@ async def update_keyset(
                 "unit": keyset.unit.name,
                 "input_fee_ppk": keyset.input_fee_ppk,
                 "balance": keyset.balance,
+                "final_expiry": keyset.final_expiry,  # NEW: Update final expiry
             },
         )
 

cashu/mint/keysets.py

@@ -97,6 +97,7 @@ async def rotate_next_keyset(
             amounts=amounts,
             input_fee_ppk=input_fee_ppk,
             active=True,
+            final_expiry=None
         )
 
         logger.debug(f"New keyset was generated with Id {new_keyset.id}. Saving...")
@@ -166,6 +167,7 @@ async def activate_keyset(
                 amounts=self.amounts,
                 version=version,
                 input_fee_ppk=settings.mint_input_fee_ppk,
+                final_expiry=None,
             )
             logger.debug(f"Generated new keyset with ID '{keyset.id}'.")
 

cashu/mint/migrations.py

@@ -968,3 +968,16 @@ async def m027_add_balance_to_keysets_and_log_table(db: Database):
                 );
             """
         )
+
+
+async def m028_add_final_expiry_to_keysets(db: Database):
+    """
+    Add final_expiry column to keysets table for keysets v2 support.
+    """
+    async with db.connect() as conn:
+        await conn.execute(
+            f"""
+                ALTER TABLE {db.table_with_schema('keysets')}
+                ADD COLUMN final_expiry INTEGER NULL
+            """
+        )

cashu/mint/router.py

@@ -81,6 +81,7 @@ async def keys():
                     id=keyset.id,
                     unit=keyset.unit.name,
                     keys={k: v for k, v in keyset.public_keys_hex.items()},
+                    final_expiry=keyset.final_expiry,  # NEW: Include final expiry to align with NUT-02 PR #182
                 )
             )
     return KeysResponse(keysets=keyset_for_response)
@@ -117,6 +118,7 @@ async def keyset_keys(keyset_id: str) -> KeysResponse:
         id=keyset.id,
         unit=keyset.unit.name,
         keys={k: v for k, v in keyset.public_keys_hex.items()},
+        final_expiry=keyset.final_expiry,
     )
     return KeysResponse(keysets=[keyset_for_response])
 
@@ -139,6 +141,7 @@ async def keysets() -> KeysetsResponse:
                 unit=keyset.unit.name,
                 active=keyset.active,
                 input_fee_ppk=keyset.input_fee_ppk,
+                final_expiry=keyset.final_expiry,
             )
         )
     return KeysetsResponse(keysets=keysets)

cashu/wallet/helpers.py

@@ -42,7 +42,14 @@ async def redeem_TokenV3(wallet: Wallet, token: TokenV3) -> Wallet:
     """
     if not token.unit:
         # load unit from wallet keyset db
-        keysets = await get_keysets(id=token.token[0].proofs[0].id, db=wallet.db)
+        # Note: if the keyset ID is a v2 short ID, we might not find it in the db
+        # In that case, we'll just skip setting the unit here and let it be set later
+        proof_keyset_id = token.token[0].proofs[0].id
+        keysets = await get_keysets(id=proof_keyset_id, db=wallet.db)
+        if not keysets and proof_keyset_id.startswith("01") and len(proof_keyset_id) == 16:
+            # This might be a v2 short ID, try to find a matching full ID
+            all_keysets = await get_keysets(db=wallet.db)
+            keysets = [k for k in all_keysets if k.id.startswith(proof_keyset_id)]
         if keysets:
             token.unit = keysets[0].unit.name
 
@@ -58,6 +65,11 @@ async def redeem_TokenV3(wallet: Wallet, token: TokenV3) -> Wallet:
         keyset_ids = mint_wallet._get_proofs_keyset_ids(t.proofs)
         logger.trace(f"Keysets in tokens: {' '.join(set(keyset_ids))}")
         await mint_wallet.load_mint()
+        
+        # Expand v2 short keyset IDs to full IDs
+        # V2 short IDs are 16 chars (version '01' + 7 bytes), full IDs are 66 chars
+        await mint_wallet._expand_short_keyset_ids(t.proofs)
+        
         proofs_to_keep, _ = await mint_wallet.redeem(t.proofs)
         print(f"Received {mint_wallet.unit.str(sum_proofs(proofs_to_keep))}")
 
@@ -70,7 +82,15 @@ async def redeem_TokenV4(wallet: Wallet, token: TokenV4) -> Wallet:
     Redeem a token with a single mint.
     """
     await wallet.load_mint()
-    proofs_to_keep, _ = await wallet.redeem(token.proofs)
+    
+    # Get proofs from token (these will have short keyset IDs)
+    proofs = token.proofs
+    
+    # Expand v2 short keyset IDs to full IDs in-place
+    await wallet._expand_short_keyset_ids(proofs)
+    
+    # Use the expanded proofs for redemption
+    proofs_to_keep, _ = await wallet.redeem(proofs)
     print(f"Received {wallet.unit.str(sum_proofs(proofs_to_keep))}")
     return wallet