Skip to content

SE: Implement basic searchable encryption #3681

@islamaliev

Description

@islamaliev

Implement the foundational Searchable Encryption capability for basic equality (_eq) queries on designated fields. This involves client-side tag generation using a deterministic, keyed approach (similar to HMAC beacons), storage of the index in the remote peer's KV store via replication updates, and a dedicated query mechanism.

Deliverables:

  • Key Management: Introduce a new persistent, symmetric searchable-encryption-key per user identity within the defradb keyring system.
  • Schema Definition: Implement the @encryptedIndex(type: "equality") directive (or simply @encryptedIndex defaulting to equality) for marking fields in the SDL.
  • Client-Side Index Metadata: Implement local storage and management of metadata for defined encrypted indexes (tracking IndexID, target field(s), type, config).
  • Search Tag Computation: Implement client-side tag generation using HMAC-SHA256 (truncated) with the SE key and appropriate domain separation (using CollectionID + IndexID).
  • Client Indexing Logic: Integrate logic into commit processing to identify indexed fields (initially focusing on LWW CRDTs), compute old/new tags, and queue index update operations.
  • Replication Channel: Define and implement IndexAddOperation and IndexDeleteOperation P2P messages; modify the active replicator to send these after block commits.
  • Remote Node KV Handling: Implement logic on the remote node's P2P service to receive index update messages and apply corresponding Put/Delete operations to its local KV store using the defined key structure (/se/////).
  • Query API: Auto-generate dedicated GraphQL mutations per collection (e.g., encrypted_User(filter: {...})) for SE queries. The filter input type should only allow _eq on indexed fields. Configure the target remote peer implicitly (e.g., via startup config).
  • Query Execution: Implement the client-side logic to compute the query tag/prefix and make a P2P RPC call (e.g., requestKVPrefixScan) to the remote node. Implement the remote node RPC handler to scan the KV index and return matching DocIDs.
  • Documentation: Basic documentation on how to enable and use equality-based SE.

Metadata

Metadata

Assignees

Labels

area/queryRelated to the query componentarea/schemaRelated to the schema systemsecurityRelated to security

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions