Skip to content

Improve purging performance #10389

@dianabarsan

Description

@dianabarsan

Describe the performance issue
Server-side purging is expensive.

Describe the improvement you'd like
Placeholder for an idea I just had in the shower. Will come back with more details, but the gist is to change how we store purged docs from a per-document basis to a per-contact basis.

To Reproduce
Steps to record the performance metrics:

  1. Enable purging and add some minimal purge function.
  2. Create a BIIIIG user. Make it have 200.000 docs, with many reports. It doesn't even matter if your purging function purges them.
  3. Run purging.
  4. Replicate with this user.

Measurements
The idea is to use smaller requests, or requests that don't linearly grow with user report count. The time to call api/v1/replication/get-ids should be improved.
(don't have numbers yet)

Note: because purged docs are cached, only the first request after purging will actually run the code. to clear the cache, easiest option is to delete the user's cache document from the medic-purge-cache database
.
Additional context
#10384

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Status

💻 In Progress

Relationships

None yet

Development

No branches or pull requests

Issue actions