-
-
Notifications
You must be signed in to change notification settings - Fork 112
refactor: shard meta-schemas per draft #763
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: shard meta-schemas per draft #763
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #763 +/- ##
==========================================
+ Coverage 85.71% 85.79% +0.08%
==========================================
Files 95 95
Lines 15531 15620 +89
==========================================
+ Hits 13312 13401 +89
Misses 2219 2219 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Amazing! Thank you so much! Do you have some numbers on the memory usage change? |
CodSpeed Performance ReportMerging #763 will not alter performanceComparing Summary
|
This introduces some new (internal-only and lazy) static entities in order to shard meta-schemas according to the draft that they belong to. Sharded meta-schemas allow to reduce the amount of registry resources that need to be constructed and statically cached.
11e4ee0
to
db184aa
Compare
I have some numbers from a private application, but unfortunately I can't share the code nor the JSON schema/docs. As a single data-point, performing validation of a single doc and a single schema under draft-202012, this seems to be saving ~300kB of lazy-cached memory. I'm checking via https://github.com/KDE/heaptrack. I'm honestly a bit disappointed as I was hoping for more, but at this point I think most of the memory usage is coming from the lazy/recursive |
Aha, I see. 300kb is not nothing, and the change seems to be relatively straightforward :) Yesterday, I got at least some stuff working in #749, and I believe that approach should decrease the memory usage significantly, as it allows using an id to refer to a subschema, instead of building that lazy validator, which currently is implemented for |
Thanks, I'll try to have a look at #749 if I manage to (I'm currently traveling). |
Thank you! |
This introduces some new (internal-only and lazy) static entities in order to shard meta-schemas according to the draft that they belong to. Sharded meta-schemas allow to reduce the amount of registry resources that need to be constructed and statically cached.