Add per-context memory accounting and limits

[aabbdev]

Introduce memory usage tracking at the JSContext level, allowing embedding runtimes to enforce hard memory caps per tenant or sandbox.

• JSContext gains:

  • size_t mem_limit: configurable hard limit (0 = unlimited)
  • size_t mem_used: live allocation counter

• New API:

  • void JS_SetContextMemoryLimit(JSContext *ctx, size_t limit);
  • void JS_ResetContextMemory(JSContext *ctx); size_t JS_GetContextMemoryUsage(JSContext *ctx);

With this change, embedders can harden multi-tenant runtimes by sandboxing memory usage per JSContext, instead of only at the runtime level.

TODO:

• Disable evals
• Disable Atomics/SharedArrayBuffer
• Disable new Function("...")
• Restricted import()
• helpers or an API to measure cpu time?

[saghul]

Can't this be accomplished with the custom memory allocation functions already?

[aabbdev]

@saghul we can set limits at the runtime level, not per context. The runtime loses track of the context after the high-level allocation functions.

[saghul]

What use case do you have which has multiple contexts per runtime?

[aabbdev]

@saghul ultra dense use cases like serverless functions: e.g 128 contexts per runtime per core. Many companies using QuickJS face this challenge; I've seen several discussions on the official channels, and I’m dealing with the same use case. Because they don't have this capability they instantiate one runtime per context.

[aabbdev]

There's a significant inconsistency in the allocation functions: sometimes they take rt, sometimes ctx, and in many cases if not the majority *_rt(...) are called even when ctx is available like *_rt(ctx->rt). We should enforce a clear rule: we must never deconstruct ctx and give rt in param but directly the ctx pointer.
No allocation or manipulations should be made without explicitly precise who is the initiator (the context) especially if 100% of the time it's indirectly linked to a context

[aabbdev]

My planning:

• Rewrite all memory paths for consistent memory accounting and a cleaner implementation.
• Add a build-time flag to enable hardening (and disable certain features), or possibly an hybrid runtime switch needs benchmarking.

This introduces breaking API changes, except If I include compatibility APIs, so it may end up as a fork/divergence. I completely understand if it’s not something maintainers want to integrate into quickjs-ng.

[past-due]

Just a note that upstream may be planning to refactor how JSRuntime / JSContext works (see: bellard/quickjs#421 ). As such, it might be prudent to wait a bit to see what the plans are there (and that may also clear up some of your inconsistency concerns listed above?)

[aabbdev]

@past-due They’re basically just renaming things: since there are no memory limits at the context level, they feel uneasy about instantiating a runtime, so they renamed runtime → context and context → realm.

With proper memory limits on contexts (or realms), there’s no need to create a separate runtime+context for every script. You can run a single runtime with N contexts, each configured with its own memory limit. If you need strict multi-tenancy, you can still isolate by creating one runtime per script.

My proposal is to avoid allocations from the Runtime entirely and always go through the Context APIs (or Realm, depending on naming). Right now the internal API is inconsistent: sometimes you allocate with a context, but free through the runtime. Instead, allocations and frees should always go through context APIs, which can delegate internally to the runtime if needed, never directly through the runtime.

This approach prevents bypassing memory accounting, eliminates confusion from mixed APIs, and ensures that per-context memory limits are applied consistently.

Regarding memory accounting: it can be enabled with a compile-time flag. If the flag is not used, the API still exists but will throw an error indicating that you need to recompile with memory accounting enabled to enforce limits.

[bnoordhuis]

I'm not dead set against the idea but I don't want to add functionality that only has a single user. I'd like to see first if more people want/need this.

As a bit of background, I run a quickjs-based multi-tenant service but I don't have a need for per-context memory tracking, a JSRuntime + JSContext works and performs well enough. I keep some around pre-initialized so there's always one ready to go.

---

The fact JSRuntime's atoms/class/shape tables are shared across contexts undeniably has some performance benefits but they're mostly minor though.

My back-of-the-napkin math says that on a 64 bits system at 128 contexts, there's ~6.6 MiB additional overhead in the JSRuntime + JSContext model vis-a-vis a single shared JSRuntime. Not nothing but pretty close to nothing in this day and age.

I guess if you preload your contexts with a lot of additional stuff (like WinterCG) the overhead will be bigger but we'd still be talking low dozens of MiBs.

[bnoordhuis]

Can't use __builtin_mul_overflow, not universally supported.

[aabbdev]

Yes, you're right, I only put it there as a draft.
Instead of using __builtin_mul_overflow, we can #include <stdckdint.h>.
For portability, I'll add a compile-time fallback with a pure C implementation as backup.

[bnoordhuis]

This is arguably excessive. js_malloc_usable_size_rt (wrapper around malloc_usable_size on linux) is not necessarily very cheap. Probably not worth it to avoid going a few bytes over the limit.

[aabbdev]

You're right, I should only check once before the pre-allocation. Fragmentation and the actual physical size shouldn't be included in the limits. I will make the change

[bnoordhuis]

This is better handled by breaking out js_alloc_string_rt into two separate functions, one that calls js_malloc_rt and then calls the second function to initialize the string. Call said second function from here with memory allocated with js_malloc.

[aabbdev]

@bnoordhuis Glad you took the time to review this and share your experience.
This work is still in its early stages I need to run more benchmarks to properly assess the impact. For context, I’m currently working on full WinterTC compatibility, a preemptive scheduler, and a few other interesting optimizations.

I opened this PR to share the idea and start a discussion. From what I've seen, several companies are running multi-tenant architectures with a single runtime and multiple contexts, but without any per-context memory limits.

The proposal here is to simplify and improve the allocate/free function signatures and introduce optional per-context memory limits, enabled at build time via a flag + some disabled features.

We don't necessarily have to merge this feature into quickjs-ng, it's a proposal