Skip to content

[Experimental Feature] Add support for cross-origin storage#748

Merged
akaashrp merged 13 commits intomlc-ai:mainfrom
akaashrp:cross-origin-storage
Mar 22, 2026
Merged

[Experimental Feature] Add support for cross-origin storage#748
akaashrp merged 13 commits intomlc-ai:mainfrom
akaashrp:cross-origin-storage

Conversation

@akaashrp
Copy link
Copy Markdown
Collaborator

@akaashrp akaashrp commented Nov 27, 2025

  1. Add support for cross-origin storage caching
  2. Deprecate useIndexedDBCache

See apache/tvm#18893

Copy link
Copy Markdown
Contributor

@tomayac tomayac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. I just left a couple of suggestions, mostly minor cosmetic things like updating links and such.

Here's an additional suggestion for what you could add to the docs at docs/user/advanced_usage.rst (line 119):

Using Cross-Origin Storage Cache
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

WebLLM also supports caching model artifacts across different origins using the experimental Cross-Origin Storage API. You can enable this cache backend by setting ``appConfig.cacheBackend = "cross-origin"``. For users with the `Cross-Origin Storage browser extension <https://chromewebstore.google.com/detail/cross-origin-storage/denpnpcgjgikjpoglpjefakmdcbmlgih>`_ installed, resources will then be cached and shared across origins. This means two independent apps opted into this cache backend using the same AI model will download and cache the required resources only once. See the `cache usage example <https://github.com/mlc-ai/web-llm/tree/main/examples/cache-usage>`_ for more details. If Cross-Origin Storage isn't available, WebLLM will automatically fall back to using the default cache.

.. code-block:: typescript

   const engine = await CreateMLCEngine("Llama-3.1-8B-Instruct", {
       appConfig: {
           cacheBackend: "cross-origin",
       },
   });

@akaashrp akaashrp mentioned this pull request Mar 22, 2026
16 tasks
@akaashrp akaashrp force-pushed the cross-origin-storage branch from 201ae5c to 731eaab Compare March 22, 2026 22:31
@akaashrp akaashrp merged commit 5cb898a into mlc-ai:main Mar 22, 2026
6 checks passed
@tomayac
Copy link
Copy Markdown
Contributor

tomayac commented Mar 23, 2026

Amazing, thank you very much for landing this! ♥️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants