Add support to pass libraries as a compressed file

[jpverdejo]

I need to pass a decent amount of declaration files to the editor. These declaration files will be defined in a separate repo and published to NPM.

In our current implementation (react app bundled with rsbuild) the only option I had was defining each file independently, as the bundler doesn't allow dynamic imports (files.forEach({ ...; require(filename); ...}))

The solution we came up with is to compress all the files into a single zip and pass it to the web worker for it to decompress and add as a library

I thought it could be useful to somebody else and decided to open a PR.

I had a hard time coming up with a way to test it, and this is the best I could come up with, but it's not something I'd want to get merged. Because of that I'll open it as a draft PR to gather feedback before getting to a prod-ready solution

[mkslanc]

@jpverdejo Hi! I’m not sure I fully understand the context. For example, why isn’t something like this an option?

const urls = [ 'lib1.d.ts', 'lib2.d.ts', /*…*/ ];
const contents = await Promise.all(urls.map(u => fetch(u).then(r => r.text())));

Could you please clarify the requirements or provide a small, reproducible example?
Personally, I’d prefer not to introduce another dependency for a single use case.

[jpverdejo]

I understand this may be a niche solution, so no big deal if you think we should keep it in our fork.

Your solution would work under these conditions:

• The number of files to be included are somewhat limited (ie one bundle per library, only a few libraries)
• We don't need to support new libraries too often
• We can easily bundle the declarations into a single file per library

However in our use case none or those are true:

• The number of files to be included are somewhat limited (ie one bundle per library, only a few libraries)
  • For example the YAML package has 78 declaration files, and that's just one library
• We don't need to support new libraries too often
  • We're building our feature under the assumption that we'll keep adding support for new libraries, we need it to be extremely maintainable
• We can easily bundle the declarations into a single file per library
  • I tried this, for some libraries it was fire-and-forget, for others we required a ton of patches for the bundler to work, if I was able to make it work

On top of that we use our own sandboxed JS runtime to run the code written in the editor in a different environment (backend), so keeping those 2 environments in sync is a must

To solve this issue we built a middleground/setup repo, where we'll configure all the packages we want, plus anything extra that we need (ie, patch a library, add extra declaration files for custom supported behavior, etc), and we need this to output "something" that's easy to use for both the backend JS runtime and the ACE editor

By supporting passing the declaration files as a package to ace-linters we can just do it like:

package.json

...
"dependencies": {
  "@internalpackage/js-libraries": "v1.2.3"
  ...
}
...

editor.ts

import { librariesDeclarationsZipFile } from '@internalpackage/js-libraries';
...
languageProvider.setGlobalOptions('typescript', {
  ...
  extraLibsZip: librariesDeclarationsZipFile,
});

(And a similar solution for the backend)

With this whenever we need to update the libraries and/or declarations, we can go to the setup repo, make the changes, and then just use the new version in the FE and BE

Again, I understand if this is not something we want to support upstream, but wanted to contribute back what is a useful new feature for us

[mkslanc]

@jpverdejo Thanks for such detailed answer! I need to think a bit whether to proceed with this or not