Sprockets breaks the sourcemap comment

[chloerei]

I'm using esbuild for js bundling, command:

esbuild app/assets/javascripts/*.* --bundle --outdir=app/assets/builds --sourcemap

Then sourcemap comment is added to builds/application.js:

//# sourceMappingURL=application.js.map

But after process by assets pipeline, it becomes:

//# sourceMappingURL=application.js.map;

It add a ; at the end and break the sourcemap config.

I found it's caused by https://github.com/rails/sprockets/blob/cddf9fb841eece80276a1ccaee1e018a356547a0/lib/sprockets/utils.rb#L100 , and there is a related fixed. rails/sprockets#515

But looks like this fix not work for js bundle from external.

I don't know the internal of asset pipeline very well. How can I make the sourcemap from esbuild work?

[rmacklin]

Can you share a reproducing public repository/gist? I'm interested in looking into a potential solution.

[chloerei]

Sure, I create a demo here: https://github.com/chloerei/rails-esbuild-demo .

bin/setup to setup and bin/dev to start service.

It generate sourcemap for js and css, css map is working but js not.

I also found I need to set config.assets.debug = false in config/environments/development.rb, otherwise it add digest in sourcemap path, both css and js map file can not found:

//# sourceMappingURL=application.js.map;

//# sourceMappingURL=application.js-2ff448f42b4456f23c978070fce3c2639ccb5652106b9af890dfb697c39e2415.map

[chloerei]

I did some learning about the internals of sprockets, I found it define a lot of processor by default, some of them is unnecessary or conflict with external bundling program.

I think sprockets need to provide a way to disable some processor, or create a clean environment, when css and js is bundling externally.

[dhh]

Working on a path to a dramatically simpler sprockets. With our new path, we don't need 80% of what's currently in Sprockets.

[dhh]

A basic version of Sprockets that just gives us a load path, digesting, copying, and compression would be all that's needed for this new approach.

[chloerei]

It looks good. 👍

[jhirn]

@dhh You are doing the lord's work for frontend in Rails 7. Thank you!

Currently using this and cssbundling-rails in a new Rails 6.1 app and I can't believe how long we dealt wit the status quo.

Are sourcemaps possible at this point through esbuild/sprokets or should I just wait for an update to Rails/Sprockets?

[dhh]

Thanks! It's great to see it all come together.

Need to fix sprockets before this is fully possible. Intend to do so soon, and definitely before Rails 7 final.

For people using bundlers for both js and css, the new Propshaft option for the asset pipeline will also shortly be an excellent alternative to sprockets.

All this just need a few more weeks baking.

[jhirn]

I've had success so far with sourcemap: 'inline',. Not ideal for shipping but at least i'm getting local debugging.

[dhh]

After reviewing this further, we don't have a good path yet for getting source maps going with esbuild. There's not enough configuration control to ensure that the digests match what we need for Sprockets.

I'd encourage anyone using esbuild to output es6, and thus not needing any source maps.

Otherwise some of the other bundlers, like webpack, do make this possible.

I'll document the constraint.

[tsrivishnu]

@dhh We are facing this issue with Webpack as well. In your last comment, you mentioned that Webpack does make something possible but I'm not sure if you meant that there is a way to overcome this issue when using Webpack. If this is documented somewhere, could you please direct me to that?

In our setup, Webpack generates the following files to app/assets/builds/:

app.js
app.js-2900f7bc66bb9248004610a2a6aaad98.digested.map

After asset pipeline process, we having the following in public/assets/

app-c030ba35ee6a76754b09871773215453129630da92e84c4ffb304c4e22f54f6b.js
app.js-2900f7bc66bb9248004610a2a6aaad98.digested.map

When I inspect the file the final file: app-c030ba35ee6a76754b09871773215453129630da92e84c4ffb304c4e22f54f6b.js

...
//# sourceMappingURL=app.js-2900f7bc66bb9248004610a2a6aaad98.digested.map;

[graydavid]

For anyone that comes across this issue, here's a workaround: end the source map filename with a semicolon. Here's what an example webpack config looks like:

  output: {
    ...
    // We include the trailing ";" in the filename in order to avoid the bug
    // here: https://github.com/rails/jsbundling-rails/issues/24 . If we
    // didn't, sprockets would add a semicolon to the end anyway and break
    // the reference to the source map file.
    sourceMapFilename: '[name]-[contenthash].digested.js.map;',
    ...
  },

For example, given an application.js output filename and a content hash of 72d293f0, the above would create a source map file with the literal name of application-72d293f0.digested.js.map;, semicolon and all. The final version of the application.js file, after Sprockets processes it, would then contain this sourceMappingURL line at the end:

//# sourceMappingURL=application-72d293f0.digested.js.map;

Sprockets skips adding a semicolon because the line already ends with one, thanks to the filename. It's definitely odd to see a filename that has a semicolon in it (I didn't know this was legal until I tried), but it worked.

(Thanks to @chloerei for linking the sprockets source code, which showed that sprockets has this conditional logic in it.)

---

[UPDATE on 4/5/2023]

The workaround mentioned above was necessary and worked with rails=6.1.4.1; sprockets=4.1.1; sprockets-rails=3.2.2. After upgrading to rails=6.1.4.7; sprockets=4.2.0; sprockets-rails=3.4.2, it no longer worked: sourceMappingURL was removed from the generated application.js file completely. On the plus side, the workaround was no longer necessary, as rails/sprockets no longer added a trailing semi colon to the sourceMappingURL.

[tsrivishnu]

If anyone is interested, we overcome the issue with rewriting the request to remove the semicolon in NGINX with the following snippet:

  location ~ "^\/assets\/.*\.map;$" {
    # Not sure if we should use +permanent+ and make a permanent redirect
    # vs using +redirect+ for a temporary redirect
    rewrite ^/assets/(.*).map\;$ /assets/$1.map permanent;
  }