New injections for forked elm packages

[lydell]

Summary

This adds https://github.com/lydell/elm-safe-virtual-dom to Lamdera, simplifies the internal implementation and allows for “proper” hot reloading during development in the future.

Details

I’m working on adding hot reloading and stopping of apps directly to elm/core. Read about it here:

https://github.com/lydell/core/blob/hot-reload-stop/javascript-interface.md#appstop

Branches:

• https://github.com/lydell/core/tree/lamdera (master -> safe -> hot-reload-stop -> lamdera)
• https://github.com/lydell/browser/tree/lamdera (master -> safe -> hot-reload-stop -> lamdera)
• https://github.com/lydell/virtual-dom/tree/hot-reload-stop (master -> safe -> hot-reload-stop)
• https://github.com/lydell/html/tree/safe (master -> safe)

As you can see above, elm/core has a lamdera branch:

• It makes it possible for Lamdera to inject some the things that differ between the four modes (NotLamdera, LamderaBackend, LamderaFrontend, LamderaLive).
• It includes the lamdera/containers patch directly so we don’t need to inject it.

elm/browser also has a lamdera branch:

• It includes three lines related to Browser.Navigation.Key.

Benefits of this PR:

• Everything is much simpler.
• We get the safe virtual DOM.
• We can add real hot reloading to lamdera live in a future PR.
• We can close Add support for lydell’s fork of elm/virtual-dom #40

TODO

• Install the forked packages as mentioned below.
• Test.
• Maybe implement hot reloading as part of this PR after all. (See below.)

Installing the forked packages

Here’s the plan: Start out simple and make things more complicated in the future if needed.

Add a hard coded map like this in the Lamdera Compiler source code (the below is JSON but would be Haskell):

{
    "elm/core": {
        "version": "1.0.5",
        "url": "https://github.com/lamdera/elm-core/commit/123"
    },
    "elm/virtual-dom": {
        "version": "1.0.4",
        "url": "https://github.com/lamdera/elm-virtual-dom/commit/456"
    },
    "elm/html": {
        "version": "1.0.0",
        "url": "https://github.com/lamdera/elm-html/commit/789"
    },
    "elm/browser": {
        "version": "1.0.2",
        "url": "https://github.com/lamdera/elm-browser/commit/321"
    }
}

Use the following install strategy:

1. If the requested package is in the map, follow the steps below. Otherwise install as usual.
2. Validate versions.
3. Install from the URL provided instead of going through the package site.

Validate versions

In the JSON above, each elm/* package that we replace specifies a version. That’s the version that the forked package was based on. In elm.json, there will still be lines like "elm/core": "1.0.5". What if the version in elm.json and the “fork version” do not match?

First, let’s go through the scenarios where the fork version is higher than the version specified in elm.json:

• elm.json major < fork major -> 🤷 This can’t happen. Only 1.x versions exists of all elm/* packages we replace. But we could code it to be a hard error.
• elm.json minor < fork minor -> 🤷 This can’t happen. Only 1.0.x versions exists of all elm/* packages we replace. But we could code it to be a hard error.
• elm.json patch < fork patch -> ✅ This is very likely to happen. Lots of people probably have "elm/virtual-dom": "1.0.3" in their elm.json, and haven’t bothered updating to the recent 1.0.4 version. If we made this a hard error, it would be annoying for lots of people. I think it’s better to simply allow the fork patch version to be greater than specified. If we want to, we could inform about this somehow (log message, or maybe the comments in the compiled JS shown below is enough?).

Then, let’s go through the opposite scenarios, where the fork version is lower than the version specified in elm.json:

• elm.json major > fork major -> ❌ It’s very unlikely that Evan will suddenly release 2.x of some package. But if that happens, this should be a hard error.
• elm.json minor > fork minor -> ❌ It’s unlikely that Evan will suddenly release 1.1.x of some package. But if that happens, this should be a hard error.
• elm.json patch > fork patch -> 🚨 This is somewhat likely. As mentioned, elm/virtual-dom 1.0.4 was recently released. If elm/virtual-dom 1.0.5 is released with a security fix, and a user tries to put "elm/virtual-dom": "1.0.5" in their elm.json, I think it should be a hard error, informing them that you can’t go above 1.0.4 with this release of the Lamdera compiler. (Silently using 1.0.4 anyway would be misleading, leading to a false sense of security.) They need to wait for a new Lamdera compiler version that has pulled in the security fix.

In the compiled JS, it might be nice adding comments for debugging:

// Lamdera vX.Y.Z-W: Using the following sources for some packages, rather than the official ones:
// https://github.com/lamdera/elm-core/commit/123
// https://github.com/lamdera/elm-virtual-dom/commit/456
// https://github.com/lamdera/elm-html/commit/789
// https://github.com/lamdera/elm-browser/commit/321

Install from github.com/lamdera

Notes:

• The install URL:s contain the exact commit so that users can be confident that using the same compiler binary in the future suddenly does not work differently. We should probably store hashes of the packages as well and verify those for security (mimicking how installing from the package site works).

• We shouldn’t install into for example ~/.elm/0.19.1/packages/elm/core/1.0.5, but into ~/.elm/0.19.1/packages/lamdera/elm-core/1.0.5, and when compiling redirect reads for the former to the latter. Otherwise we would affect projects using the original Elm compiler on the same computer. Specifying "lamdera/elm-core": "1.0.5" should be an error.

• lamdera/core already exists, which is why we call the elm/core replacement lamdera/elm-core. For consistency, I used the elm- prefix for all the replacement packages.

Pros and cons

The benefit of having a hard-coded map of replacements is simplicity. We know that the provided packages are going to work with Lamdera and hot reloading.

The downside is that we need to make a new compiler release if there’s a hotfix in one of the replaced elm/* packages. However, that happens so infrequently that it feels safe to try out this approach.

How hot reload could work

Here’s how hot reload could work. I think we can do it as a follow-up PR, but I’m also open for doing it in this PR if somebody wants to pair on it.

The last commit of #29 adds two interesting things:

• A "j" WebSocket message, telling the client to hot reload. (It should only be used when the main type hasn’t changed.) It does that by adding a script tag pointing to the URL /_x/js.
• An endpoint for /_x/js.

If the endpoint returns JavaScript like this, hot reload should Just Work:

(function () {
    var scope = {};

    (function () {
        // Put the compiled Elm JavaScript code here. (Only the compiler output, no live.js or anything.)
    }).call(scope);

    Elm.hot.reload(scope);
})();

[lydell]

@supermario Has this ever happened in Bugsnag? Was it worth it? I have a feeling we don’t need this anymore, so I haven’t ported it. What do you think?

[supermario]

Just checked and nope, looks like we've never gotten this bugsnag yet... but this feels like one of those things that's not that big a deal until it happens, and then not having it will be very confusing. Perhaps just being overly paranoid. Happy to leave it for now 👍