`shards install` silently allows version mismatch in lib/ and fails to replace incorrect code

[sbsoftware]

I am actually describing two issues here. First, on one of my server instances, shards install checked out dependency code into lib/ that did not match the version specified in shard.lock and lib/.shards.info. The code in lib/ was from a different tag (or main/HEAD), but the lockfiles were correct.

This will probably be hard to reproduce and fix, so let's focus on my second issue:

Subsequent shards install runs did not replace the incorrect code in lib/, even though shard.lock and lib/.shards.info specified a different version. There was no warning or error about the version mismatch.

Expected Behavior

I would have expected one of the following:

• shards install always replacing the code in lib/ to match shard.lock.
• Error out or at least print a warning when the version in lib/<shard>/shard.yml does not match shard.lock.

Actual Behavior

Incorrect code persisted in lib/ across multiple shards install runs. No feedback was provided about the mismatch.

Environment

Shards version: 0.20.0 (2025-12-19)
OS: Ubuntu Noble (arm64) on EC2

Notes

• The cached repository in ~/.cache/shards/ was correct.
• Only deleting lib/ and manually and reinstalling resolved the issue.

[straight-shoota]

The directory at lib/$shard does not contain any information about which version is installed there. The installed version is recorded in lib/.shards.info. That's the source of truth for shards install. If the recorded version in that file is incorrect, that's an error.
But unfortunately, shards install can't easily detect that. A mismatch between the versions in shard.lock/lib/.shards.info and lib/$shard/shard.yml might be detectable, but there are limits to that. Generally, shards doesn't require the shard.yml version to match the tagged version. This only produces a warning. So detecting a mismatch in the installed version is not necessarily an indicator that the wrong version is installed.

I suppose technically, we could confirm whether the installed files match the file tree at the recorded version. But the whole point of lib/.shards.info is to record this information, and it should not be expected to be out of sync with the actual files. So we should usually be able to trust it.

Now the real question would be how this mess came to be. But as you mentioned, it might be impossible to trace the exact steps that lead to the mismatch.

[straight-shoota]

That being said, maybe a tool to verify the integrity of the lib/ folder could also be useful to detect local modifications and warn about that.

[sbsoftware]

But the whole point of lib/.shards.info is to record this information, and it should not be expected to be out of sync with the actual files. So we should usually be able to trust it.

Well, in this case it was out of sync. And that leads to very subtle bugs, when this phenomenon only occurs on one machine, while the resulting behavior isn't reproducible on any other system (in this case, it was a production node, and I spent hours wondering why "it works on my machine"). Checking out main instead of a tag version might often not lead to a compiler error, but only to runtime bugs. (This was a very time-consuming adventure for me 😉 )

I am already thinking of having my deploy script prune the whole lib folder before running shards install. But would it actually hurt if that command just copied the whole shard folder again in any case? To be honest, until yesterday I thought that any local modifications in lib would be overwritten by the next shards install, which doesn't seem to be the case currently.

[straight-shoota]

But would it actually hurt if that command just copied the whole shard folder again in any case?

I think this would be very surprising behaviour. We only install changed dependencies.

[sbsoftware]

I think this would be very surprising behaviour. We only install changed dependencies.

You mean with regard to (intended) local modifications? Isn't that an anti-pattern in general? I can't think of any other use case right now where it would even be noticable.

[straight-shoota]

No I mean very generally having file activity when there is not supposed to be change.

[sbsoftware]

I don't really follow - when I install something, I actually expect file activity, even if it's just a re-install. Pretty sure my Linux package manager might print a warning in this case, but it will overwrite the files nonetheless.

But regardless of this discussion, I will just prune the lib folder from my deploy script before running shards install from now on, simulating my desired behavior. Might be just me due to this incident, but I don't really trust that "keeping sync" mechanism anymore.

[ysbaddaden]

@sbsoftware Package managers will never reinstall a package when its database says it's already installed: apt-get doesn't, cargo doesn't, etc. You must force reinstallation with an explicit reinstall command or --reinstall flag, which Shards could provide since it can be useful sometimes.

Though, if you don't trust shards on your server, you can decide to always prune the lib folder, or vendor the lib folder into your app, or deploy prebuilt executables (or packages), so you need neither of Shards or Crystal on the server.

[ysbaddaden]

The issue is: why did the wrong version got installed?

If it ever happens again, it would be great to have the shard.yml, shard.lock and lib/.shards.info files, which shard got affected, and which tag/commit got actually installed, so we can try to decipher the problem.

[sbsoftware]

Package managers will never reinstall a package when its database says it's already installed: apt-get doesn't, cargo doesn't, etc.

Pacman just prints a warning a la " is up to date - reinstalling". You still have to confirm, but that is the normal flow even for new installations (and could be prevented with --noconfirm). I still admit this is just one counter-example and I am losing this battle 😉

You must force reinstallation with an explicit reinstall command or --reinstall flag, which Shards could provide since it can be useful sometimes.

I think I would be fine with a shards reinstall command, as long as it doesn't fail for not-yet-installed shards. I could then just use that instead of erasing the lib folder before. But on the other hand, the latter isn't that much effort to do, so I'm also fine with that.

The issue is: why did the wrong version got installed?

If it ever happens again, it would be great to have the shard.yml, shard.lock and lib/.shards.info files, which shard got affected, and which tag/commit got actually installed, so we can try to decipher the problem.

Of course I ran a shards install after deleting the shard folder in lib/, but technically all the files you mentioned were correct all the time and shouldn't have been changed by that run. I have doubts that this will be very helpful, but here are the contents of the files you requested. The shard with the mismatch was to_html. It should have been at 1.6.0 as correctly stated in the files, but the lib/to_html folder contained the code for 1.7.0, which at the time was also main/HEAD of its repo.
My deploy script always called shards install --production at the time the mismatch must have happened. I can't say for sure anymore, but it's possible I updated the system's packages just before, and it's likely that this included the latest shards release from December.

shard.yml

name: einkaufsliste
version: 18.5.0

dependencies:
  crumble:
    github: sbsoftware/crumble
    version: ">= 0.29.0"
  crumble-material:
    github: sbsoftware/crumble-material
    version: ">= 0.5.0"
  crumble-turbo:
    github: sbsoftware/crumble-turbo
    version: ">= 0.17.0"
  sqlite3:
    github: crystal-lang/crystal-sqlite3
  orma:
    github: sbsoftware/orma
    version: ">= 0.6.0"
  crumble-orma:
    github: sbsoftware/crumble-orma
    version: ">= 0.3.0"
  crumble-stimulus:
    github: sbsoftware/crumble-stimulus
    version: ">= 0.4.3"

shard.lock

version: 2.0
shards:
  base58:
    git: https://github.com/wyhaines/base58.cr.git
    version: 0.2.2

  crumble:
    git: https://github.com/sbsoftware/crumble.git
    version: 0.30.0

  crumble-material:
    git: https://github.com/sbsoftware/crumble-material.git
    version: 0.6.0

  crumble-orma:
    git: https://github.com/sbsoftware/crumble-orma.git
    version: 0.3.1

  crumble-stimulus:
    git: https://github.com/sbsoftware/crumble-stimulus.git
    version: 0.4.3

  crumble-turbo:
    git: https://github.com/sbsoftware/crumble-turbo.git
    version: 0.18.0

  css:
    git: https://github.com/sbsoftware/css.cr.git
    version: 0.3.0

  csuuid:
    git: https://github.com/wyhaines/csuuid.cr.git
    version: 1.0.2

  db:
    git: https://github.com/crystal-lang/crystal-db.git
    version: 0.14.0

  debug:
    git: https://github.com/sija/debug.cr.git
    version: 2.0.4

  js:
    git: https://github.com/sbsoftware/js.cr.git
    version: 1.8.0

  nbchannel:
    git: https://github.com/wyhaines/nbchannel.cr.git
    version: 0.1.0

  opentelemetry-api:
    git: https://github.com/wyhaines/opentelemetry-api.cr.git
    version: 0.5.1

  opentelemetry-sdk:
    git: https://github.com/wyhaines/opentelemetry-sdk.cr.git
    version: 0.6.2

  orma:
    git: https://github.com/sbsoftware/orma.git
    version: 0.11.0

  parse_date:
    git: https://github.com/wyhaines/parsedate.cr.git
    version: 0.1.2

  protobuf:
    git: https://github.com/jeromegn/protobuf.cr.git
    version: 2.3.1

  retriable:
    git: https://github.com/sija/retriable.cr.git
    version: 0.2.5

  splay_tree_map:
    git: https://github.com/wyhaines/splay_tree_map.cr.git
    version: 0.3.0

  sqlite3:
    git: https://github.com/crystal-lang/crystal-sqlite3.git
    version: 0.22.0

  stimulus:
    git: https://github.com/sbsoftware/stimulus.cr.git
    version: 0.6.0

  time-ext:
    git: https://github.com/wyhaines/time-ext.cr.git
    version: 1.0.1

  to_html:
    git: https://github.com/sbsoftware/to_html.cr.git
    version: 1.6.0

lib/.shards.info

---
version: 1.0
shards:
  js:
    git: https://github.com/sbsoftware/js.cr.git
    version: 1.8.0
  stimulus:
    git: https://github.com/sbsoftware/stimulus.cr.git
    version: 0.6.0
  to_html:
    git: https://github.com/sbsoftware/to_html.cr.git
    version: 1.6.0
  splay_tree_map:
    git: https://github.com/wyhaines/splay_tree_map.cr.git
    version: 0.3.0
  nbchannel:
    git: https://github.com/wyhaines/nbchannel.cr.git
    version: 0.1.0
  time-ext:
    git: https://github.com/wyhaines/time-ext.cr.git
    version: 1.0.1
  parse_date:
    git: https://github.com/wyhaines/parsedate.cr.git
    version: 0.1.2
  csuuid:
    git: https://github.com/wyhaines/csuuid.cr.git
    version: 1.0.2
  debug:
    git: https://github.com/sija/debug.cr.git
    version: 2.0.4
  opentelemetry-api:
    git: https://github.com/wyhaines/opentelemetry-api.cr.git
    version: 0.5.1
  db:
    git: https://github.com/crystal-lang/crystal-db.git
    version: 0.14.0
  protobuf:
    git: https://github.com/jeromegn/protobuf.cr.git
    version: 2.3.1
  retriable:
    git: https://github.com/sija/retriable.cr.git
    version: 0.2.5
  opentelemetry-sdk:
    git: https://github.com/wyhaines/opentelemetry-sdk.cr.git
    version: 0.6.2
  crumble:
    git: https://github.com/sbsoftware/crumble.git
    version: 0.30.0
  crumble-stimulus:
    git: https://github.com/sbsoftware/crumble-stimulus.git
    version: 0.4.3
  crumble-material:
    git: https://github.com/sbsoftware/crumble-material.git
    version: 0.6.0
  orma:
    git: https://github.com/sbsoftware/orma.git
    version: 0.11.0
  crumble-turbo:
    git: https://github.com/sbsoftware/crumble-turbo.git
    version: 0.18.0
  sqlite3:
    git: https://github.com/crystal-lang/crystal-sqlite3.git
    version: 0.22.0
  crumble-orma:
    git: https://github.com/sbsoftware/crumble-orma.git
    version: 0.3.1
  css:
    git: https://github.com/sbsoftware/css.cr.git
    version: 0.3.0
  base58:
    git: https://github.com/wyhaines/base58.cr.git
    version: 0.2.2

[ysbaddaden]

Thanks for the details. The only thing I can think of is maybe shards got confused by shard.yml at HEAD still being 1.6.0 (but with commits for 1.7.0).

That sounds dubious, Shards should still have checkout the v1.6.0 tag, not the main branch, but Shards only saves the version and not the actual commit (be it shards.lock or lib/.shards.info).