Crash when multiple install stanzas are targetting share_root for their directory

[WardBrian]

Expected Behavior

Both sets of files end up in the _build/install/share directory

Actual Behavior

Dune crashes.

Reproduction

1. Clone https://github.com/WardBrian/dune-install-crash
2. Run dune build @install
3. Get the following crash

Details

Internal error, please report upstream including the contents of _build/log.
Description:
  ("Map.add_exn: key already exists", { key = "install/default/share" })
Raised at Stdune__Code_error.raise in file "stdune__Code_error.ml", line 11,
  characters 30-62
Called from Stdlib__Map.Make.update in file "map.ml", line 287, characters
  18-28
Called from Stdlib__Set.Make.fold in file "set.ml", line 384, characters
  34-55
Called from Stdlib__Map.Make.fold in file "map.ml", line 329, characters
  19-42
Called from Stdlib__Map.Make.fold in file "map.ml", line 329, characters
  19-42
Called from Stdune__Map.Make.foldi in file "stdune__Map.ml" (inlined), line
  77, characters 25-79
Called from Stdune__Map.Make.fold in file "stdune__Map.ml" (inlined), line
  78, characters 24-65
Called from Dune_engine__Rules.directory_targets in file
  "dune_engine__Rules.ml" (inlined), lines 186-197, characters 2-57
Called from Dune_rules__Gen_rules.gen_rules.(fun) in file
  "dune_rules__Gen_rules.ml", line 711, characters 32-61
Called from Fiber__Core.O.(>>|).(fun) in file "fiber__Core.ml", line 254,
  characters 36-41
Called from Fiber__Scheduler.exec in file "fiber__Scheduler.ml", line 77,
  characters 8-11
-> required by ("gen-rules", In_build_dir "install/default")
-> required by ("load-dir", In_build_dir "install/default")
-> required by ("build-file", In_build_dir "install/default/share")
-> required by ("Rule.make", ())
-> required by ("execute-rule", { id = 6; info = Internal })
-> required by ("<unnamed>", ())
-> required by ("build-file", In_build_dir "default/bin1.install")
-> required by ("<unnamed>", ())
-> required by
   ("build-alias", { dir = In_build_dir "default"; name = "install" })
-> required by ("toplevel", ())

I must not crash.  Uncertainty is the mind-killer. Exceptions are the
little-death that brings total obliteration.  I will fully express my cases. 
Execution will pass over me and through me.  And when it has gone past, I
will unwind the stack along its path.  Where the cases are handled there will
be nothing.  Only I will remain.

_build/log contents

# dune build @install
# OCAMLPARAM: unset
# Shared cache: enabled-except-user-rules
# Shared cache location: /home/brian/.cache/dune/db
# Workspace root: /home/brian/Dev/ml/installme
# Auto-detected concurrency: 8
# Dune context:
#  { name = "default"
#  ; kind = "default"
#  ; profile = Dev
#  ; merlin = true
#  ; fdo_target_exe = None
#  ; build_dir = In_build_dir "default"
#  ; instrument_with = []
#  }
$ /home/brian/.opam/5.4.0/bin/ocamlc.opt -config > /tmp/dune_41d77e_output

Specifications

• Version of dune (output of dune --version): 3.20.2
• Version of ocaml (output of ocamlc --version): 5.4.0
• Operating system (distribution and version): Ubuntu 24.04

Additional information

Arose 'in the wild' in ocsigen/js_of_ocaml#2140

[Alizter]

At the moment, we don't inspect the insides of directory targets that are to be installed instead opting to symlink the entire directory. When such directories collide it's not obvious how we can merge their symlinks without going for the finer file-by-file symlinks as we do for source_tree.

[Alizter]

I think being able to merge the directories is the best way to go here, but will require some further thinking.

[hhugo]

As a workaround, ocsigen/js_of_ocaml#2140 now lists the content of directory targets and generate a sexp file that is included in the install stanza.

(install
 (section share_root)
 (package js_of_ocaml-compiler)
 (files
  (include install-in-share.sexp)))

[nojb]

Just to chime in to mention that the workaround in the previous comment is not enough: monorepo builds are still broken (see #13500 (comment)).