Implicit `javac` Task Doesn't Receive Project Map as Produced by `update-in`

[timvisher]

Initial debugging steps
Before creating a report, especially around exceptions being thrown when running Leiningen, please check if the error still occurs after:

• Updating to using the latest released version of Leiningen (lein upgrade).
• Moving your ~/.lein/profiles.clj (if present) out of the way. This contains third-party dependencies and plugins that can cause problems inside Leiningen.
• Updating any old versions of plugins in your project.clj, especially if the problem is with a plugin not working. Old versions of plugins like nREPL and CIDER (as well as others) can cause problems with newer versions of Leiningen.
• (If you are using Java 9 or newer), updating your dependencies to their most recent versions. Recent JDK's have introduced changes which can break some Clojure libraries.

Describe the bug
Implicit javac Task Doesn't Receive Project Map as Produced by update-in.

To Reproduce
Steps to reproduce the behavior:

1. git clone https://github.com/timvisher/leiningen-bug-update-in-implicit-javac && cd leiningen-bug-update-in-implicit-javac
2. Ensure docker, curl, tar, and mktemp all are present.
3. Run the command ./run_bug_repro
4. See that the final execution fails despite update-in being used to correct :java-cmd.

Actual behavior
javac uses the original definition of :java-cmd rather than the updated definition.

Expected behavior
javac uses the update-ined definition of :java-cmd to use the correct install of javac.

Link to sample project
https://github.com/timvisher/leiningen-bug-update-in-implicit-javac

Logs
See https://github.com/timvisher/leiningen-bug-update-in-implicit-javac#log-of-local-run-2020-03-19t101112

Environment

• Leiningen Version:

  Leiningen 2.9.3 on Java 1.8.0_242 OpenJDK 64-Bit Server VM
  

• Leiningen installation method: manual
• JDK Version:

  openjdk version "1.8.0_242"
  OpenJDK Runtime Environment (Zulu 8.44.0.13-CA-linux64) (build 1.8.0_242-b20)
  OpenJDK 64-Bit Server VM (Zulu 8.44.0.13-CA-linux64) (build 25.242-b20, mixed mode)
  

• OS: Ubuntu 18.04
• Anything else that might be relevant to your problem?
  N/A

Additional context
N/A

[timvisher]

A workaround if you're in an ephemeral environment (like GitHub Actions) is to use lein change rather than lein update-in to physically change the project.clj file. Unfortunately (and presumably because of the bug outlined here) this must be done in a separate lein process from the one doing the rest of your activities in. In other words, lein do foo, change …, change …, bar won't work. You need to do lein do change …, change … && lein do foo, bar.

[technomancy]

I think this might be an inherent problem in how we implement profile merging.

In order to allow javac to work for pure Java projects, we have to sneak in a Clojure dependency just for the duration of the javac task, because javac is implemented in Clojure. The way we sneak dependencies into the project is using a profile, and the way profile merging works, it goes back to the original pristine project map as it was read from the project.clj file, and then merges the profile into that.

We could fix this by not using profiles in javac and splicing the Clojure dependency in manually. This would involve changing the leiningen.javac/javac-project-for-subprocess function. Happy to take a patch for this.

In the mean time, a recommended workaround is to use a profile instead of update-in to effect changes to the project map, because that should work reliably.