Skip to content

Exceptions thrown in test fixtures result in bad lein test :only suggestion #2694

@frenchy64

Description

@frenchy64

Initial debugging steps
Before creating a report, especially around exceptions being thrown when running Leiningen, please check if the error still occurs after:

  • Updating to using the latest released version of Leiningen (lein upgrade).
  • Moving your ~/.lein/profiles.clj (if present) out of the way. This contains third-party dependencies and plugins that can cause problems inside Leiningen.
  • Updating any old versions of plugins in your project.clj, especially if the problem is with a plugin not working. Old versions of plugins like nREPL and CIDER (as well as others) can cause problems with newer versions of Leiningen.
  • (If you are using Java 9 or newer), updating your dependencies to their most recent versions. Recent JDK's have introduced changes which can break some Clojure libraries.

Describe the bug
When a test fails, I expect to be able to copy-paste the suggested lein test :only ... log into a shell to reproduce the issue.

However, if an exception is thrown in a test fixture, a strange suggestion is printed with the namespace and name of the symbol being identical.

For example, if the test is in namespace foo.bar, the suggestion command is lein test :only foo.bar/foo.bar.

To Reproduce
Steps to reproduce the behavior:

  1. Attach a test fixture to a test using clojure.test/use-fixtures (using :each or :only) and clojure.test/deftest.
  2. Run lein test such that the test in step 1 is run.
  3. See that suggested lein test :only .. command is wrong.

Actual behavior
An irrelevant lein test :only ... suggestion was printed.

Expected behavior
Either:

  • no lein test :only ... suggestion is printed
  • a lein test :only <the-ns> suggestion is printed that runs all tests in the namespace (perhaps this is the best alternative for :once fixtures)
  • a lein test :only <the-ns>/<the-test suggestion is printed (perhaps for :each fixtures).
  • some other explanation that this failure may be difficult to reproduce

Link to sample project
https://github.com/frenchy64/fixture-only

Logs

AMBONNAI-M-F0Y6 fixture-only$ lein test

lein test fixture-only.core-test

lein test :only fixture-only.core-test/fixture-only.core-test

ERROR in (fixture-only.core-test) (core_test.clj:4)
Uncaught exception in test fixture
expected: nil
  actual: java.lang.Exception: null
 at fixture_only.core_test$fixture.invokeStatic (core_test.clj:4)
    fixture_only.core_test$fixture.invoke (core_test.clj:4)
    clojure.test$compose_fixtures$fn__9731$fn__9732.invoke (test.clj:694)
    clojure.test$default_fixture.invokeStatic (test.clj:687)
    clojure.test$default_fixture.invoke (test.clj:683)
    clojure.test$compose_fixtures$fn__9731.invoke (test.clj:694)
    clojure.test$test_vars.invokeStatic (test.clj:731)
    clojure.test$test_all_vars.invokeStatic (test.clj:737)
    clojure.test$test_ns.invokeStatic (test.clj:758)
    clojure.test$test_ns.invoke (test.clj:743)
    user$eval224$fn__285.invoke (form-init5593054482259167404.clj:1)
    clojure.lang.AFn.applyToHelper (AFn.java:156)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$apply.invoke (core.clj:660)
    leiningen.core.injected$compose_hooks$fn__154.doInvoke (form-init5593054482259167404.clj:1)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:665)
    clojure.core$apply.invoke (core.clj:660)
    leiningen.core.injected$run_hooks.invokeStatic (form-init5593054482259167404.clj:1)
    leiningen.core.injected$run_hooks.invoke (form-init5593054482259167404.clj:1)
    leiningen.core.injected$prepare_for_hooks$fn__159$fn__160.doInvoke (form-init5593054482259167404.clj:1)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.AFunction$1.doInvoke (AFunction.java:31)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.core$map$fn__5866.invoke (core.clj:2755)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.Cons.next (Cons.java:39)
    clojure.lang.RT.boundedLength (RT.java:1792)
    clojure.lang.RestFn.applyTo (RestFn.java:130)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.test$run_tests.invokeStatic (test.clj:768)
    clojure.test$run_tests.doInvoke (test.clj:768)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:665)
    clojure.core$apply.invoke (core.clj:660)
    user$eval224$fn__297$fn__330.invoke (form-init5593054482259167404.clj:1)
    user$eval224$fn__297$fn__298.invoke (form-init5593054482259167404.clj:1)
    user$eval224$fn__297.invoke (form-init5593054482259167404.clj:1)
    user$eval224.invokeStatic (form-init5593054482259167404.clj:1)
    user$eval224.invoke (form-init5593054482259167404.clj:1)
    clojure.lang.Compiler.eval (Compiler.java:7177)
    clojure.lang.Compiler.eval (Compiler.java:7167)
    clojure.lang.Compiler.load (Compiler.java:7636)
    clojure.lang.Compiler.loadFile (Compiler.java:7574)
    clojure.main$load_script.invokeStatic (main.clj:475)
    clojure.main$init_opt.invokeStatic (main.clj:477)
    clojure.main$init_opt.invoke (main.clj:477)
    clojure.main$initialize.invokeStatic (main.clj:508)
    clojure.main$null_opt.invokeStatic (main.clj:542)
    clojure.main$null_opt.invoke (main.clj:539)
    clojure.main$main.invokeStatic (main.clj:664)
    clojure.main$main.doInvoke (main.clj:616)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.Var.applyTo (Var.java:705)
    clojure.main.main (main.java:40)

Ran 0 tests containing 1 assertions.
0 failures, 1 errors.
Tests failed.

Environment

  • Leiningen Version: 2.9.4
  • Leiningen installation method: [e.g. Homebrew, apt, manual].
  • JDK Version: openjdk version "1.8.0_242"
  • OS: macOS 10.15.6

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions