Add scipy-stubs as development depedency

[jorenham]

Description

I noticed that scipy is used quite a bit and that you're using mypy. SciPy doesn't support type-checking on its own, so you'll need scipy-stubs to get the full static typing benefits, such as improved IDE support (e.g. import autocompletion and inline documentation).
I'm it to the conda deps, which I assume is what you use for development.

I'm afraid I wasn't able to locally run the mypy script (using uv (because I'm a conda noob) on the main branch it reports that 8 files "unexpectedly failed"). So I can't guarantee that the CI will pass in one go 😅.

Related Issue

• Closes #
• Related to #

Checklist

• Checked that the pre-commit linting/style checks pass
• Included tests that prove the fix is effective or that the new feature works
• Added necessary documentation (docstrings and/or example notebooks)
• If you are a pro: each commit corresponds to a relevant logical change

Type of change

• New feature / enhancement
• Bug fix
• Documentation
• Maintenance
• Other (please specify): Static typing

---

📚 Documentation preview 📚: https://pytensor--1598.org.readthedocs.build/en/1598/

[maresb]

Looks great to me, thanks for the tip!

[jorenham]

Should I fix the new mypy errors this PR, or ignore them for now, so that they can be addressed in a follow-up?

Several (maybe even most) of them that don't seem to be related to scipy-stubs I believe

[ricardoV94]

I suspect they are indeed related to the changes

[maresb]

@jorenham, only a subset of files are currently maintained with passing mypy. See at the bottom of the logs:

!!!!!!!!!
13 files unexpectedly failed.
/home/runner/work/pytensor/pytensor/pytensor/link/jax/linker.py
/home/runner/work/pytensor/pytensor/pytensor/link/numba/dispatch/linalg/decomposition/lu.py
/home/runner/work/pytensor/pytensor/pytensor/link/numba/dispatch/linalg/decomposition/lu_factor.py
/home/runner/work/pytensor/pytensor/pytensor/link/numba/dispatch/linalg/decomposition/qr.py
/home/runner/work/pytensor/pytensor/pytensor/link/numba/dispatch/linalg/solve/lu_solve.py
/home/runner/work/pytensor/pytensor/pytensor/link/numba/dispatch/vectorize_codegen.py
/home/runner/work/pytensor/pytensor/pytensor/npy_2_compat.py
/home/runner/work/pytensor/pytensor/pytensor/scan/utils.py
/home/runner/work/pytensor/pytensor/pytensor/tensor/conv/abstract_conv.py
/home/runner/work/pytensor/pytensor/pytensor/tensor/einsum.py
/home/runner/work/pytensor/pytensor/pytensor/tensor/random/rewriting/numba.py
/home/runner/work/pytensor/pytensor/pytensor/tensor/rewriting/blockwise.py
/home/runner/work/pytensor/pytensor/pytensor/tensor/rewriting/numba.py
These files did not fail before, so please check the above output for errors

Would you be able to make a best-effort attempt to clean up the failing annotations in just these indicated files, ignoring failures in other files?

The good news is that a large number of the relevant failures are just unused ignores, thanks to the now-improved typing.

[jorenham]

Would you be able to make a best-effort attempt to clean up the failing annotations in just these indicated files, ignoring failures in other files?

Yea sure no problem. It's just that I wonder whether some of these aren't also occurring on main at the moment. Because there's quite a bit of overlap between them when I run the mypy script locally on main, where I see 8 errors.

[maresb]

Because there's quite a bit of overlap between them when I run the mypy script locally on main, where I see 8 errors.

Indeed!!! I can reproduce locally:

8 files unexpectedly failed.
/tmp/pytensor/pytensor/link/jax/linker.py
/tmp/pytensor/pytensor/link/numba/dispatch/vectorize_codegen.py
/tmp/pytensor/pytensor/npy_2_compat.py
/tmp/pytensor/pytensor/scan/utils.py
/tmp/pytensor/pytensor/tensor/einsum.py
/tmp/pytensor/pytensor/tensor/random/rewriting/numba.py
/tmp/pytensor/pytensor/tensor/rewriting/blockwise.py
/tmp/pytensor/pytensor/tensor/rewriting/numba.py

@ricardoV94, assuming that this is due to an uncontrolled change of version of some package, this is the type of problem that lockfiles (#1542) would solve.

[maresb]

Ah, I think I see what's going on. Due to the way we've defined the caching key, this PR busts the package cache because it changes the environment files. That's why #1599 passes.

If I add a comment to the environment.yml, it should bust the cache and trigger the 8 errors. I'll do that now...

[jorenham]

Ah, I think I see what's going on. Due to the way we've defined the caching key, this PR busts the package cache because it changes the environment files. That's why #1599 passes.

If I add a comment to the environment.yml, it should bust the cache and trigger the 8 errors. I'll do that now...

"There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors."

[maresb]

Hey @jorenham, sorry about the broken mypy. I fixed it in #1599, and it looks like that's turning green.

Hopefully rebasing on that branch will unblock you. Thanks so much! 🚀

[jorenham]

The scipy-stubs-induced mypy squiggly lines should now be gone. Do you want me to rebase already, or should I wait until #1599 is merged?

[maresb]

Amazing, thanks so much @jorenham!!!

I'm fairly confident about #1599. I'll make you a deal: if you rebase on that, then I'll fix any future merge conflicts that may arise as a result.

[codecov]

Codecov Report

❌ Patch coverage is 45.16129% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.70%. Comparing base (75e8f99) to head (d84e12e).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
...sor/link/numba/dispatch/linalg/decomposition/qr.py	7.69%	12 Missing ⚠️
...k/numba/dispatch/linalg/decomposition/lu_factor.py	20.00%	4 Missing ⚠️
pytensor/tensor/conv/abstract_conv.py	85.71%	1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (45.16%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1598      +/-   ##
==========================================
- Coverage   81.71%   81.70%   -0.02%     
==========================================
  Files         230      230              
  Lines       52935    52950      +15     
  Branches     9404     9404              
==========================================
+ Hits        43257    43262       +5     
- Misses       7246     7256      +10     
  Partials     2432     2432              

Files with missing lines	Coverage Δ
...sor/link/numba/dispatch/linalg/decomposition/lu.py	66.66% <100.00%> (ø)
...ensor/link/numba/dispatch/linalg/solve/lu_solve.py	43.63% <100.00%> (+2.12%)	⬆️
pytensor/tensor/conv/abstract_conv.py	76.10% <85.71%> (+0.02%)	⬆️
...k/numba/dispatch/linalg/decomposition/lu_factor.py	55.31% <20.00%> (-1.50%)	⬇️
...sor/link/numba/dispatch/linalg/decomposition/qr.py	19.19% <7.69%> (-0.22%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jorenham]

m fairly confident about #1599. I'll make you a deal: if you rebase on that, then I'll fix any future merge conflicts that may arise as a result.

Haha sure; rebased

[maresb]

Excellent. Now that #1599 is in, the rebase was justified. This is now ready to go.

[maresb]

Only the last two commits are new. The rest are from me and have already been merged.

(@jorenham, if you get around to it, could you rebase once more on the latest main since the merge to tidy the history?)

Thanks!!!

[jorenham]

I'm guessing that the test failure is unrelated?

[maresb]

Looks like it might be related to #1602

I'll let you guess what is my suggested mitigation this time... 😂

[jessegrabowski]

@maresb this is good to merge now?

[maresb]

@jessegrabowski, yes!

[jessegrabowski]

@jorenham thanks for this, you're on an epic streak across the repo :D