Skip to content

MNT: Execute slowest tests on schedule #258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions .github/workflows/tox.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ jobs:
datalad get -J 2 -d $TEST_DATA_HOME

test:
if: github.event_name != 'schedule'
runs-on: ubuntu-latest
needs: [cache-test-data]
strategy:
Expand Down Expand Up @@ -110,6 +111,53 @@ jobs:
token: ${{ secrets.CODECOV_TOKEN }}
if: ${{ always() }}


slow-tests:
if: github.event_name == 'schedule':
runs-on: ubuntu-latest
needs: [cache-test-data]
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13"]
dependencies: [latest, pre]
include:
- python-version: "3.10"
dependencies: min
exclude:
# Do not test pre-releases for versions out of SPEC0
- python-version: "3.10"
dependencies: pre
- python-version: "3.11"
dependencies: pre

env:
DEPENDS: ${{ matrix.dependencies }}

steps:
- uses: actions/checkout@v4
with:
submodules: recursive
fetch-depth: 0
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v6
- uses: actions/cache/restore@v4
with:
path: ${{ env.TEST_DATA_HOME }}
key: data-cache-v2-${{ needs.cache-test-data.outputs.SHA }}
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- name: Install tox
run: |
uv tool install tox --with=tox-uv --with=tox-gh-actions
- name: Show tox config
run: tox c
- name: Run tox
run: tox -v -e slow-tests --exit-and-dump-after 1200

checks:
runs-on: 'ubuntu-latest'
continue-on-error: true
Expand Down
1 change: 1 addition & 0 deletions nitransforms/tests/test_resampling.py
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,7 @@ def test_displacements_field2(tmp_path, testdata_path, sw_tool):
assert np.sqrt((diff**2).mean()) < RMSE_TOL_LINEAR


@pytest.mark.slow
def test_apply_transformchain(tmp_path, testdata_path):
"""Check a translation-only field on one or more axes, different image orientations."""
os.chdir(str(tmp_path))
Expand Down
2 changes: 2 additions & 0 deletions nitransforms/tests/test_surface.py
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ def test_SurfaceCoordinateTransformIO(testdata_path, tmpdir):
assert sct == sct2


@pytest.mark.slow
def test_ProjectUnproject(testdata_path):

sphere_reg_path = (
Expand Down Expand Up @@ -155,6 +156,7 @@ def test_ProjectUnproject(testdata_path):
assert np.all(transformed._triangles == projunproj_ref.agg_data()[1])


@pytest.mark.slow
def test_SurfaceResampler(testdata_path, tmpdir):
dif_tol = 0.001
fslr_sphere_path = (
Expand Down
34 changes: 33 additions & 1 deletion tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,39 @@ uv_resolution =

commands =
python -c "import nitransforms; print(nitransforms.__version__)"
pytest --durations=20 --durations-min=1.0 --cov-report term-missing {posargs:-n auto}
pytest --durations=20 --durations-min=1.0 --cov-report term-missing {posargs:-n auto -m "not slow"}

[testenv:slow-tests]
description = Pytest with coverage
labels = test
pip_pre =
pre: true
pass_env =
# Overrides default location for finding test data
TEST_DATA_HOME
# getpass.getuser() sources for Windows:
LOGNAME
USER
LNAME
USERNAME
# Pass user color preferences through
PY_COLORS
FORCE_COLOR
NO_COLOR
CLICOLOR
CLICOLOR_FORCE
PYTHON_GIL
extras = tests
setenv =
FSLOUTPUTTYPE=NIFTI_GZ
pre: PIP_EXTRA_INDEX_URL=https://pypi.anaconda.org/scientific-python-nightly-wheels/simple
pre: UV_EXTRA_INDEX_URL=https://pypi.anaconda.org/scientific-python-nightly-wheels/simple
uv_resolution =
min: lowest-direct

commands =
python -c "import nitransforms; print(nitransforms.__version__)"
pytest --durations=20 --durations-min=1.0 --cov-report term-missing {posargs:-n auto -m slow}

[testenv:style]
description = Check our style guide
Expand Down