Skip to content

Commit 7edddd1

Browse files
committed
Migrate selftests to pytest
This moves selftests into 10 focused pytest modules under 'tests', with shared fixtures and helpers in tests/conftest.py. Original testsuite.py now delegates selftests to pytest and retains only kernel compatibility tests. - Add pytest.ini with testpaths and compat marker - Add tests/conftest.py with shared fixtures and assertion helpers (verify_value, assign_and_verify, verify_str, etc.) - Split inline tests into test_{symbols,repr,expressions,semantics, properties,deps,config_io,lexer,preprocess,compat} - Update CI workflow to run pytest selftests as a separate step It also removes testsuite.py in favor of pytest test modules.
1 parent dab1a67 commit 7edddd1

16 files changed

Lines changed: 3805 additions & 3681 deletions

‎.github/workflows/test.yml‎

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,9 @@ jobs:
1818
strategy:
1919
fail-fast: false
2020
matrix:
21-
# NOTE: Full testsuite on Windows is disabled because the test script
22-
# requires Linux kernel source tree. However, headless mode tests
23-
# are now enabled for Windows.
21+
# NOTE: Windows runs only headless + menuconfig import tests (no kernel
22+
# tree, no Unix shell). Linux/macOS run selftests, compatibility
23+
# tests, and example scripts.
2424
target:
2525
# Python 3.12
2626
- python: '3.12'
@@ -57,14 +57,12 @@ jobs:
5757
5858
- name: Install Python dependencies
5959
run: |
60-
pip install --user setuptools wheel
60+
python -m pip install --user setuptools wheel pytest
6161
6262
- name: Check out Linux source code
6363
# Skip for Windows (headless-only mode)
6464
if: ${{ matrix.target.headless-only != true }}
6565
uses: actions/checkout@v6
66-
# On Windows, checkout of 'aux.c' is expected to fail because ... Windows.
67-
continue-on-error: true
6866
with:
6967
repository: torvalds/linux
7068
ref: v5.4
@@ -76,18 +74,31 @@ jobs:
7674
# Linux/macOS (full test): checkout to Kconfiglib subdirectory
7775
path: ${{ matrix.target.headless-only && '.' || 'Kconfiglib' }}
7876

77+
- name: Run pytest selftests
78+
# Skip on Windows: several tests depend on Unix shell ($(shell,...)),
79+
# gcc, and forward-slash paths that are unavailable on Windows CI.
80+
if: ${{ matrix.target.headless-only != true }}
81+
working-directory: ${{ matrix.target.headless-only && '.' || 'Kconfiglib' }}
82+
run: |
83+
python -m pytest tests/ -v --tb=short --ignore=tests/test_compat.py
84+
7985
- name: Apply Linux Kconfig Makefile patch
8086
# Skip for Windows (headless-only mode)
8187
if: ${{ matrix.target.headless-only != true }}
8288
run: |
8389
git apply Kconfiglib/makefile.patch
8490
85-
- name: Run testsuite
91+
- name: Run compatibility tests and example scripts
8692
# Skip for Windows (headless-only mode)
8793
if: ${{ matrix.target.headless-only != true }}
8894
run: |
8995
Kconfiglib/tests/reltest python
9096
97+
- name: Install windows-curses (Windows only)
98+
if: matrix.target.os == 'Windows'
99+
run: |
100+
python -m pip install windows-curses
101+
91102
- name: Test headless mode
92103
# Use root dir for Windows, Kconfiglib subdir for Linux/macOS
93104
working-directory: ${{ matrix.target.headless-only && '.' || 'Kconfiglib' }}
@@ -101,11 +112,6 @@ jobs:
101112
print('Headless mode test passed')
102113
EOF
103114
104-
- name: Install windows-curses (Windows only)
105-
if: matrix.target.os == 'Windows'
106-
run: |
107-
pip install windows-curses
108-
109115
- name: Test menuconfig import (Windows Python 3.12)
110116
if: matrix.target.os == 'Windows'
111117
# Use root dir for Windows (headless-only mode)

‎README.md‎

Lines changed: 17 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -300,7 +300,7 @@ This warning can also be toggled by setting `Kconfig.warn_assign_undef` to `True
300300

301301
- Robust and highly compatible with the C Kconfig tools
302302

303-
The [test suite](testsuite.py) automatically compares output from Kconfiglib and
303+
The [test suite](tests/) automatically compares output from Kconfiglib and
304304
the C tools by diffing the generated `.config` files for the real kernel Kconfig and defconfig files across all ARCHes.
305305
Currently, this involves comparing output for 36 ARCHes and 498 defconfig files (or over 18,000 ARCH/defconfig combinations in
306306
"obsessive" test suite mode). All tests are expected to pass.
@@ -462,32 +462,33 @@ Some examples include:
462462
463463
## Test suite
464464
465-
The test suite is run with:
465+
The self-tests can be run from the project root with [pytest](https://docs.pytest.org/):
466466
```shell
467-
python Kconfiglib/testsuite.py
467+
python -m pytest tests/ -v
468468
```
469469
470-
[pypy](https://pypy.org/) also works and is much faster for most tasks,
471-
except for `allnoconfig.py`, `allnoconfig_simpler.py`, and `allyesconfig.py`,
472-
where it has no time to warm up because those scripts are invoked via `make scriptconfig`.
473-
474-
The test suite must be run from the top-level kernel directory.
475-
It requires that the Kconfiglib Git repository has been cloned into it and that the Makefile patch has been applied.
470+
To run the full suite -- self-tests, compatibility tests against the C Kconfig tools, and example scripts -- use
471+
[tests/reltest](tests/reltest) from the top-level kernel directory (requires the Makefile patch):
472+
```shell
473+
Kconfiglib/tests/reltest python
474+
```
476475
477476
To suppress warnings generated for the kernel `Kconfig` files, redirect `stderr` to `/dev/null`:
478477
```
479-
python Kconfiglib/testsuite.py 2>/dev/null
478+
Kconfiglib/tests/reltest python 2>/dev/null
480479
```
481480
482-
Note: Forgetting to apply the Makefile patch will cause some tests that compare generated configurations to fail.
481+
[pypy](https://pypy.org/) also works and is much faster for most tasks,
482+
except for `allnoconfig.py`, `allnoconfig_simpler.py`, and `allyesconfig.py`,
483+
where it has no time to warm up because those scripts are invoked via `make scriptconfig`.
483484
484-
Note: The test suite overwrites `.config` in the kernel root, so make sure to back it up.
485+
Note: Forgetting to apply the Makefile patch will cause some compatibility tests that compare generated configurations to fail.
485486
486-
The test suite consists of a set of self-tests and a set of compatibility tests that compare configurations
487-
generated by Kconfiglib with those generated by the C tools across various scenarios.
488-
See [testsuite.py](testsuite.py) for available options.
487+
Note: The compatibility tests overwrite `.config` in the kernel root, so make sure to back it up.
489488
490-
The [tests/reltest](tests/reltest) script runs the test suite and all example scripts to verify that everything works as expected.
489+
The test suite consists of self-tests (under [tests/](tests/)) and compatibility tests
490+
([tests/test_compat.py](tests/test_compat.py)) that compare configurations generated by Kconfiglib
491+
with those generated by the C tools across various scenarios.
491492
492493
Occasionally, the C tools' output may change slightly (for example, due to a [recent change](https://www.spinics.net/lists/linux-kbuild/msg17074.html)).
493494
If the test suite reports failures, try running it again against the [linux-next tree](https://www.kernel.org/doc/man-pages/linux-next.html),

‎pytest.ini‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[pytest]
2+
testpaths = tests
3+
markers =
4+
compat: compatibility tests requiring Linux kernel source tree

‎tests/conftest.py‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Copyright (c) 2011-2019 Ulf Magnusson
2+
# SPDX-License-Identifier: ISC
3+
#
4+
# Shared fixtures and assertion helpers for the Kconfiglib pytest suite.
5+
6+
import glob
7+
import os
8+
import sys
9+
10+
import pytest
11+
12+
# Ensure kconfiglib is importable from the project root
13+
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
14+
15+
from kconfiglib import TRI_TO_STR # noqa: E402
16+
17+
# ---------------------------------------------------------------------------
18+
# Fixtures
19+
# ---------------------------------------------------------------------------
20+
21+
22+
@pytest.fixture(autouse=True)
23+
def _clean_env(monkeypatch):
24+
"""Save and restore environment variables between tests.
25+
26+
Also removes KCONFIG_ALLCONFIG to prevent accidental config loading.
27+
"""
28+
monkeypatch.delenv("KCONFIG_ALLCONFIG", raising=False)
29+
yield
30+
31+
32+
@pytest.fixture(autouse=True)
33+
def _cleanup_config_files():
34+
"""Remove config_test* files after each test."""
35+
yield
36+
tests_dir = os.path.join(os.path.dirname(__file__))
37+
for f in glob.glob(os.path.join(tests_dir, "config_test*")):
38+
os.remove(f)
39+
# Also clean from project root (some tests write there)
40+
project_root = os.path.join(os.path.dirname(__file__), "..")
41+
for f in glob.glob(os.path.join(project_root, "config_test*")):
42+
os.remove(f)
43+
44+
45+
# ---------------------------------------------------------------------------
46+
# Assertion helpers
47+
#
48+
# These take an explicit Kconfig instance `c` rather than closing over one.
49+
# ---------------------------------------------------------------------------
50+
51+
52+
def verify_value(c, sym_name, val):
53+
"""Verify that a symbol has a particular value."""
54+
if isinstance(val, int):
55+
val = TRI_TO_STR[val]
56+
57+
sym = c.syms[sym_name]
58+
assert sym.str_value == val, f"{sym_name} value mismatch"
59+
60+
61+
def assign_and_verify_value(c, sym_name, val, new_val):
62+
"""Assign val to a symbol and verify its value becomes new_val."""
63+
if isinstance(new_val, int):
64+
new_val = TRI_TO_STR[new_val]
65+
66+
sym = c.syms[sym_name]
67+
assert sym.set_value(val), f"Failed to assign '{val}' to {sym_name}"
68+
assert sym.str_value == new_val, f"{sym_name} value after assignment"
69+
70+
71+
def assign_and_verify(c, sym_name, user_val):
72+
"""Like assign_and_verify_value(), with the expected value being the
73+
value just set."""
74+
assign_and_verify_value(c, sym_name, user_val, user_val)
75+
76+
77+
def assign_and_verify_user_value(c, sym_name, val, user_val, valid):
78+
"""Assign a user value and verify the new user value and validity."""
79+
sym = c.syms[sym_name]
80+
assert sym.set_value(val) == valid, f"{sym_name} validity mismatch for '{val}'"
81+
assert sym.user_value == user_val, f"{sym_name} user_value mismatch"
82+
83+
84+
def verify_str(item, expected):
85+
"""Verify str(item) matches expected (strip leading/trailing newline)."""
86+
assert str(item) == expected[1:-1]

‎tests/reltest‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,14 @@
11
#!/usr/bin/env bash
22

3-
# Runs the test suite and all examples scripts, bailing immediately if anything
4-
# fails. For the examples that aren't tested in the test suite, we just confirm
5-
# that they at least run.
3+
# Runs compatibility tests and example scripts from a kernel tree root,
4+
# bailing immediately if anything fails. For the examples that aren't
5+
# tested by the test suite, we just confirm that they at least run.
66
#
77
# Should be run from the kernel root with $ Kconfiglib/tests/reltest
8+
#
9+
# Selftests (tests/test_*.py excluding test_compat.py) should be run
10+
# separately from the Kconfiglib project root:
11+
# $ cd Kconfiglib && python -m pytest tests/ -v
812

913
test_script() {
1014
echo -e "\n================= $1 with $py =================\n"
@@ -26,11 +30,15 @@ else
2630
py_execs=$@
2731
fi
2832

33+
kconfiglib_dir="$(cd "$(dirname "$0")/.." && pwd)"
34+
2935
for py in $py_execs; do
30-
echo -e "\n================= Test suite with $py =================\n"
36+
echo -e "\n================= Compatibility tests with $py =================\n"
3137

32-
if ! $py Kconfiglib/testsuite.py; then
33-
echo "test suite failed for $py"
38+
# Compat tests run from the kernel root (cwd) where they compare output
39+
# against the C Kconfig tools.
40+
if ! $py -m pytest "$kconfiglib_dir/tests/test_compat.py" -x -v --tb=short; then
41+
echo "compatibility tests failed for $py"
3442
exit 1
3543
fi
3644

0 commit comments

Comments
 (0)