Skip to content

Commit 219d6fe

Browse files
committed
Migrate selftests to pytest
This moves selftests into 10 focused pytest modules under 'tests', with shared fixtures and helpers in tests/conftest.py. Original testsuite.py now delegates selftests to pytest and retains only kernel compatibility tests. - Add pytest.ini with testpaths and compat marker - Add tests/conftest.py with shared fixtures and assertion helpers (verify_value, assign_and_verify, verify_str, etc.) - Split inline tests into test_{symbols,repr,expressions,semantics, properties,deps,config_io,lexer,preprocess,compat} - Update CI workflow to run pytest selftests as a separate step It also removes testsuite.py in favor of pytest test modules.
1 parent dab1a67 commit 219d6fe

16 files changed

Lines changed: 3802 additions & 3678 deletions

‎.github/workflows/test.yml‎

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,8 @@ jobs:
1818
strategy:
1919
fail-fast: false
2020
matrix:
21-
# NOTE: Full testsuite on Windows is disabled because the test script
22-
# requires Linux kernel source tree. However, headless mode tests
23-
# are now enabled for Windows.
21+
# NOTE: Windows runs only selftests + headless tests (no kernel tree).
22+
# Linux/macOS run selftests, compatibility tests, and example scripts.
2423
target:
2524
# Python 3.12
2625
- python: '3.12'
@@ -57,7 +56,7 @@ jobs:
5756
5857
- name: Install Python dependencies
5958
run: |
60-
pip install --user setuptools wheel
59+
python -m pip install --user setuptools wheel pytest
6160
6261
- name: Check out Linux source code
6362
# Skip for Windows (headless-only mode)
@@ -76,18 +75,28 @@ jobs:
7675
# Linux/macOS (full test): checkout to Kconfiglib subdirectory
7776
path: ${{ matrix.target.headless-only && '.' || 'Kconfiglib' }}
7877

78+
- name: Run pytest selftests
79+
working-directory: ${{ matrix.target.headless-only && '.' || 'Kconfiglib' }}
80+
run: |
81+
python -m pytest tests/ -v --tb=short --ignore=tests/test_compat.py
82+
7983
- name: Apply Linux Kconfig Makefile patch
8084
# Skip for Windows (headless-only mode)
8185
if: ${{ matrix.target.headless-only != true }}
8286
run: |
8387
git apply Kconfiglib/makefile.patch
8488
85-
- name: Run testsuite
89+
- name: Run compatibility tests and example scripts
8690
# Skip for Windows (headless-only mode)
8791
if: ${{ matrix.target.headless-only != true }}
8892
run: |
8993
Kconfiglib/tests/reltest python
9094
95+
- name: Install windows-curses (Windows only)
96+
if: matrix.target.os == 'Windows'
97+
run: |
98+
python -m pip install windows-curses
99+
91100
- name: Test headless mode
92101
# Use root dir for Windows, Kconfiglib subdir for Linux/macOS
93102
working-directory: ${{ matrix.target.headless-only && '.' || 'Kconfiglib' }}
@@ -101,11 +110,6 @@ jobs:
101110
print('Headless mode test passed')
102111
EOF
103112
104-
- name: Install windows-curses (Windows only)
105-
if: matrix.target.os == 'Windows'
106-
run: |
107-
pip install windows-curses
108-
109113
- name: Test menuconfig import (Windows Python 3.12)
110114
if: matrix.target.os == 'Windows'
111115
# Use root dir for Windows (headless-only mode)

‎README.md‎

Lines changed: 17 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -300,7 +300,7 @@ This warning can also be toggled by setting `Kconfig.warn_assign_undef` to `True
300300

301301
- Robust and highly compatible with the C Kconfig tools
302302

303-
The [test suite](testsuite.py) automatically compares output from Kconfiglib and
303+
The [test suite](tests/) automatically compares output from Kconfiglib and
304304
the C tools by diffing the generated `.config` files for the real kernel Kconfig and defconfig files across all ARCHes.
305305
Currently, this involves comparing output for 36 ARCHes and 498 defconfig files (or over 18,000 ARCH/defconfig combinations in
306306
"obsessive" test suite mode). All tests are expected to pass.
@@ -462,32 +462,33 @@ Some examples include:
462462
463463
## Test suite
464464
465-
The test suite is run with:
465+
The self-tests can be run from the project root with [pytest](https://docs.pytest.org/):
466466
```shell
467-
python Kconfiglib/testsuite.py
467+
python -m pytest tests/ -v
468468
```
469469
470-
[pypy](https://pypy.org/) also works and is much faster for most tasks,
471-
except for `allnoconfig.py`, `allnoconfig_simpler.py`, and `allyesconfig.py`,
472-
where it has no time to warm up because those scripts are invoked via `make scriptconfig`.
473-
474-
The test suite must be run from the top-level kernel directory.
475-
It requires that the Kconfiglib Git repository has been cloned into it and that the Makefile patch has been applied.
470+
To run the full suite -- self-tests, compatibility tests against the C Kconfig tools, and example scripts -- use
471+
[tests/reltest](tests/reltest) from the top-level kernel directory (requires the Makefile patch):
472+
```shell
473+
Kconfiglib/tests/reltest python
474+
```
476475
477476
To suppress warnings generated for the kernel `Kconfig` files, redirect `stderr` to `/dev/null`:
478477
```
479-
python Kconfiglib/testsuite.py 2>/dev/null
478+
Kconfiglib/tests/reltest python 2>/dev/null
480479
```
481480
482-
Note: Forgetting to apply the Makefile patch will cause some tests that compare generated configurations to fail.
481+
[pypy](https://pypy.org/) also works and is much faster for most tasks,
482+
except for `allnoconfig.py`, `allnoconfig_simpler.py`, and `allyesconfig.py`,
483+
where it has no time to warm up because those scripts are invoked via `make scriptconfig`.
483484
484-
Note: The test suite overwrites `.config` in the kernel root, so make sure to back it up.
485+
Note: Forgetting to apply the Makefile patch will cause some compatibility tests that compare generated configurations to fail.
485486
486-
The test suite consists of a set of self-tests and a set of compatibility tests that compare configurations
487-
generated by Kconfiglib with those generated by the C tools across various scenarios.
488-
See [testsuite.py](testsuite.py) for available options.
487+
Note: The compatibility tests overwrite `.config` in the kernel root, so make sure to back it up.
489488
490-
The [tests/reltest](tests/reltest) script runs the test suite and all example scripts to verify that everything works as expected.
489+
The test suite consists of self-tests (under [tests/](tests/)) and compatibility tests
490+
([tests/test_compat.py](tests/test_compat.py)) that compare configurations generated by Kconfiglib
491+
with those generated by the C tools across various scenarios.
491492
492493
Occasionally, the C tools' output may change slightly (for example, due to a [recent change](https://www.spinics.net/lists/linux-kbuild/msg17074.html)).
493494
If the test suite reports failures, try running it again against the [linux-next tree](https://www.kernel.org/doc/man-pages/linux-next.html),

‎pytest.ini‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[pytest]
2+
testpaths = tests
3+
markers =
4+
compat: compatibility tests requiring Linux kernel source tree

‎tests/conftest.py‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Copyright (c) 2011-2019 Ulf Magnusson
2+
# SPDX-License-Identifier: ISC
3+
#
4+
# Shared fixtures and assertion helpers for the Kconfiglib pytest suite.
5+
6+
import glob
7+
import os
8+
import sys
9+
10+
import pytest
11+
12+
# Ensure kconfiglib is importable from the project root
13+
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
14+
15+
from kconfiglib import TRI_TO_STR # noqa: E402
16+
17+
# ---------------------------------------------------------------------------
18+
# Fixtures
19+
# ---------------------------------------------------------------------------
20+
21+
22+
@pytest.fixture(autouse=True)
23+
def _clean_env(monkeypatch):
24+
"""Save and restore environment variables between tests.
25+
26+
Also removes KCONFIG_ALLCONFIG to prevent accidental config loading.
27+
"""
28+
monkeypatch.delenv("KCONFIG_ALLCONFIG", raising=False)
29+
yield
30+
31+
32+
@pytest.fixture(autouse=True)
33+
def _cleanup_config_files():
34+
"""Remove config_test* files after each test."""
35+
yield
36+
tests_dir = os.path.join(os.path.dirname(__file__))
37+
for f in glob.glob(os.path.join(tests_dir, "config_test*")):
38+
os.remove(f)
39+
# Also clean from project root (some tests write there)
40+
project_root = os.path.join(os.path.dirname(__file__), "..")
41+
for f in glob.glob(os.path.join(project_root, "config_test*")):
42+
os.remove(f)
43+
44+
45+
# ---------------------------------------------------------------------------
46+
# Assertion helpers
47+
#
48+
# These take an explicit Kconfig instance `c` rather than closing over one.
49+
# ---------------------------------------------------------------------------
50+
51+
52+
def verify_value(c, sym_name, val):
53+
"""Verify that a symbol has a particular value."""
54+
if isinstance(val, int):
55+
val = TRI_TO_STR[val]
56+
57+
sym = c.syms[sym_name]
58+
assert sym.str_value == val, f"{sym_name} value mismatch"
59+
60+
61+
def assign_and_verify_value(c, sym_name, val, new_val):
62+
"""Assign val to a symbol and verify its value becomes new_val."""
63+
if isinstance(new_val, int):
64+
new_val = TRI_TO_STR[new_val]
65+
66+
sym = c.syms[sym_name]
67+
assert sym.set_value(val), f"Failed to assign '{val}' to {sym_name}"
68+
assert sym.str_value == new_val, f"{sym_name} value after assignment"
69+
70+
71+
def assign_and_verify(c, sym_name, user_val):
72+
"""Like assign_and_verify_value(), with the expected value being the
73+
value just set."""
74+
assign_and_verify_value(c, sym_name, user_val, user_val)
75+
76+
77+
def assign_and_verify_user_value(c, sym_name, val, user_val, valid):
78+
"""Assign a user value and verify the new user value and validity."""
79+
sym = c.syms[sym_name]
80+
assert sym.set_value(val) == valid, f"{sym_name} validity mismatch for '{val}'"
81+
assert sym.user_value == user_val, f"{sym_name} user_value mismatch"
82+
83+
84+
def verify_str(item, expected):
85+
"""Verify str(item) matches expected (strip leading/trailing newline)."""
86+
assert str(item) == expected[1:-1]

‎tests/reltest‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
#!/usr/bin/env bash
22

3-
# Runs the test suite and all examples scripts, bailing immediately if anything
4-
# fails. For the examples that aren't tested in the test suite, we just confirm
3+
# Runs the test suite and all example scripts, bailing immediately if anything
4+
# fails. For the examples that aren't tested by the test suite, we just confirm
55
# that they at least run.
66
#
77
# Should be run from the kernel root with $ Kconfiglib/tests/reltest
8+
#
9+
# Selftests can also be run standalone:
10+
# $ cd Kconfiglib && python -m pytest tests/ -v
811

912
test_script() {
1013
echo -e "\n================= $1 with $py =================\n"
@@ -27,10 +30,19 @@ else
2730
fi
2831

2932
for py in $py_execs; do
30-
echo -e "\n================= Test suite with $py =================\n"
33+
echo -e "\n================= pytest selftests with $py =================\n"
34+
35+
kconfiglib_dir="$(cd "$(dirname "$0")/.." && pwd)"
36+
if ! $py -m pytest "$kconfiglib_dir/tests/" -x -v --tb=short \
37+
--ignore="$kconfiglib_dir/tests/test_compat.py"; then
38+
echo "pytest selftests failed for $py"
39+
exit 1
40+
fi
41+
42+
echo -e "\n================= Compatibility tests with $py =================\n"
3143

32-
if ! $py Kconfiglib/testsuite.py; then
33-
echo "test suite failed for $py"
44+
if ! $py -m pytest "$kconfiglib_dir/tests/test_compat.py" -x -v --tb=short; then
45+
echo "compatibility tests failed for $py"
3446
exit 1
3547
fi
3648

0 commit comments

Comments
 (0)