Skip to content

Commit 3cb6262

Browse files
committed
tests: add documentation for cross-platform testing
Document the cross-platform fixture naming scheme, Docker testing workflow, and troubleshooting steps for CI vs local test discrepancies. Co-authored-by: Claude <claude@anthropic.com> Signed-off-by: Joseph Mearman <joseph@mearman.co.uk>
1 parent 057155a commit 3cb6262

File tree

1 file changed

+106
-0
lines changed

1 file changed

+106
-0
lines changed

tests/README.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# Running Tests
2+
3+
## Cross-Platform Test Fixtures
4+
5+
Graphics test fixtures are platform-specific due to differences in:
6+
- Font rendering libraries (FreeType, HarfBuzz)
7+
- Standard library implementations
8+
- ARM toolchain behavior
9+
10+
Test fixtures are named with the format: `test_name~platform-os.pbi`
11+
- `~spalding-linux.pbi` - Generated on Linux (CI environment)
12+
- `~spalding-darwin.pbi` - Generated on macOS (local development)
13+
14+
## Local Development
15+
16+
### macOS Developers
17+
18+
**Option 1: Use Docker (Recommended)**
19+
20+
Run tests in Docker to match the CI environment exactly:
21+
22+
```bash
23+
# Run all tests
24+
./tests/run-tests-docker.sh
25+
26+
# Run specific tests
27+
./tests/run-tests-docker.sh -M "test_kickstart"
28+
29+
# Use specific board
30+
TEST_BOARD=snowy_bb2 ./tests/run-tests-docker.sh
31+
```
32+
33+
This ensures your test results match CI exactly.
34+
35+
**Option 2: Generate macOS Fixtures**
36+
37+
If you prefer to run tests natively on macOS:
38+
39+
```bash
40+
# Configure and build
41+
./waf configure --board=snowy_bb2
42+
./waf test
43+
44+
# This will generate macOS-specific fixtures (~spalding-darwin.pbi)
45+
# which will be used instead of the Linux fixtures
46+
```
47+
48+
Note: macOS-generated fixtures will differ from Linux fixtures. This is expected
49+
and doesn't indicate a problem with your changes. Use Docker to verify against CI.
50+
51+
### Linux Developers
52+
53+
Run tests normally - your environment matches CI:
54+
55+
```bash
56+
./waf configure --board=snowy_bb2
57+
./waf test
58+
```
59+
60+
## Updating Fixtures
61+
62+
When you intentionally change rendering behavior:
63+
64+
1. **Run tests in Docker** to generate new Linux fixtures:
65+
```bash
66+
./tests/run-tests-docker.sh
67+
```
68+
69+
2. **Copy the generated fixtures** from the failed test directory:
70+
```bash
71+
cp build/test/tests/failed/*-expected.pbi tests/fixtures/graphics/
72+
```
73+
74+
3. **Update filenames** to include the `-linux` suffix if needed:
75+
```bash
76+
# Rename from ~spalding.pbi to ~spalding-linux.pbi
77+
```
78+
79+
4. **Commit and push** the updated fixtures
80+
81+
## CI Environment
82+
83+
- Container: `ghcr.io/coredevices/pebbleos-docker:v3`
84+
- OS: Ubuntu 24.04 (Linux)
85+
- Board: snowy_bb2
86+
- Compiler: arm-none-eabi-gcc 14.2.Rel1
87+
88+
## Troubleshooting
89+
90+
### Tests pass locally but fail on CI
91+
92+
Run tests in Docker to reproduce CI results:
93+
```bash
94+
./tests/run-tests-docker.sh
95+
```
96+
97+
### Tests fail locally but pass on CI
98+
99+
Generate macOS-specific fixtures or use Docker for local development.
100+
101+
### Fixture naming confusion
102+
103+
The test framework automatically selects the correct fixture based on your OS:
104+
- On Linux: Uses `~spalding-linux.pbi`
105+
- On macOS: Uses `~spalding-darwin.pbi`
106+
- Falls back to `~spalding.pbi` if OS-specific doesn't exist

0 commit comments

Comments
 (0)