Skip to content

Commit a1e42b5

Browse files
committed
tests: add documentation for cross-platform testing
Document the cross-platform fixture naming scheme, Docker testing workflow, and troubleshooting for CI vs local test discrepancies. Co-authored-by: Claude <claude@anthropic.com> Signed-off-by: Joseph Mearman <joseph@mearman.co.uk>
1 parent 6616c55 commit a1e42b5

File tree

2 files changed

+109
-4
lines changed

2 files changed

+109
-4
lines changed

tests/README.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# Running Tests
2+
3+
## Cross-Platform Test Fixtures
4+
5+
Graphics test fixtures are platform-specific due to differences in:
6+
- Font rendering libraries (FreeType, HarfBuzz)
7+
- Standard library implementations
8+
- ARM toolchain behavior
9+
10+
Test fixtures are named with the format: `test_name~platform-os.pbi`
11+
- `~spalding-linux.pbi` - Generated on Linux (CI environment)
12+
- `~spalding-darwin.pbi` - Generated on macOS (local development)
13+
14+
## Local Development
15+
16+
### macOS Developers
17+
18+
**Option 1: Use Docker (Recommended)**
19+
20+
Run tests in Docker to match the CI environment exactly:
21+
22+
```bash
23+
# Run all tests
24+
./tests/run-tests-docker.sh
25+
26+
# Run specific tests
27+
./tests/run-tests-docker.sh -M "test_kickstart"
28+
29+
# Use specific board
30+
TEST_BOARD=snowy_bb2 ./tests/run-tests-docker.sh
31+
```
32+
33+
This ensures your test results match CI exactly.
34+
35+
**Option 2: Generate macOS Fixtures**
36+
37+
If you prefer to run tests natively on macOS:
38+
39+
```bash
40+
# Configure and build
41+
./waf configure --board=snowy_bb2
42+
./waf test
43+
44+
# This will generate macOS-specific fixtures (~spalding-darwin.pbi)
45+
# which will be used instead of the Linux fixtures
46+
```
47+
48+
Note: macOS-generated fixtures will differ from Linux fixtures. This is expected
49+
and doesn't indicate a problem with your changes. Use Docker to verify against CI.
50+
51+
### Linux Developers
52+
53+
Run tests normally - your environment matches CI:
54+
55+
```bash
56+
./waf configure --board=snowy_bb2
57+
./waf test
58+
```
59+
60+
## Updating Fixtures
61+
62+
When you intentionally change rendering behavior:
63+
64+
1. **Run tests in Docker** to generate new Linux fixtures:
65+
```bash
66+
./tests/run-tests-docker.sh
67+
```
68+
69+
2. **Copy the generated fixtures** from the failed test directory:
70+
```bash
71+
cp build/test/tests/failed/*-expected.pbi tests/fixtures/graphics/
72+
```
73+
74+
3. **Update filenames** to include the `-linux` suffix if needed:
75+
```bash
76+
# Rename from ~spalding.pbi to ~spalding-linux.pbi
77+
```
78+
79+
4. **Commit and push** the updated fixtures
80+
81+
## CI Environment
82+
83+
- Container: `ghcr.io/coredevices/pebbleos-docker:v3`
84+
- OS: Ubuntu 24.04 (Linux)
85+
- Board: snowy_bb2
86+
- Compiler: arm-none-eabi-gcc 14.2.Rel1
87+
88+
## Troubleshooting
89+
90+
### Tests pass locally but fail on CI
91+
92+
Run tests in Docker to reproduce CI results:
93+
```bash
94+
./tests/run-tests-docker.sh
95+
```
96+
97+
### Tests fail locally but pass on CI
98+
99+
Generate macOS-specific fixtures or use Docker for local development.
100+
101+
### Fixture naming confusion
102+
103+
The test framework automatically selects the correct fixture based on your OS:
104+
- On Linux: Uses `~spalding-linux.pbi`
105+
- On macOS: Uses `~spalding-darwin.pbi`
106+
- Falls back to `~spalding.pbi` if OS-specific doesn't exist

tests/fw/graphics/util.h

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -58,12 +58,11 @@ static const char *namecat(const char* str1, const char* str2){
5858
printf("filename and filename_xbit %s : %s\n", filename, filename_xbit);
5959
} else {
6060
#if !PLATFORM_DEFAULT
61-
// Add ~platform to files with unit-tests built for a specific platform
62-
// On macOS, append -darwin suffix to allow different fixtures for local dev
63-
// Linux (CI) uses the standard ~platform naming to match existing fixtures
61+
// On macOS, append ~platform-darwin suffix to allow different fixtures for local dev
62+
// Linux (CI) uses the base fixture name without any platform suffix
63+
#if defined(__APPLE__)
6464
strcat(filename, "~");
6565
strcat(filename, PLATFORM_NAME);
66-
#if defined(__APPLE__)
6766
strcat(filename, "-darwin");
6867
#endif
6968
#endif

0 commit comments

Comments
 (0)