-
Notifications
You must be signed in to change notification settings - Fork 0
231 lines (205 loc) · 10.6 KB
/
Copy pathmain.yml
File metadata and controls
231 lines (205 loc) · 10.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
name: CI
on: [push, pull_request]
jobs:
test:
name: Build and test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
# SDL is only needed by the frontend, which the job below covers, and
# nothing under tests/ links it. Leaving it out here keeps the job that
# matters - the one running the ROM oracles - independent of the GUI.
- name: Install build tools
run: |
# Kept from the previous workflow: without this, every apt-get install
# on a GitHub runner spends a long time rebuilding the man-db index.
sudo rm -f /var/lib/man-db/auto-update
sudo apt-get update
sudo apt-get install -y --no-install-recommends ninja-build cmake g++
# The suite runs against external test ROMs and hardware-derived vectors
# which are NOT committed - they are unlicensed dumps. Without them the
# ROM suites fail outright, so fetching happens in its own step: a
# network problem then fails HERE, visibly, instead of surfacing as a
# dozen inexplicable emulation failures further down.
#
# fetch_single_step_tests.sh is deliberately absent. It is 1.1 GB of
# generated JSON, which is not reasonable to pull on every push. Those
# tests call GTEST_SKIP when the vectors are missing, so they report as
# skipped rather than passing silently - hence the skip count below.
- name: Fetch test fixtures
run: |
tests/test_files/fetch_nestest.sh
tests/test_files/fetch_blargg_ppu.sh
tests/test_files/fetch_cpu_interrupts.sh
tests/test_files/fetch_ppu_address_space.sh
tests/test_files/fetch_blargg_ppu_2005.sh
tests/test_files/fetch_sprite_hit.sh
tests/test_files/fetch_ppu_read_buffer.sh
tests/test_files/fetch_sprite_overflow.sh
tests/test_files/fetch_read_joy3.sh
tests/test_files/fetch_visual_roms.sh
tests/test_files/fetch_cpu_behaviour.sh
tests/test_files/fetch_instr_test.sh
tests/test_files/fetch_cpu_exec_space.sh
tests/test_files/fetch_mmc3.sh
# The default build type is Checked (-O3 -g, asserts live). The suite
# spends nearly all its time emulating, so an unoptimised build is about
# 5x slower; and Release/RelWithDebInfo would define NDEBUG and silently
# neuter every assert-based regression test.
- name: Configure
run: cmake -S . -B build -G Ninja -DNES_BUILD_FRONTEND=OFF
- name: Build
run: cmake --build build
- name: Test
run: ctest --test-dir build -j"$(nproc)" --output-on-failure
# "100% tests passed out of 875" is a MISLEADING headline here, and this
# step exists to say so on every run.
#
# GTEST_SKIP makes a test exit 0, so ctest counts a skipped test as a
# passing one. The 512 per-opcode SingleStepTests cases skip in CI
# because their 1.1 GB of vectors is not fetched - so the honest figure
# is the difference, not the total. What CI really covers is the ROM
# oracles: nestest's 8991 instructions, Klaus, the Blargg PPU and
# interrupt suites, and the sprite-hit reader.
- name: Report what was actually verified
if: always()
run: |
listing=$(ctest --test-dir build -N)
total=$(echo "$listing" | sed -n 's/^Total Tests: //p')
# commercialRom tests need a ROM the user supplies; CI never has one.
skipped=$(echo "$listing" | grep -cE 'SingleStepBusTrace|SingleStepVectors|commercialRom' || true)
echo "registered: $total"
echo "skipped: $skipped (SingleStepTests vectors + commercialRom, neither available in CI)"
echo "executed: $((total - skipped))"
sanitizers:
name: Test under ASan + UBSan
runs-on: ubuntu-latest
# The emulator is a pile of pointer arithmetic over fixed-size arrays -
# bus mirroring, PPU nametable folding, OAM indexing - and the suite drives
# it for hundreds of millions of cycles. That is exactly the shape of code
# ASan is good at, and until now the sanitizers had been run once, by hand,
# by a reviewer. This job makes that routine.
#
# It is a separate job from `test` rather than a step inside it so the
# normal suite still reports in a few seconds; this one is ~5x slower and
# should not sit in front of that feedback.
steps:
- uses: actions/checkout@v5
- name: Install build tools
run: |
sudo rm -f /var/lib/man-db/auto-update
sudo apt-get update
sudo apt-get install -y --no-install-recommends ninja-build cmake g++
# Same fixtures as the `test` job, and same omission of
# fetch_single_step_tests.sh - see the comment there. Those 512 tests
# skip, so this job executes 359 of them.
- name: Fetch test fixtures
run: |
tests/test_files/fetch_nestest.sh
tests/test_files/fetch_blargg_ppu.sh
tests/test_files/fetch_cpu_interrupts.sh
tests/test_files/fetch_ppu_address_space.sh
tests/test_files/fetch_blargg_ppu_2005.sh
tests/test_files/fetch_sprite_hit.sh
tests/test_files/fetch_ppu_read_buffer.sh
tests/test_files/fetch_sprite_overflow.sh
tests/test_files/fetch_read_joy3.sh
tests/test_files/fetch_visual_roms.sh
tests/test_files/fetch_cpu_behaviour.sh
tests/test_files/fetch_instr_test.sh
tests/test_files/fetch_cpu_exec_space.sh
tests/test_files/fetch_mmc3.sh
# NES_SANITIZE keeps the default build type (Checked: -O3 -g, asserts
# live) and adds instrumentation on top. Asserts staying live matters
# here - they are the project's other correctness net, and a sanitizer
# build that quietly dropped them would cover less than the normal one.
- name: Configure
run: cmake -S . -B build -G Ninja -DNES_BUILD_FRONTEND=OFF -DNES_SANITIZE=address,undefined
- name: Build
run: cmake --build build
# detect_leaks is ON deliberately. The usual argument for turning it off
# - that a short-lived test binary leaking at exit is harmless noise - is
# an argument about tolerating a known-dirty baseline. This suite has no
# such baseline: it currently reports zero leaks, so leaving detection on
# costs nothing and makes the next leak a build failure instead of a
# thing nobody notices.
#
# detect_stack_use_after_return is measurably free here (19.9s vs 19.6s,
# inside the noise) and catches a class of bug the rest of this job
# cannot, so it is on too.
#
# print_stacktrace makes UBSan reports name a file:line instead of just
# a message. The binary is built with -fno-sanitize-recover=all, so a
# UBSan hit aborts and fails the test rather than printing and moving on.
- name: Test
env:
ASAN_OPTIONS: detect_leaks=1:detect_stack_use_after_return=1
UBSAN_OPTIONS: print_stacktrace=1:report_error_type=1
run: ctest --test-dir build -j"$(nproc)" --output-on-failure
analyze:
name: Clang static analyzer (scan-build)
runs-on: ubuntu-latest
# The sanitizers above and the ROM oracles both need the code to RUN before
# they can say anything. That is a real limit: a bug on a path no test
# takes is invisible to a suite of 875 passing tests. The analyzer walks
# paths symbolically instead of executing them, so it covers the arms of
# every branch the tests happen not to enter.
#
# NOTE WHAT IS MISSING FROM THIS JOB: there is no fixture fetching, because
# nothing is executed. That makes it the only job here that cannot be
# turned red by a network problem or a moved ROM, and the only one that
# still says something useful when the download hosts are down.
steps:
- uses: actions/checkout@v5
# scan-build ships with Clang; on Debian and Ubuntu it lives in
# clang-tools rather than in clang itself. g++ is still installed because
# scan-build interposes on the EXISTING compiler - it wraps g++ for the
# real compilation and runs clang alongside it purely to analyse. Keeping
# g++ as the compiler is deliberate: it means this job's compile warnings
# are the same ones the `test` job produces, so a new line in this log is
# an analyzer finding rather than a difference of dialect. Building with
# clang instead surfaced four extra warnings that are a separate
# question - -Wmismatched-tags, -Wunused-const-variable, -Wgnu-case-range
# - and would have buried the thing this job exists to report.
- name: Install build tools
run: |
sudo rm -f /var/lib/man-db/auto-update
sudo apt-get update
sudo apt-get install -y --no-install-recommends ninja-build cmake g++ clang clang-tools
# The script owns the invocation - both scan-build calls, the exclusion
# of _deps, and --status-bugs, which is what turns a finding into a
# non-zero exit instead of a line nobody reads. See its header for the
# measurements and for the check that it can actually go red.
- name: Analyze
run: tests/run_scan_build.sh
# The one-line warning in the log names a file and a line; the HTML
# report is the path that reaches it, which is the part worth having when
# the finding is a six-branch route through the PPU. Only on failure -
# a clean run produces no reports to upload.
- name: Upload analyzer reports
if: failure()
uses: actions/upload-artifact@v4
with:
name: scan-build-report
path: build-scan/report
if-no-files-found: ignore
frontend:
name: Build the SDL2/ImGui frontend
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
# libsdl2-dev is the only system package the frontend needs; Dear ImGui
# is fetched from source by CMake and compiled into the build, so there
# is no GUI toolkit to install beyond SDL itself.
- name: Install build tools and SDL2
run: |
sudo rm -f /var/lib/man-db/auto-update
sudo apt-get update
sudo apt-get install -y --no-install-recommends ninja-build cmake g++ libsdl2-dev
- name: Configure
run: cmake -S . -B build -G Ninja -DNES_BUILD_FRONTEND=ON
# Compile-checks the frontend and its ImGui dependency. This job cannot
# RUN it - a GitHub runner has no display - so it proves the code builds
# and nothing more. The tests are covered by the job above.
- name: Build the frontend
run: cmake --build build --target nes_frontend