-
Notifications
You must be signed in to change notification settings - Fork 119
Description
Now that we have replay checking in headless mode and via commandline, we should use that in CI for automatic compatibility checking.
I have curated a list of replays. It started with the golden replay #1 and I added a replay each time a mismatch was discovered and where the mismatch was not detected by one of the existing list of replays:
replay_list_min.zip
It currently contains 9 replays and only requires official maps. I also tried to select replays that can be simulated quickly. I think it would make sense to use these replays in CI. We can still check more replays offline and expand that set of replays if more mismatches are discovered.
In CI, those replays first need to be unpacked into the replay folder. The game binary then needs to be called like this:
generalszh.exe -jobs 2 -headless -replay replay_list_min/*.rep
The return value needs to be evaluated (1 means mismatch or crash in one or more replays).
I'd also like to see another build configuration added where DEBUG_LOGGING and DEBUG_CRASHING are enabled (but not RTS_DEBUG or RTS_INTERNAL). That configuration should also be checked for compatibility.