v0.2.2 — Gameplay test harness
A tooling release. It changes how the port is tested, not how it plays: with the
test probes off, which is the default, the game behaves as in v0.2.1.
New
- A combat check that reads the game's own state. A scripted One on One is
played until the game shows its match summary, and the test asserts that the
match started with the right fighters, the player's fighter walked, a hit by the
player followed a button press, its target lost health, the game recorded one
decisive result with a winner and a loser, and the summary followed. A fight
that draws correctly but ignores the pad, never does damage or never ends now
fails. - Scenario reports.
scripts/scenario_suite.pyruns one route with all its
assertions and writes a JSON, JUnit and HTML report with the log, captures and
an audio preview. Runs use a throw-away copy of a hashed save fixture, never the
save folder. - More gates per run: audio health (silence, clipping, dropped and dry
buffers), frame pacing, memory growth, and optional comparison against locally
approved images. - A longer regression run: twelve checks, adding the combat check and a
four-minute four-fighter match at the Terrordome, the venue fixed in v0.2.1. The
quick run is unchanged. - Read-only game-state probes behind
RECOMP_TEST_OBSERVATIONS=1: the fight's
simulation step, fighter positions, health and buttons, hits, damage and the
result. Notes on what was confirmed against live fights are in
docs/research/combat-telemetry.md.
See Gameplay test harness for how to run it and what
each assertion proves.
Build this version
git clone --recursive --branch v0.2.2 https://github.com/vyanhursky/DefJamFFNY-recomp.git
cd DefJamFFNY-recompUpdating from v0.2.1: the probes are attached where the game's code is translated,
so rerun the lift and the build (.\scripts\recomp.ps1, then
.\scripts\build.ps1 -Preset win-x64-release). The analysis step does not need
rerunning. Saves and settings.ini carry over.
This is a source release: no game data, generated code, captures or
executable. The tests need a local build from your own dump; hosted CI runs only
the unit tests.
Validation
On one machine (Windows 11, 2560x1440 at 120 Hz), after merging v0.2.1:
- Release: the twelve-check regression passed. The combat check passed 27 of 27
assertions (seven from game state) on a fight played to a knockout; the
four-fighter Terrordome match ran its four minutes at 60 frames a second. - Debug: eleven of twelve. The four-fighter Terrordome match crashed once,
about three and a half minutes in, inside the game's sound library
(sub_002626B5), and then ran clean twice. See the known issue below. The
combat check passed 27 of 27 on Debug as well. - Unit tests: 157 pass with the compiler available, including native tests of
the probes on synthetic fighter records; without a compiler the native ones skip. - Separately on Release: the crib-and-gym tour, a Foundation fight and a
Terrordome Free For All each passed as scenarios on an isolated save fixture.
The frame-pacing and memory limits are first settings taken from these healthy
runs, not long-calibrated budgets.
Not in this release
The testing roadmap (docs/08-testing-roadmap.md) is not finished; its checklist
marks each item done, partly done or open. Still open:
- scripted input timed in simulation steps, without which two runs of the same
script play different fights; - approved visual baselines and calibrated image tolerances;
- sounds tied to events;
- several matches in one launch, four-fighter results, two pads, save and load,
long sessions; - the three-game upstream-release comparison.
Known issues found by the new tests
- A rare crash in the sound library. One of three Debug runs of the four-fighter
Terrordome match crashed in the game's DirectSound code while it walked a voice list
read from the emulated audio chip. Not seen on Release in five such runs, and not
investigated yet. - Both scripted fights that were played to a knockout dropped exactly one audio
buffer in the slow motion after it. Not investigated, and not noticed by ear.