GT4Recomp 1.3.0-rc16

Release candidate for testing. This is the 1.3.0-rc16 release candidate of the 1.3.0 line: CoreGS 2.0 no longer
compiles a pipeline during play, the shader preparation covers the render scale the session will use, and the complete
shader-variant mapping from TASK-205 ships in both Gran Turismo 4 v2.00 and GT4 Spec II. Both executables are built
with PGO and ThinLTO (profile-guided optimisation and thin link-time optimisation), as rc15 was, so this build
carries the new shader behaviour at the optimised race performance. It is a pre-release: the launcher offers release
candidates only on the Beta channel, so Stable users do not see it. If something looks or behaves wrong, report it with
the launcher's data\logs folder and the session's game.log; that report is what decides whether this candidate
becomes the final 1.3.0.

CoreGS 2.0 stops compiling graphics pipelines in the middle of a race: an unknown shader variant is drawn with the
general pipeline (the same pixels) and recorded, and the next Play's preparation compiles it. The preparation now expands
the variant list over the session's render scale, and the list itself now covers every draw state the TASK-205 mapping found
in both games: on the recorded Paris scene at 4x, 0 deferred keys and identical frames, with the 1.9 ms per VBlank the
fallback used to cost gone. Both games are in this build, both compiled with PGO and ThinLTO.

New

• The complete shader-variant mapping ships in both games (TASK-208). TASK-205 recorded 81 sessions over GT4 v2.00 and
  Spec II and mapped every draw state that asks for a CoreGS 2.0 pipeline. The shipped list now holds 644 keys: the 544
  that shipped before plus 100 new, covering 204 draw-state signatures (60 of them new). The new states are the ones
  the older recordings never met: the v2.00 photo mode and license test (the first Zformat=Z16 depth states), the used-car
  showroom, the Spec II license and menu states, the SCANMSK row-mask flag, region-clamp texture wraps, raw CT24 textures,
  alpha-test and blend combinations, and the car-material function Tfx=Highlight2. A key is draw state only, so one
  list serves both games: this build ships gt4.exe (v2.00) and gt4-spec2.exe (Spec II) from the same source and the
  same list. It is still well under the 2048-pipeline cap.

• The shader preparation covers the session's render scale (TASK-206). The render scale is part of the CoreGS 2.0
  pipeline key: a key is a draw state at a scale, so a 4x session asks for a parallel set of every state it touches.
  --prepare-shaders now takes --prepare-scale N (N is the CoreGS 2.0 internal multiplier, GT4_HW_SCALE, 1 to 12,
  never the Render scale percentage), and variants_to_prepare expands every shipped and recorded key over it:

  • the 1x shape of the key: the sampled image at 1x (the atlas, a replacement image, a mipmapped game texture, a 1x
    effect target);
  • and, when the key samples a texture image and the frame is above 1x, the target shape: the same image drawn at
    the scale, the main frame included. Push bits 28 and 30 (depth from position, fine dither) follow the frame: set above
    1x where the key keeps them and cleared at 1x.

  A key recorded at another scale is not added as it is (no session at this scale can ask for it); its state is kept
  through its 1x shape and returns at any scale by expansion. The expansion runs before the card/feature filters, the
  quarantine and the 2048-key cap, and the expanded list is what the preparation writes to variants.bin, so the session
  queues exactly what was prepared. No shader, key-format or specialisation change.

• The launcher passes the scale it will use (TASK-206). The launcher resolves the CoreGS 2.0 multiplier from the
  display and the Render scale exactly as its session does (launch::coregs20_scale, the same number it sends as
  GT4_HW_SCALE) and passes it to the preparation, so there is no guess and no second source of truth. The preparation
  prints shaders scale N and shaders added K, logs N variants for Sx (R recorded, K added for the scale), and the
  launcher logs shaders: N keys prepared at Sx. Without the option the preparation keeps its old behaviour, scale 1, and
  says so in its log.

• The preparation stamp carries the scale (TASK-206). PreparedStamp (magic GT4PREP2) now holds the scale next to
  the variant count and hash. A session whose GT4_HW_SCALE does not match the stamp uses no prepared list: it compiles
  nothing, draws with the general pipeline (the same pixels) and records its keys for the next preparation. Changing the
  Render scale is therefore a documented new preparation, not a silent mismatch.

Fixes

• CoreGS 2.0 never compiles a pipeline during play (TASK-204). A player on rc15 at 4x met 0xC0000005 inside a
  pipeline compile on the preparation thread 20 minutes into a lap (compile_max_ms 1,047.9). The in-play compile had no
  retry loop and no quarantine: an unknown key now enters the session's recorded list exactly once, returns no pipeline,
  and its batch draws with the general (uber) pipeline, which produces the same pixels. The session start recreates only
  the pipelines a matching prepared.bin vouches for, from the disk cache. GT4_CORE2_INGAME_COMPILE=1 restores the old
  compile path for an A/B; a developer run without a shader cache keeps compiling.
• Recorded keys make the next Play prepare again (TASK-204). prepared.bin did not hash variants.bin, so after a
  session recorded new keys the preparation still reported itself ready and the game compiled them itself. The stamp now
  carries the FNV-1a hash and the count of variants.bin; the preparation writes the stamp of the list it compiled, so
  new keys always ask for a new preparation. A missing, empty or damaged variants.bin has a defined stamp value, so a
  bad stamp only asks for a preparation.
• One strike in play quarantines a key (TASK-204). A key left in compiling.bin at a game start is quarantined at
  once, so the same driver crash cannot repeat every start. The launcher's preparation keeps its two strikes
  (TASK-170c), and a quarantined key is never recorded again (no preparation loop).
• The launcher's preparation logs name the recorded keys (TASK-204). The preparation prints shaders recorded R and
  the launcher logs shaders: R recorded keys compiled; the game's [core2] specialised pipelines= line gained
  deferred=N deferred_uber_batches=M.

Performance

• The optimised build is back, for both games. Both executables are compiled with PGO (profile-guided
  optimisation) and ThinLTO, from one profile per game taken on that game's recorded scenes (Spec II: George V Paris
  at 1x with Performance Mode and at 2x, the Spec II night race at 2x; v2.00: the night race at 2x and Arcade at 1x), the
  rc15 recipe. Where it acts, it shows: the prepared 4x Spec II Paris scene took 7.41 ms per frame against the plain
  rc16 test build's 8.44 ms - the same scene, list and cache, headless - 1.03 ms per frame less, about 12%, while
  the graphics work stayed the same (gpu_busy 3.79 against 3.78 ms): the gain is on the processor side.
• A 4x session prepared at 4x stops paying the fallback (TASK-206, larger list in TASK-208). Headless Spec II, George
  V Paris, B-Spec, 4x, 2,400 VBlanks of race, four frame dumps (VBlank 4800/5400/6000/6600). The deferred run (fresh
  cache) recorded 75 keys, created 0 pipelines and drew 2,645,167 batches with the general pipeline. --prepare-shaders --prepare-scale 4 then compiled 245 variants, 49 added, 0 failed. The prepared run had 0 deferred keys, 0
  deferral-caused uber batches, 245 pipelines recreated from the disk cache in 231.9 ms, and 4 of 4 frames identical
  (0 differing pixels). Over the race's four 600-VBlank intervals, gpu_busy was 3.79 against 5.28 ms per VBlank
  (prepared against deferred) and wall time 7.41 against 7.97 ms: 1.49 ms per VBlank less of graphics work, the
  fallback's cost. The residual is 178 general-pipeline batches in the first moments, before the worker finished
  recreating the pipelines; they are not deferrals.
• The same gate on v2.00. Headless v2.00 night race (GT Mode, B-Spec, Hong Kong Reverse), 4x, 11,500 VBlanks: the
  deferred run recorded 97 keys and 3,287,021 general-pipeline batches; the preparation compiled 245 variants and the
  prepared run had 0 deferred keys, 0 deferral-caused uber batches and 9 of 9 identical frames (pipelines recreated
  from the cache in 184.9 ms), with gpu_busy 3.94 against 5.27 ms per VBlank over four race intervals and wall time
  6.61 against 7.20 ms (measured again with the interval profile on, the same scene and dumps).
• The variant list on this card (RX 9060 XT, before the card filters). The 644-key corpus becomes 315 distinct keys
  at 1x (644 written) and 406 at 2x/4x/6x (917 written, 273 keys with both shapes; the count is the same above 1x
  because the shape replaces the scale with 1 or S). After the card's plain/feature filters, 245 keys at 4x (49
  targets) where the 544-key list gave 175, and 196 at 1x. A bigger list, and every state of the mapping specialised
  after one preparation.
• The mapping covers the tracks it measured. Spec II, 1x, with the list prepared: Nordschleife, Tsukuba, Infineon
  Sports and Hong Kong - the four tracks that asked for the most new keys - ran with 0 deferred keys and 0
  deferral-caused uber batches each, where the 544-key list deferred 6 to 25 keys per track.

The same game

• Both games in this build. gt4.exe is Gran Turismo 4 v2.00 (aspect profile) and gt4-spec2.exe is GT4 Spec II
  (its own target, IOP runtime and VU0 tables, with the Spec II aspect-hooked EE library). The same source, the same
  shaders and the same 644-key variant list. The launcher installs and updates both.
• Pixels and game logic, TASK-208: against the published rc15 PGO builds, headless: Spec II three recorded laps at 1x
  (George V Paris, El Capitan, Tokyo R246), state.bin identical and the race clock identical, 24, 35 and 35 frames
  identical; Spec II at 6x, 12 of 12 frames; v2.00 night race at 3x, state.bin identical (12,097 VBlanks both,
  4,786 racing) and 10 of 10 frames; v2.00 Arcade at 3x, 14 of 14 frames; VU1/VU0 verification and the MTVU check
  with 0 mismatches (VU1 24,096,768 and VU0 3,973,120 calls checked on Spec II, VU1 36,249,600 and VU0 5,615,616 on
  v2.00, 0 interpreted and 0 mismatches; MTVU 8,032 frames, 0 mismatches). No physics, timing, AI, save or game-logic
  change.
• Pixels, TASK-204: headless v2.00 night race at 1x, 11,500 VBlanks, dumps from VBlank 6500 every 600. The default
  run (0 pipelines, 97 keys deferred, 3,153,803 general-pipeline batches caused by them) and the run with
  GT4_CORE2_INGAME_COMPILE=1 (97 pipelines, 35,498.9 ms of driver compile, max 503.1 ms) produced 9 of 9 identical
  frames (SHA-256 per frame).
• Pixels, TASK-206/208: the 4x gates above, 4 of 4 identical frames on Spec II and 9 of 9 on v2.00 between the
  deferred and the prepared run.

Checks

• Native tests: the full CTest of both PGO builds, 321 tests each: 320 passed, 1 skipped
  (hw_replace_q07_pcsx2_dumps, the known skip), 0 failed (480.6 s v2.00, 511.9 s Spec II). hw_core2_* 36 of 36
  (127.0 s). The hw_core2_prepare_scale fixture covers the transformation at 1x/2x/4x/6x with the new corpus (315 and
  406 distinct keys, both shapes and their push words, no duplicate), a real preparation at 4x (the file holds the
  expanded list, the stamp names the scale, a second preparation does nothing), a 1x session refusing a 4x preparation
  (0 pipelines, deferred) and using a 1x preparation (0 deferred, the same pixels), and the cap.
• Launcher: cargo test --workspace, 784 passed, 9 ignored, 0 failed (34 test binaries). This build changes no
  launcher code.
• Python suite: 804 tests, OK.
• Distribution: both executables boot headless from the distribution folder, loading the nine shaders from their own
  shaders\; --portable-check reports the installed folder complete (9 of 9 shaders, no build-time fallback) for both,
  and the binaries contain no build or user path. The distribution's files match the two new pins by size and SHA-256.

Known issues

• The optimisation is trained on recorded scenes. PGO and ThinLTO with the rc15 recipe, one profile per game from its
  own recorded scenes; a scene far from them gets plain code quality, never different behaviour.
• TASK-201 is not included (planned for later). rc15 already has TASK-199's info-card layer; the extra check of the next
  row's "(i)" icon and label over an open card is not in this build.
• The first Play after a Render scale change prepares again. The cache prepared for the old scale is refused on
  purpose (the scale is in the stamp): that session draws the unseen keys with the general pipeline (the same pixels) and
  records them; the next Play prepares at the new scale. The same happens once after this update.
• A failed or skipped preparation leaves every variant on the general pipeline (the same pixels, more fragment cost)
  until the next Play's preparation. A quarantined variant stays there on this card and driver; the way back is deleting
  the shader cache's quarantine.bin (the support log lists it).
• Photo mode on slower graphics cards (CoreGS 2.0): each picture still waits for the graphics card twice, so frame
  times can be uneven (TASK-162). Unchanged.
• Not every mode of the mapping is recorded. Split-screen racing (waits for a second controller), replay (no menu
  path), the Spec II GT Mode race and photo mode (disabled by the mod) and the Spec II rotating showroom (the used-car
  dealer display was recorded instead) are not covered by the list. A session that meets an unrecorded state draws it with
  the general pipeline (the same pixels), records it, and the next Play prepares it (TASK-204/207).
• 2x/3x/6x/12x sessions are covered by the list fixture, not by pixels in game; the headless gates are the 4x Paris
  and night scenes and the 1x spot check. In-game testing covers the other scales.

Install or update

• From an earlier 1.3.0 build: run the installer below. Settings, saves and replays are kept.
• Installer: run GT4Recomp-Setup-1.3.0-rc16.exe.
• Portable: unpack GT4Recomp-1.3.0-rc16-portable.zip anywhere. Settings, saves and data stay inside the folder.
• Both games are in the build: gt4.exe for Gran Turismo 4 v2.00 and gt4-spec2.exe for GT4 Spec II; the launcher
  shows Spec II when its disc and derived data are set up, as in rc15.
• After the update, the first Play may prepare CoreGS 2.0's shaders once, as after every update; the preparation now
  uses the Render scale the launcher will pass to the game and the new, larger list.
• Disc images: you need your own Gran Turismo 4 (USA, v2.00); for Spec II, your own GT4 Spec II disc. GT4Recomp ships
  no copyrighted game files.

Requirements

Windows 10 or 11, 64-bit, a processor with SSE4.1. CoreGS 2.0 needs a graphics card with Vulkan 1.2.

Credits and legal

Our code is MIT. GSdx is the PCSX2 GS (GPL-3.0-or-later), downloaded by the launcher and run in its own process; the game
executable contains no GPL code. Thanks to the player whose 4x crash report on rc15 started TASK-204, and to the TASK-205
survey that mapped what creates a shader variant in both games.