From 472500bf4e191c2080764471b6b08486c1909b3a Mon Sep 17 00:00:00 2001 From: Cristian Cerroni Date: Fri, 13 Nov 2026 16:44:59 -0500 Subject: [PATCH] test(01-07): integrate counted fidelity regressions and exact local results Run existing fidelity Python, Node and observer checks with positive-count guards and explicit TAP output. Record the completed 782.05-second local suite and preserve human acceptance at 0/120. --- docs/compatibility.md | 23 +++++++++++++++++++++++ docs/decisions.md | 21 +++++++++++++++++++++ docs/progress.md | 26 ++++++++++++++++++++++++++ docs/tasks.md | 20 ++++++++++++++++++++ scripts/test.sh | 39 ++++++++++++++++++++++++++++++++++++++- 5 files changed, 128 insertions(+), 1 deletion(-) diff --git a/docs/compatibility.md b/docs/compatibility.md index 1618e47..48ae032 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -1,5 +1,28 @@ # Compatibility and validation +## Phase 1 final checkpoint — materials ready, runtime acceptance OPEN + +Fresh `dist/fidelity/phase01` readiness passes11,000 immutable members. Strict audit +exits1 with0/120 accepted and ten pending human findings. Failed Linux preflight +`linux-01-06-matrix-preflight-linux` identifies no display/auto_null only; it is +diagnostic evidence, not captured gameplay. Native Windows and actual original- +DxWnd remain unexecuted; automated GPU/SDL fixtures cannot establish physical +input/audibility. Runtime packages retain446930d dirty/diff90cef956, separately +from later tools/docs commits; originals and generated packages/media stay local. + +Complete local suite: exit 0 in 782.05s (runner: 783s), with 556 Python +(180 fidelity), 27 Node, 23 managed observer, 2 separate HUD Python tests, +51 managed test entry points and the main build. The 36 GPU rendered receipts, +27 saved readbacks and 17 material/readback assertions report zero validation +errors. SDL dummy-audio checks pass; 539 original hashes match. Optional native +window tests were not enabled; dummy consumption supplies no audible proof. +Final exit/duration: `artifacts/fidelity/01-07-full-suite.json`, with the complete +log alongside it; the first74.65s attempt is retained +separately for the corrected Node reporter/count guard failure. Exact final +checkpoint identities: `artifacts/fidelity/verification/01-07-final-checkpoint.json`. +See [remaining acceptance](evidence/phase01-baseline.md). All EVID requirements and +Phase1 acceptance stay OPEN under the roadmap source-backed continuation exception. + ## Phase 1 plan 01-06 — reviewed tooling evidence Primary repeats23 capture tests; executor records156 broader fidelity tests diff --git a/docs/decisions.md b/docs/decisions.md index 614805c..280ab33 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -1,5 +1,26 @@ # Decisions +## Final fidelity audit — phase 1 plan 01-07 + +Readiness verifies immutable prepared bytes and current copied materials; +strict acceptance separately requires complete original/native matrix coverage, +matching package/source/resource identities and receipt-bound human judgments. +Structural `--allow-open` success never promotes acceptance. Preserve raw collector +receipts and failed/superseded history; the integration agent performs explicitly +specified evidence-backed post-return normalization, while the tester only +collects, observes and returns local files. A1/A2, both unclassified assumptions +and P1/P2/P3 remain visible until actual review. No universal fidelity tolerance, +automatic physical/audible pass or movement/landing reset is introduced. + +Current result: ready11,000 files; strict audit human_needed,0/120. Positive-count +guards cover the existing fidelity suites in one consolidated local run; Node +uses explicit TAP so installed reporter defaults cannot hide the executed count. +The completed local run exits 0 in 782.05s: 556 Python (180 fidelity), 27 Node, +23 observer and 2 separate HUD Python tests; all 51 managed test entry points, +the main build, 36 GPU rendered receipts, 17 material/readback assertions, +dummy-audio checks and 539 original hashes pass. These bounded regression facts +do not resolve physical/audible/native/original acceptance. + ## Linux capture access — phase 1 plan 01-06 A common failed native display/audio preflight may explain all40 Linux open diff --git a/docs/progress.md b/docs/progress.md index 5bff785..0f28dba 100644 --- a/docs/progress.md +++ b/docs/progress.md @@ -1,5 +1,31 @@ # Progress — 2026-11-07 +## Phase 1 final preparation — acceptance human_needed + +The fresh local `dist/fidelity/phase01` checkpoint passes readiness over11,000 +immutable files. Strict audit exits1:0/120 original-DxWnd/native Linux/native +Windows obligations accepted, all ten human findings pending. The actual failed +`linux-01-06-matrix-preflight-linux` receipt is imported with a diagnostic viewer; +no gameplay was launched. Runtime source remains446930d dirty, diff90cef956; +exact package/tool identities are local in +`artifacts/fidelity/verification/01-07-final-checkpoint.json`. + +The consolidated runner now rejects zero fidelity Python/viewer/observer counts. +The complete local suite passed in 782.05s (runner clock: 783s): 556 Python +(180 fidelity), 27 Node, 23 managed observer, 2 separate HUD Python tests and +51 managed test entry points, followed by the main build. The log records +36 GPU rendered receipts, 27 saved readbacks and 17 material/readback assertions, +all with zero Vulkan validation errors; SDL dummy-audio checks pass and all +539 originals remain unchanged. No physical input, audible or window-session +acceptance is inferred. Receipt: `artifacts/fidelity/01-07-full-suite.json`; +full output: `artifacts/fidelity/01-07-full-suite.log`. +The first74.65s attempt passed Python but exposed the new Node reporter/count +guard mismatch; explicit TAP fixes it and the complete rerun is retained. +See [baseline](evidence/phase01-baseline.md) and the single +[combined session](evidence/checkpoint-session.md). All EVID requirements and +Phase1 acceptance remain OPEN; source-backed continuation follows the roadmap +exception. Movement/gameplay/admission and historical runtime evidence are unchanged. + ## Phase 1 plan 01-06 — reviewed completion Linux capture tooling and full40-obligation matrix are complete. Primary repeats diff --git a/docs/tasks.md b/docs/tasks.md index 262183c..fd86efa 100644 --- a/docs/tasks.md +++ b/docs/tasks.md @@ -1,5 +1,25 @@ # Tasks +## Phase 1 final preparation — human session pending + +- [x] Strict ready/audit tools:23 acceptance checks and24 checkpoint checks pass. +- [x] Fresh11,000-file offline bundle and actual failed Linux diagnostic import. +- [x] Strict audit rejects acceptance:0/120 obligations; all named human findings pending. +- [x] Integrate positive-count fidelity Python/Node/managed observer checks into runner. +- [x] Complete local suite: exit 0 in 782.05s; 556 Python (180 fidelity), + 27 Node, 23 observer, 2 separate HUD Python tests, 51 managed test entry points, + main build, 36 GPU rendered receipts, 17 material/readback assertions, + SDL dummy-audio checks and 539 unchanged originals. No window-session claim. +- [ ] Actual original-DxWnd40/native Windows40/native Linux40 capture obligations. +- [ ] Single combined physical/audible/browser review, A1/A2, both unclassified + assumptions and P1/P2/P3 judgments; post-return agent-owned reconciliation. +- [ ] Phase1/EVID-02/EVID-04 acceptance; retain all EVID requirements Pending. + +Full-suite receipt: `artifacts/fidelity/01-07-full-suite.json`; final local bundle: +`dist/fidelity/phase01`. See [complete checklist](evidence/checkpoint-session.md). +The roadmap's independent source-backed recovery exception does not waive these +acceptance dependencies or broaden gameplay admission. + ## Phase 1 plan 01-06 — reviewed completion - [x] Linux collector and23 targeted checks, full40-obligation open matrix. diff --git a/scripts/test.sh b/scripts/test.sh index eebc84a..90a7582 100755 --- a/scripts/test.sh +++ b/scripts/test.sh @@ -1,7 +1,43 @@ #!/usr/bin/env bash set -euo pipefail source "$(dirname "$0")/env.sh" -python3 -m unittest discover -s tests -p 'test_*.py' -v +suite_started=$SECONDS +mkdir -p artifacts/fidelity +python3 - <<'PY' +import sys +import unittest +suite = unittest.defaultTestLoader.discover('tests', pattern='test_*.py') +def cases(node): + for item in node: + if isinstance(item, unittest.TestSuite): + yield from cases(item) + else: + yield item +counts = {} +for case in cases(suite): + module = case.id().split('.')[0] + if module.startswith('test_fidelity_'): + counts[module] = counts.get(module, 0) + 1 +required = ('sessions', 'scenarios', 'review', 'ledger', 'observation', + 'checkpoint', 'capture', 'acceptance') +assert all(counts.get('test_fidelity_' + name, 0) > 0 for name in required), counts +print('Fidelity Python checks:', sum(counts.values()), counts, flush=True) +sys.exit(not unittest.TextTestRunner(verbosity=2).run(suite).wasSuccessful()) +PY +node --test --test-reporter=tap tests/fidelity-viewer.test.mjs | tee artifacts/fidelity/consolidated-viewer.log +python3 - <<'PY' +import re +from pathlib import Path +text = Path('artifacts/fidelity/consolidated-viewer.log').read_text() +assert re.search(r'^# tests [1-9][0-9]*$', text, re.M), 'zero/missing viewer checks' +PY +dotnet run --project tests/GamePort.FidelityObservationTests | tee artifacts/fidelity/consolidated-observer.log +python3 - <<'PY' +import re +from pathlib import Path +text = Path('artifacts/fidelity/consolidated-observer.log').read_text() +assert re.search(r'Fidelity observation checks: [1-9][0-9]* passed', text), 'zero/missing observer checks' +PY python3 tools/extractors/prepare.py --all-scenes --audio sounds musicsecretlab dotnet run --project tests/GamePort.DataTests -- ../Original dotnet run --project tests/GamePort.CollisionTests -- ../Original @@ -118,3 +154,4 @@ if [[ "${NEUTRON_WINDOW_TEST:-0}" == 1 ]]; then dotnet run --no-build -- --scene artifacts/extracted/level1b/scene.json --smoke-window --frames 6 --screenshot artifacts/test-window.ppm dotnet run --no-build -- --smoke-walk --screenshot artifacts/test-walk-window.ppm fi +printf 'Complete local suite passed in %s seconds; fidelity Python/Node/observer counts verified positive.\n' "$((SECONDS - suite_started))"