test(01-07): integrate counted fidelity regressions and exact local results

Run existing fidelity Python, Node and observer checks with positive-count guards and explicit TAP output. Record the completed 782.05-second local suite and preserve human acceptance at 0/120.
This commit is contained in:
UnixNight committed 2026-11-13 16:44:59 -05:00
1 parent b732256751
commit 472500bf4e
5 files changed
+128 -1

No files matched your search

+23
View File
@@ -1,5 +1,28 @@
# Compatibility and validation
## Phase 1 final checkpoint — materials ready, runtime acceptance OPEN
Fresh `dist/fidelity/phase01` readiness passes11,000 immutable members. Strict audit
exits1 with0/120 accepted and ten pending human findings. Failed Linux preflight
`linux-01-06-matrix-preflight-linux` identifies no display/auto_null only; it is
diagnostic evidence, not captured gameplay. Native Windows and actual original-
DxWnd remain unexecuted; automated GPU/SDL fixtures cannot establish physical
input/audibility. Runtime packages retain446930d dirty/diff90cef956, separately
from later tools/docs commits; originals and generated packages/media stay local.
Complete local suite: exit 0 in 782.05s (runner: 783s), with 556 Python
(180 fidelity), 27 Node, 23 managed observer, 2 separate HUD Python tests,
51 managed test entry points and the main build. The 36 GPU rendered receipts,
27 saved readbacks and 17 material/readback assertions report zero validation
errors. SDL dummy-audio checks pass; 539 original hashes match. Optional native
window tests were not enabled; dummy consumption supplies no audible proof.
Final exit/duration: `artifacts/fidelity/01-07-full-suite.json`, with the complete
log alongside it; the first74.65s attempt is retained
separately for the corrected Node reporter/count guard failure. Exact final
checkpoint identities: `artifacts/fidelity/verification/01-07-final-checkpoint.json`.
See [remaining acceptance](evidence/phase01-baseline.md). All EVID requirements and
Phase1 acceptance stay OPEN under the roadmap source-backed continuation exception.
## Phase 1 plan 01-06 — reviewed tooling evidence
Primary repeats23 capture tests; executor records156 broader fidelity tests
+21
View File
@@ -1,5 +1,26 @@
# Decisions
## Final fidelity audit — phase 1 plan 01-07
Readiness verifies immutable prepared bytes and current copied materials;
strict acceptance separately requires complete original/native matrix coverage,
matching package/source/resource identities and receipt-bound human judgments.
Structural `--allow-open` success never promotes acceptance. Preserve raw collector
receipts and failed/superseded history; the integration agent performs explicitly
specified evidence-backed post-return normalization, while the tester only
collects, observes and returns local files. A1/A2, both unclassified assumptions
and P1/P2/P3 remain visible until actual review. No universal fidelity tolerance,
automatic physical/audible pass or movement/landing reset is introduced.
Current result: ready11,000 files; strict audit human_needed,0/120. Positive-count
guards cover the existing fidelity suites in one consolidated local run; Node
uses explicit TAP so installed reporter defaults cannot hide the executed count.
The completed local run exits 0 in 782.05s: 556 Python (180 fidelity), 27 Node,
23 observer and 2 separate HUD Python tests; all 51 managed test entry points,
the main build, 36 GPU rendered receipts, 17 material/readback assertions,
dummy-audio checks and 539 original hashes pass. These bounded regression facts
do not resolve physical/audible/native/original acceptance.
## Linux capture access — phase 1 plan 01-06
A common failed native display/audio preflight may explain all40 Linux open
+26
View File
@@ -1,5 +1,31 @@
# Progress — 2026-11-07
## Phase 1 final preparation — acceptance human_needed
The fresh local `dist/fidelity/phase01` checkpoint passes readiness over11,000
immutable files. Strict audit exits1:0/120 original-DxWnd/native Linux/native
Windows obligations accepted, all ten human findings pending. The actual failed
`linux-01-06-matrix-preflight-linux` receipt is imported with a diagnostic viewer;
no gameplay was launched. Runtime source remains446930d dirty, diff90cef956;
exact package/tool identities are local in
`artifacts/fidelity/verification/01-07-final-checkpoint.json`.
The consolidated runner now rejects zero fidelity Python/viewer/observer counts.
The complete local suite passed in 782.05s (runner clock: 783s): 556 Python
(180 fidelity), 27 Node, 23 managed observer, 2 separate HUD Python tests and
51 managed test entry points, followed by the main build. The log records
36 GPU rendered receipts, 27 saved readbacks and 17 material/readback assertions,
all with zero Vulkan validation errors; SDL dummy-audio checks pass and all
539 originals remain unchanged. No physical input, audible or window-session
acceptance is inferred. Receipt: `artifacts/fidelity/01-07-full-suite.json`;
full output: `artifacts/fidelity/01-07-full-suite.log`.
The first74.65s attempt passed Python but exposed the new Node reporter/count
guard mismatch; explicit TAP fixes it and the complete rerun is retained.
See [baseline](evidence/phase01-baseline.md) and the single
[combined session](evidence/checkpoint-session.md). All EVID requirements and
Phase1 acceptance remain OPEN; source-backed continuation follows the roadmap
exception. Movement/gameplay/admission and historical runtime evidence are unchanged.
## Phase 1 plan 01-06 — reviewed completion
Linux capture tooling and full40-obligation matrix are complete. Primary repeats
+20
View File
@@ -1,5 +1,25 @@
# Tasks
## Phase 1 final preparation — human session pending
- [x] Strict ready/audit tools:23 acceptance checks and24 checkpoint checks pass.
- [x] Fresh11,000-file offline bundle and actual failed Linux diagnostic import.
- [x] Strict audit rejects acceptance:0/120 obligations; all named human findings pending.
- [x] Integrate positive-count fidelity Python/Node/managed observer checks into runner.
- [x] Complete local suite: exit 0 in 782.05s; 556 Python (180 fidelity),
27 Node, 23 observer, 2 separate HUD Python tests, 51 managed test entry points,
main build, 36 GPU rendered receipts, 17 material/readback assertions,
SDL dummy-audio checks and 539 unchanged originals. No window-session claim.
- [ ] Actual original-DxWnd40/native Windows40/native Linux40 capture obligations.
- [ ] Single combined physical/audible/browser review, A1/A2, both unclassified
assumptions and P1/P2/P3 judgments; post-return agent-owned reconciliation.
- [ ] Phase1/EVID-02/EVID-04 acceptance; retain all EVID requirements Pending.
Full-suite receipt: `artifacts/fidelity/01-07-full-suite.json`; final local bundle:
`dist/fidelity/phase01`. See [complete checklist](evidence/checkpoint-session.md).
The roadmap's independent source-backed recovery exception does not waive these
acceptance dependencies or broaden gameplay admission.
## Phase 1 plan 01-06 — reviewed completion
- [x] Linux collector and23 targeted checks, full40-obligation open matrix.
+38 -1
View File
@@ -1,7 +1,43 @@
#!/usr/bin/env bash
set -euo pipefail
source "$(dirname "$0")/env.sh"
python3 -m unittest discover -s tests -p 'test_*.py' -v
suite_started=$SECONDS
mkdir -p artifacts/fidelity
python3 - <<'PY'
import sys
import unittest
suite = unittest.defaultTestLoader.discover('tests', pattern='test_*.py')
def cases(node):
for item in node:
if isinstance(item, unittest.TestSuite):
yield from cases(item)
else:
yield item
counts = {}
for case in cases(suite):
module = case.id().split('.')[0]
if module.startswith('test_fidelity_'):
counts[module] = counts.get(module, 0) + 1
required = ('sessions', 'scenarios', 'review', 'ledger', 'observation',
'checkpoint', 'capture', 'acceptance')
assert all(counts.get('test_fidelity_' + name, 0) > 0 for name in required), counts
print('Fidelity Python checks:', sum(counts.values()), counts, flush=True)
sys.exit(not unittest.TextTestRunner(verbosity=2).run(suite).wasSuccessful())
PY
node --test --test-reporter=tap tests/fidelity-viewer.test.mjs | tee artifacts/fidelity/consolidated-viewer.log
python3 - <<'PY'
import re
from pathlib import Path
text = Path('artifacts/fidelity/consolidated-viewer.log').read_text()
assert re.search(r'^# tests [1-9][0-9]*$', text, re.M), 'zero/missing viewer checks'
PY
dotnet run --project tests/GamePort.FidelityObservationTests | tee artifacts/fidelity/consolidated-observer.log
python3 - <<'PY'
import re
from pathlib import Path
text = Path('artifacts/fidelity/consolidated-observer.log').read_text()
assert re.search(r'Fidelity observation checks: [1-9][0-9]* passed', text), 'zero/missing observer checks'
PY
python3 tools/extractors/prepare.py --all-scenes --audio sounds musicsecretlab
dotnet run --project tests/GamePort.DataTests -- ../Original
dotnet run --project tests/GamePort.CollisionTests -- ../Original
@@ -118,3 +154,4 @@ if [[ "${NEUTRON_WINDOW_TEST:-0}" == 1 ]]; then
dotnet run --no-build -- --scene artifacts/extracted/level1b/scene.json --smoke-window --frames 6 --screenshot artifacts/test-window.ppm
dotnet run --no-build -- --smoke-walk --screenshot artifacts/test-walk-window.ppm
fi
printf 'Complete local suite passed in %s seconds; fidelity Python/Node/observer counts verified positive.\n' "$((SECONDS - suite_started))"