test(01-07): integrate counted fidelity regressions and exact local results
Run existing fidelity Python, Node and observer checks with positive-count guards and explicit TAP output. Record the completed 782.05-second local suite and preserve human acceptance at 0/120.
This commit is contained in:
1 parent
b732256751
commit
472500bf4e
5 files changed
+128
-1
No files matched your search
@@ -1,5 +1,28 @@
|
||||
# Compatibility and validation
|
||||
|
||||
## Phase 1 final checkpoint — materials ready, runtime acceptance OPEN
|
||||
|
||||
Fresh `dist/fidelity/phase01` readiness passes11,000 immutable members. Strict audit
|
||||
exits1 with0/120 accepted and ten pending human findings. Failed Linux preflight
|
||||
`linux-01-06-matrix-preflight-linux` identifies no display/auto_null only; it is
|
||||
diagnostic evidence, not captured gameplay. Native Windows and actual original-
|
||||
DxWnd remain unexecuted; automated GPU/SDL fixtures cannot establish physical
|
||||
input/audibility. Runtime packages retain446930d dirty/diff90cef956, separately
|
||||
from later tools/docs commits; originals and generated packages/media stay local.
|
||||
|
||||
Complete local suite: exit 0 in 782.05s (runner: 783s), with 556 Python
|
||||
(180 fidelity), 27 Node, 23 managed observer, 2 separate HUD Python tests,
|
||||
51 managed test entry points and the main build. The 36 GPU rendered receipts,
|
||||
27 saved readbacks and 17 material/readback assertions report zero validation
|
||||
errors. SDL dummy-audio checks pass; 539 original hashes match. Optional native
|
||||
window tests were not enabled; dummy consumption supplies no audible proof.
|
||||
Final exit/duration: `artifacts/fidelity/01-07-full-suite.json`, with the complete
|
||||
log alongside it; the first74.65s attempt is retained
|
||||
separately for the corrected Node reporter/count guard failure. Exact final
|
||||
checkpoint identities: `artifacts/fidelity/verification/01-07-final-checkpoint.json`.
|
||||
See [remaining acceptance](evidence/phase01-baseline.md). All EVID requirements and
|
||||
Phase1 acceptance stay OPEN under the roadmap source-backed continuation exception.
|
||||
|
||||
## Phase 1 plan 01-06 — reviewed tooling evidence
|
||||
|
||||
Primary repeats23 capture tests; executor records156 broader fidelity tests
|
||||
|
||||
@@ -1,5 +1,26 @@
|
||||
# Decisions
|
||||
|
||||
## Final fidelity audit — phase 1 plan 01-07
|
||||
|
||||
Readiness verifies immutable prepared bytes and current copied materials;
|
||||
strict acceptance separately requires complete original/native matrix coverage,
|
||||
matching package/source/resource identities and receipt-bound human judgments.
|
||||
Structural `--allow-open` success never promotes acceptance. Preserve raw collector
|
||||
receipts and failed/superseded history; the integration agent performs explicitly
|
||||
specified evidence-backed post-return normalization, while the tester only
|
||||
collects, observes and returns local files. A1/A2, both unclassified assumptions
|
||||
and P1/P2/P3 remain visible until actual review. No universal fidelity tolerance,
|
||||
automatic physical/audible pass or movement/landing reset is introduced.
|
||||
|
||||
Current result: ready11,000 files; strict audit human_needed,0/120. Positive-count
|
||||
guards cover the existing fidelity suites in one consolidated local run; Node
|
||||
uses explicit TAP so installed reporter defaults cannot hide the executed count.
|
||||
The completed local run exits 0 in 782.05s: 556 Python (180 fidelity), 27 Node,
|
||||
23 observer and 2 separate HUD Python tests; all 51 managed test entry points,
|
||||
the main build, 36 GPU rendered receipts, 17 material/readback assertions,
|
||||
dummy-audio checks and 539 original hashes pass. These bounded regression facts
|
||||
do not resolve physical/audible/native/original acceptance.
|
||||
|
||||
## Linux capture access — phase 1 plan 01-06
|
||||
|
||||
A common failed native display/audio preflight may explain all40 Linux open
|
||||
|
||||
@@ -1,5 +1,31 @@
|
||||
# Progress — 2026-11-07
|
||||
|
||||
## Phase 1 final preparation — acceptance human_needed
|
||||
|
||||
The fresh local `dist/fidelity/phase01` checkpoint passes readiness over11,000
|
||||
immutable files. Strict audit exits1:0/120 original-DxWnd/native Linux/native
|
||||
Windows obligations accepted, all ten human findings pending. The actual failed
|
||||
`linux-01-06-matrix-preflight-linux` receipt is imported with a diagnostic viewer;
|
||||
no gameplay was launched. Runtime source remains446930d dirty, diff90cef956;
|
||||
exact package/tool identities are local in
|
||||
`artifacts/fidelity/verification/01-07-final-checkpoint.json`.
|
||||
|
||||
The consolidated runner now rejects zero fidelity Python/viewer/observer counts.
|
||||
The complete local suite passed in 782.05s (runner clock: 783s): 556 Python
|
||||
(180 fidelity), 27 Node, 23 managed observer, 2 separate HUD Python tests and
|
||||
51 managed test entry points, followed by the main build. The log records
|
||||
36 GPU rendered receipts, 27 saved readbacks and 17 material/readback assertions,
|
||||
all with zero Vulkan validation errors; SDL dummy-audio checks pass and all
|
||||
539 originals remain unchanged. No physical input, audible or window-session
|
||||
acceptance is inferred. Receipt: `artifacts/fidelity/01-07-full-suite.json`;
|
||||
full output: `artifacts/fidelity/01-07-full-suite.log`.
|
||||
The first74.65s attempt passed Python but exposed the new Node reporter/count
|
||||
guard mismatch; explicit TAP fixes it and the complete rerun is retained.
|
||||
See [baseline](evidence/phase01-baseline.md) and the single
|
||||
[combined session](evidence/checkpoint-session.md). All EVID requirements and
|
||||
Phase1 acceptance remain OPEN; source-backed continuation follows the roadmap
|
||||
exception. Movement/gameplay/admission and historical runtime evidence are unchanged.
|
||||
|
||||
## Phase 1 plan 01-06 — reviewed completion
|
||||
|
||||
Linux capture tooling and full40-obligation matrix are complete. Primary repeats
|
||||
|
||||
@@ -1,5 +1,25 @@
|
||||
# Tasks
|
||||
|
||||
## Phase 1 final preparation — human session pending
|
||||
|
||||
- [x] Strict ready/audit tools:23 acceptance checks and24 checkpoint checks pass.
|
||||
- [x] Fresh11,000-file offline bundle and actual failed Linux diagnostic import.
|
||||
- [x] Strict audit rejects acceptance:0/120 obligations; all named human findings pending.
|
||||
- [x] Integrate positive-count fidelity Python/Node/managed observer checks into runner.
|
||||
- [x] Complete local suite: exit 0 in 782.05s; 556 Python (180 fidelity),
|
||||
27 Node, 23 observer, 2 separate HUD Python tests, 51 managed test entry points,
|
||||
main build, 36 GPU rendered receipts, 17 material/readback assertions,
|
||||
SDL dummy-audio checks and 539 unchanged originals. No window-session claim.
|
||||
- [ ] Actual original-DxWnd40/native Windows40/native Linux40 capture obligations.
|
||||
- [ ] Single combined physical/audible/browser review, A1/A2, both unclassified
|
||||
assumptions and P1/P2/P3 judgments; post-return agent-owned reconciliation.
|
||||
- [ ] Phase1/EVID-02/EVID-04 acceptance; retain all EVID requirements Pending.
|
||||
|
||||
Full-suite receipt: `artifacts/fidelity/01-07-full-suite.json`; final local bundle:
|
||||
`dist/fidelity/phase01`. See [complete checklist](evidence/checkpoint-session.md).
|
||||
The roadmap's independent source-backed recovery exception does not waive these
|
||||
acceptance dependencies or broaden gameplay admission.
|
||||
|
||||
## Phase 1 plan 01-06 — reviewed completion
|
||||
|
||||
- [x] Linux collector and23 targeted checks, full40-obligation open matrix.
|
||||
|
||||
+38
-1
@@ -1,7 +1,43 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
source "$(dirname "$0")/env.sh"
|
||||
python3 -m unittest discover -s tests -p 'test_*.py' -v
|
||||
suite_started=$SECONDS
|
||||
mkdir -p artifacts/fidelity
|
||||
python3 - <<'PY'
|
||||
import sys
|
||||
import unittest
|
||||
suite = unittest.defaultTestLoader.discover('tests', pattern='test_*.py')
|
||||
def cases(node):
|
||||
for item in node:
|
||||
if isinstance(item, unittest.TestSuite):
|
||||
yield from cases(item)
|
||||
else:
|
||||
yield item
|
||||
counts = {}
|
||||
for case in cases(suite):
|
||||
module = case.id().split('.')[0]
|
||||
if module.startswith('test_fidelity_'):
|
||||
counts[module] = counts.get(module, 0) + 1
|
||||
required = ('sessions', 'scenarios', 'review', 'ledger', 'observation',
|
||||
'checkpoint', 'capture', 'acceptance')
|
||||
assert all(counts.get('test_fidelity_' + name, 0) > 0 for name in required), counts
|
||||
print('Fidelity Python checks:', sum(counts.values()), counts, flush=True)
|
||||
sys.exit(not unittest.TextTestRunner(verbosity=2).run(suite).wasSuccessful())
|
||||
PY
|
||||
node --test --test-reporter=tap tests/fidelity-viewer.test.mjs | tee artifacts/fidelity/consolidated-viewer.log
|
||||
python3 - <<'PY'
|
||||
import re
|
||||
from pathlib import Path
|
||||
text = Path('artifacts/fidelity/consolidated-viewer.log').read_text()
|
||||
assert re.search(r'^# tests [1-9][0-9]*$', text, re.M), 'zero/missing viewer checks'
|
||||
PY
|
||||
dotnet run --project tests/GamePort.FidelityObservationTests | tee artifacts/fidelity/consolidated-observer.log
|
||||
python3 - <<'PY'
|
||||
import re
|
||||
from pathlib import Path
|
||||
text = Path('artifacts/fidelity/consolidated-observer.log').read_text()
|
||||
assert re.search(r'Fidelity observation checks: [1-9][0-9]* passed', text), 'zero/missing observer checks'
|
||||
PY
|
||||
python3 tools/extractors/prepare.py --all-scenes --audio sounds musicsecretlab
|
||||
dotnet run --project tests/GamePort.DataTests -- ../Original
|
||||
dotnet run --project tests/GamePort.CollisionTests -- ../Original
|
||||
@@ -118,3 +154,4 @@ if [[ "${NEUTRON_WINDOW_TEST:-0}" == 1 ]]; then
|
||||
dotnet run --no-build -- --scene artifacts/extracted/level1b/scene.json --smoke-window --frames 6 --screenshot artifacts/test-window.ppm
|
||||
dotnet run --no-build -- --smoke-walk --screenshot artifacts/test-walk-window.ppm
|
||||
fi
|
||||
printf 'Complete local suite passed in %s seconds; fidelity Python/Node/observer counts verified positive.\n' "$((SECONDS - suite_started))"
|
||||
Reference in new issue
Block a user