Files
Neutron/.planning/phases/01-reproducible-fidelity-baseline/01-VALIDATION.md
T

4.9 KiB
Raw Blame History

phase, slug, status, nyquist_compliant, wave_0_complete, created
phase slug status nyquist_compliant wave_0_complete created
01 reproducible-fidelity-baseline draft false false 2026-11-12

Phase 01 — Validation Strategy

Test Infrastructure

Property Value
Framework Existing Python stdlib unittest, standalone .NET 10 console projects; proposed Node built-in tests for pure viewer logic
Config file Existing scripts/test.sh and scripts/env.sh; no new package dependencies
Quick run command python3 -m unittest discover -s tests -p 'test_fidelity_*.py' -v (new tests created with their owning slice)
Viewer quick command node --test tests/fidelity-viewer.test.mjs (new test file created with viewer slice)
Full suite command scripts/test.sh plus the viewer quick command
Runtime Quick target below 30 seconds; measure during execution. Full GPU/original-data suite runs at wave boundaries and has a separate measured duration.

Sampling Rate

  • After every task: run its focused tests; require a positive test count, not empty discovery.
  • After each plan: run all fidelity tests and relevant existing managed/runtime regressions when their boundaries changed.
  • After each wave and before verify-work: run scripts/test.sh and all newly introduced viewer checks.
  • Every runnable automated command has an adjacent fails_when predicate in PLAN.md.
  • Synthetic media and receipt fixtures validate tooling only. Actual Windows/DxWnd and native gameplay receipts require identified execution.

Per-Task Verification Map

To be bound to final task IDs after planning/checking; planner must supply exact per-task automated commands and threat references. Required coverage:

Behavior Requirement / decisions Verification
Original preservation, bounded fresh staging and source/copy distinction EVID-01, D-15 Existing inventory plus temporary-tree negative tests
Receipt identity, imported media/event bounds, failed/incomplete runs remain open EVID-02, EVID-04, D-13–D-16 Fidelity Python tests with tamper/missing-media/NaN/partial-run fixtures
Capability ledger, all requirements, scope-aware current/history, impact ordering EVID-03, D-09–D-12 Fidelity Python tests and generated local evidence review
Full route, natural/skipped, fresh/retained, normal/edge inputs EVID-02, EVID-04, D-01–D-04 Matrix validator plus actual recorded original/native runs
Paired playback, opt-in overlays, real speed/drift, exclusive audio D-05–D-08 Node logic tests plus actual local browser/media review

Wave 0 Requirements

  • Each implementation slice creates meaningful tests together with its production code; no empty stubs counted as validation.
  • Create fidelity test fixtures covering changed/missing/added originals, output overlap, symlink/drive/case escape, reused session, tampered hash, invalid marker and unsupported schema.
  • Create pure viewer tests including asymmetric durations, missing/buffering tracks, preserved playbackRate 1.0 and exclusive audio switching.
  • No external test-framework installation required. Existing full suite remains available.

Manual-Only Verifications

Behavior Requirement Why Manual Test Instructions
Original visible and audible through DxWnd EVID-02 User Windows PC and wrapper/media behavior Use generated verified copy; record Windows/DxWnd/recorder versions and settings; execute the defined matrix and import hashed recordings/annotations
Native Windows port route/input/audio EVID-04 Cannot be replaced by Linux cross-publish or Wine Run local checkpoint bundle on user PC; capture actual route and physical input/audio; collect exact package/resource identities and logs
Native Linux route/input/audio EVID-04 Synthetic/headless fixtures have narrower scope Run equivalent identified package/matrix on Linux with actual input, audible checks and captures
Local playback review D-05–D-08 Browser codecs, audible output and media interaction Load real paired recordings, inspect optional overlays, event seeks/drift and source audio switching without retiming

Unavailable/failed executions are recorded as open dependencies. Completing tools or preparing a bundle does not close EVID-02/EVID-04 or mark Phase 1 accepted. Source-backed later work may continue only under the existing roadmap exception.

Validation Sign-Off

  • Final plans bind every task to automated checks or explicit human/native evidence
  • Sampling continuity has no three consecutive tasks without automated verification
  • New test files exist and execute nonzero test counts
  • No watch-mode flags
  • Quick feedback latency measured below 30 seconds
  • Native/original capture limitations recorded
  • nyquist_compliant updated only after the implemented validation contract is audited

Approval: Pending execution; this draft describes validation, not results.