Files
Neutron/.planning/phases/01-reproducible-fidelity-baseline/01-VALIDATION.md
T

72 lines
4.9 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
phase: "01"
slug: "reproducible-fidelity-baseline"
status: draft
nyquist_compliant: false
wave_0_complete: false
created: "2026-11-12"
---
# Phase 01 — Validation Strategy
## Test Infrastructure
| Property | Value |
|---|---|
| Framework | Existing Python stdlib unittest, standalone .NET 10 console projects; proposed Node built-in tests for pure viewer logic |
| Config file | Existing scripts/test.sh and scripts/env.sh; no new package dependencies |
| Quick run command | `python3 -m unittest discover -s tests -p 'test_fidelity_*.py' -v` (new tests created with their owning slice) |
| Viewer quick command | `node --test tests/fidelity-viewer.test.mjs` (new test file created with viewer slice) |
| Full suite command | `scripts/test.sh` plus the viewer quick command |
| Runtime | Quick target below 30 seconds; measure during execution. Full GPU/original-data suite runs at wave boundaries and has a separate measured duration. |
## Sampling Rate
- After every task: run its focused tests; require a positive test count, not empty discovery.
- After each plan: run all fidelity tests and relevant existing managed/runtime regressions when their boundaries changed.
- After each wave and before verify-work: run scripts/test.sh and all newly introduced viewer checks.
- Every runnable automated command has an adjacent fails_when predicate in PLAN.md.
- Synthetic media and receipt fixtures validate tooling only. Actual Windows/DxWnd and native gameplay receipts require identified execution.
## Per-Task Verification Map
To be bound to final task IDs after planning/checking; planner must supply exact per-task automated commands and threat references. Required coverage:
| Behavior | Requirement / decisions | Verification |
|---|---|---|
| Original preservation, bounded fresh staging and source/copy distinction | EVID-01, D-15 | Existing inventory plus temporary-tree negative tests |
| Receipt identity, imported media/event bounds, failed/incomplete runs remain open | EVID-02, EVID-04, D-13–D-16 | Fidelity Python tests with tamper/missing-media/NaN/partial-run fixtures |
| Capability ledger, all requirements, scope-aware current/history, impact ordering | EVID-03, D-09–D-12 | Fidelity Python tests and generated local evidence review |
| Full route, natural/skipped, fresh/retained, normal/edge inputs | EVID-02, EVID-04, D-01–D-04 | Matrix validator plus actual recorded original/native runs |
| Paired playback, opt-in overlays, real speed/drift, exclusive audio | D-05–D-08 | Node logic tests plus actual local browser/media review |
## Wave 0 Requirements
- Each implementation slice creates meaningful tests together with its production code; no empty stubs counted as validation.
- Create fidelity test fixtures covering changed/missing/added originals, output overlap, symlink/drive/case escape, reused session, tampered hash, invalid marker and unsupported schema.
- Create pure viewer tests including asymmetric durations, missing/buffering tracks, preserved playbackRate 1.0 and exclusive audio switching.
- No external test-framework installation required. Existing full suite remains available.
## Manual-Only Verifications
| Behavior | Requirement | Why Manual | Test Instructions |
|---|---|---|---|
| Original visible and audible through DxWnd | EVID-02 | User Windows PC and wrapper/media behavior | Use generated verified copy; record Windows/DxWnd/recorder versions and settings; execute the defined matrix and import hashed recordings/annotations |
| Native Windows port route/input/audio | EVID-04 | Cannot be replaced by Linux cross-publish or Wine | Run local checkpoint bundle on user PC; capture actual route and physical input/audio; collect exact package/resource identities and logs |
| Native Linux route/input/audio | EVID-04 | Synthetic/headless fixtures have narrower scope | Run equivalent identified package/matrix on Linux with actual input, audible checks and captures |
| Local playback review | D-05–D-08 | Browser codecs, audible output and media interaction | Load real paired recordings, inspect optional overlays, event seeks/drift and source audio switching without retiming |
Unavailable/failed executions are recorded as open dependencies. Completing tools or preparing a bundle does not close EVID-02/EVID-04 or mark Phase 1 accepted. Source-backed later work may continue only under the existing roadmap exception.
## Validation Sign-Off
- [ ] Final plans bind every task to automated checks or explicit human/native evidence
- [ ] Sampling continuity has no three consecutive tasks without automated verification
- [ ] New test files exist and execute nonzero test counts
- [ ] No watch-mode flags
- [ ] Quick feedback latency measured below 30 seconds
- [ ] Native/original capture limitations recorded
- [ ] nyquist_compliant updated only after the implemented validation contract is audited
**Approval:** Pending execution; this draft describes validation, not results.