docs(phase-01): add research and validation strategy

This commit is contained in:
UnixNight committed 2026-11-12 12:21:56 -05:00
1 parent 29f8cb1714
commit 36f24e4474
2 files changed
+529

No files matched your search

@@ -0,0 +1,71 @@
---
phase: "01"
slug: "reproducible-fidelity-baseline"
status: draft
nyquist_compliant: false
wave_0_complete: false
created: "2026-11-12"
---
# Phase 01 — Validation Strategy
## Test Infrastructure
| Property | Value |
|---|---|
| Framework | Existing Python stdlib unittest, standalone .NET 10 console projects; proposed Node built-in tests for pure viewer logic |
| Config file | Existing scripts/test.sh and scripts/env.sh; no new package dependencies |
| Quick run command | `python3 -m unittest discover -s tests -p 'test_fidelity_*.py' -v` (new tests created with their owning slice) |
| Viewer quick command | `node --test tests/fidelity-viewer.test.mjs` (new test file created with viewer slice) |
| Full suite command | `scripts/test.sh` plus the viewer quick command |
| Runtime | Quick target below 30 seconds; measure during execution. Full GPU/original-data suite runs at wave boundaries and has a separate measured duration. |
## Sampling Rate
- After every task: run its focused tests; require a positive test count, not empty discovery.
- After each plan: run all fidelity tests and relevant existing managed/runtime regressions when their boundaries changed.
- After each wave and before verify-work: run scripts/test.sh and all newly introduced viewer checks.
- Every runnable automated command has an adjacent fails_when predicate in PLAN.md.
- Synthetic media and receipt fixtures validate tooling only. Actual Windows/DxWnd and native gameplay receipts require identified execution.
## Per-Task Verification Map
To be bound to final task IDs after planning/checking; planner must supply exact per-task automated commands and threat references. Required coverage:
| Behavior | Requirement / decisions | Verification |
|---|---|---|
| Original preservation, bounded fresh staging and source/copy distinction | EVID-01, D-15 | Existing inventory plus temporary-tree negative tests |
| Receipt identity, imported media/event bounds, failed/incomplete runs remain open | EVID-02, EVID-04, D-13–D-16 | Fidelity Python tests with tamper/missing-media/NaN/partial-run fixtures |
| Capability ledger, all requirements, scope-aware current/history, impact ordering | EVID-03, D-09–D-12 | Fidelity Python tests and generated local evidence review |
| Full route, natural/skipped, fresh/retained, normal/edge inputs | EVID-02, EVID-04, D-01–D-04 | Matrix validator plus actual recorded original/native runs |
| Paired playback, opt-in overlays, real speed/drift, exclusive audio | D-05–D-08 | Node logic tests plus actual local browser/media review |
## Wave 0 Requirements
- Each implementation slice creates meaningful tests together with its production code; no empty stubs counted as validation.
- Create fidelity test fixtures covering changed/missing/added originals, output overlap, symlink/drive/case escape, reused session, tampered hash, invalid marker and unsupported schema.
- Create pure viewer tests including asymmetric durations, missing/buffering tracks, preserved playbackRate 1.0 and exclusive audio switching.
- No external test-framework installation required. Existing full suite remains available.
## Manual-Only Verifications
| Behavior | Requirement | Why Manual | Test Instructions |
|---|---|---|---|
| Original visible and audible through DxWnd | EVID-02 | User Windows PC and wrapper/media behavior | Use generated verified copy; record Windows/DxWnd/recorder versions and settings; execute the defined matrix and import hashed recordings/annotations |
| Native Windows port route/input/audio | EVID-04 | Cannot be replaced by Linux cross-publish or Wine | Run local checkpoint bundle on user PC; capture actual route and physical input/audio; collect exact package/resource identities and logs |
| Native Linux route/input/audio | EVID-04 | Synthetic/headless fixtures have narrower scope | Run equivalent identified package/matrix on Linux with actual input, audible checks and captures |
| Local playback review | D-05–D-08 | Browser codecs, audible output and media interaction | Load real paired recordings, inspect optional overlays, event seeks/drift and source audio switching without retiming |
Unavailable/failed executions are recorded as open dependencies. Completing tools or preparing a bundle does not close EVID-02/EVID-04 or mark Phase 1 accepted. Source-backed later work may continue only under the existing roadmap exception.
## Validation Sign-Off
- [ ] Final plans bind every task to automated checks or explicit human/native evidence
- [ ] Sampling continuity has no three consecutive tasks without automated verification
- [ ] New test files exist and execute nonzero test counts
- [ ] No watch-mode flags
- [ ] Quick feedback latency measured below 30 seconds
- [ ] Native/original capture limitations recorded
- [ ] nyquist_compliant updated only after the implemented validation contract is audited
**Approval:** Pending execution; this draft describes validation, not results.