feat(01-06): preserve full Linux obligation matrix and capture history

This commit is contained in:
UnixNight committed 2026-11-13 12:02:33 -05:00
1 parent e17cfbebe3
commit 0172916ffd
4 files changed
+299 -10

No files matched your search

+92
View File
@@ -83,3 +83,95 @@ and real audible device/monitor source, verify moving unobstructed content and
audio, then record physical normal controls and full tails. Inspect capture cost,
crop/aspect and actual browser playback/seeking separately. Preserve this failed
preflight as history; new attempts use new session IDs and fresh output roots.
## Full matrix and the final combined agenda
The current typed dependency receipt is
`linux-01-06-matrix-preflight-linux`, at
`artifacts/fidelity/linux/01-06-matrix-preflight/receipt.json`, imported under
`artifacts/fidelity/linux-current-receipts/`. The earlier task-1 receipt is
retained diagnostic history; select the current typed receipt for scenario audit.
`artifacts/fidelity/linux/01-06-matrix.json` enumerates every obligation, exact
procedures/events, failed attempt ID and shared display/audio dependency.
There were no runnable native cells after that common dependency failed. The
collector does not manufacture 40 failed game launches or unrelated scene clips.
In this table **OPEN** means the exact current dependency receipt above plus
the named cell/obligation's actual recording, lifetime evidence and human review.
| Cell | continuous | dialogue-return | bank-left | bank-right | landing-idle |
|---|---|---|---|---|---|
| fresh-natural-edge | OPEN | OPEN | OPEN | OPEN | OPEN |
| fresh-natural-ordinary | OPEN | OPEN | OPEN | OPEN | OPEN |
| fresh-skipped-edge | OPEN | OPEN | OPEN | OPEN | OPEN |
| fresh-skipped-ordinary | OPEN | OPEN | OPEN | OPEN | OPEN |
| retained-natural-edge | OPEN | OPEN | OPEN | OPEN | OPEN |
| retained-natural-ordinary | OPEN | OPEN | OPEN | OPEN | OPEN |
| retained-skipped-edge | OPEN | OPEN | OPEN | OPEN | OPEN |
| retained-skipped-ordinary | OPEN | OPEN | OPEN | OPEN | OPEN |
Follow `scenarios.json` literally. Continuous recording begins before startup or
retained setup and keeps lab → VR04 → lab → outside in one process/clock and raw
video. Use normal supported controls/gadget/RETURN, without warps/grants. Focused
clips have their own declared setup and never substitute for the continuous run.
Natural dialogue reaches its real end; skipped dialogue records the actual skip
key and observed skip/end boundaries. Ordinary play releases movement at return;
edge play records held input through dialogue and RETURN, release/first-neutral,
independent left/right banked jumps, landing and idle. Preserve legitimate bank,
TSK-retained story input order and distinct fresh direct-scene observations.
No blanket angle/pixel/timing/audio tolerance or leveling repair is implied.
For each fresh observation, use a new launch/session ID and separately identified
generated execution copy; verify its existing seal rather than rebuilding it.
Keep the source package immutable. The collector records the selected package
root and sealed pre/post identities; copying/staging is an explicit local step,
not an invented copy claim. The prepared checkpoint's immutable Linux package is
also available for a verified local copy. Never select the original installation.
For retained observations, keep one actual process, copy and clock across visible
prerequisite lab departure to VR04 and return, then the tested interval. Capture
the real generated-copy state while that process remains alive; a later launch
or synthetic snapshot cannot replace the prerequisite. Use a separate terminal
to save the observed state records inside the run directory before finalization.
Annotations may declare `artifact_roles` mapping contained recording paths to
`prerequisite`, `identity_files` mapping `copy_pre`, `retained_copy`, `copy_post`
to actual contained state-record paths, and `continuity` with `lifetime_id`,
`prerequisite_artifact`, `copy_pre_sha256`, and for retained runs
`departure_session_id`, `return_session_id`, `retained_copy_sha256`.
All hashes refer to actual identity records. The selected prerequisite recording
must have a probed duration, and prerequisite events precede the tested interval.
A byte-identical alias of raw media can identify its prerequisite role without
retiming; preserve its exact bytes. The collector never invents these observations.
`clocks.clock_id` identifies the outer capture clock; preserve the passive observer
clock's separate origin and annotate any alignment/uncertainty explicitly.
Export/update the checklist with an unused output filename:
```sh
bash scripts/collect-fidelity-linux.sh matrix --receipt artifacts/fidelity/linux-current-receipts/linux-01-06-matrix-preflight-linux/receipt.json --dependency-run artifacts/fidelity/linux/01-06-matrix-preflight --output artifacts/fidelity/linux/new-matrix.json
```
For interrupted work, pass `--previous /local/prior-matrix.json` and repeat
`--receipt` for new attempts. Old receipts are re-read and hash-checked; reused
or conflicting session/receipt IDs are rejected. Preserve failed/incomplete
history, retry only missing/incomplete obligations, and use fresh IDs/directories.
Do not hand-edit history to imply completed cells. Shared scenario validation
checks chronological required markers, one recording and retained identity;
the final acceptance audit still owns review and applicability selection.
The single final session adds these Linux actions to the Windows/original agenda:
1. Verify all 539 authoritative originals before staging and after collection with
`python3 tools/recon/inventory.py ../Original --verify docs/original-manifest.json`.
2. Select real display/audio and identified installed recorder/probe; perform a
short moving-content, audible-output, crop/aspect and physical-control check.
3. Record all 40 obligations with their real setup/lifetime; retain raw video/audio,
accepted gate/observer logs, manual input descriptions, clocks and uncertainty.
4. Import current applicable receipts and export local paired review. Verify actual
browser playback/seeking/tails at normal speed and exclusive original/port audio;
retain any preview's exact derivation and raw source. Review capture overhead.
5. Recheck source/package/original identities and matrix history, retain remaining
discrepancies/open dependencies, and transfer results locally together once.
The Linux subset is **0/40 accepted**. The full three-side protocol remains
**0/120 accepted**: original-DxWnd/native Windows/physical audible review are open.
View File
File mode changed.
+61
View File
@@ -216,6 +216,67 @@ class CaptureTests(unittest.TestCase):
self.assertEqual(attempted['state'], 'open')
self.assertTrue(any('scenario' in x.lower() for x in attempted['dependencies']))
def test_changed_package_after_launch_keeps_prelaunch_identity_and_failure(self):
self.execute()
before = receipts.identify_file(self.run / 'identity/GamePort')['sha256']
(self.package / 'GamePort').write_bytes(b'changed after launch')
receipt = self.api().collect(self.run, probe=self.probe)
self.assertEqual(receipt['identity']['executable']['sha256'], before)
self.assertEqual(receipt['status'], 'failed')
self.assertTrue(any('Post-capture' in reason for reason in receipt['missing_evidence']))
def test_retained_annotations_bind_real_files_and_clock_without_acceptance(self):
self.execute(case_id='retained-natural-ordinary', scope='landing-idle')
(self.run / 'prerequisite.mkv').write_bytes((self.run / 'raw.mkv').read_bytes())
for name in ('copy-pre.json', 'retained-copy.json'):
(self.run / name).write_text(json.dumps({'fixture': name}))
note = self.root / 'annotations.json'
note.write_text(json.dumps({'events': [
dict(name=name, artifact=artifact, media_seconds=i * .0001,
elapsed_seconds=i * .0001, method='manual', uncertainty_seconds=.01)
for i, (name, artifact) in enumerate([
('prerequisite_departure', 'prerequisite.mkv'),
('prerequisite_return', 'prerequisite.mkv'),
('landing', 'raw.mkv'), ('idle', 'raw.mkv')])],
'artifact_roles': {'prerequisite.mkv': 'prerequisite'},
'identity_files': {'copy_pre': 'copy-pre.json', 'retained_copy': 'retained-copy.json'},
'continuity': {'lifetime_id': 'test-launch', 'departure_session_id': 'test-launch',
'return_session_id': 'test-launch', 'prerequisite_artifact': 'prerequisite.mkv',
'copy_pre_sha256': receipts.identify_file(self.run / 'copy-pre.json')['sha256'],
'retained_copy_sha256': receipts.identify_file(self.run / 'retained-copy.json')['sha256']}}))
receipt = self.api().collect(self.run, probe=self.probe, annotations=note)
protocol = receipts.read_json(ROOT / 'docs/evidence/scenarios.json')
cell = next(c for c in protocol['cells'] if c['id'] == receipt['case_id'])
receipts._scenario_observation(protocol, cell, receipt)
receipt['continuity']['return_session_id'] = 'other-process'
with self.assertRaises(ValueError): receipts._scenario_observation(protocol, cell, receipt)
self.assertEqual(receipt['acceptance']['state'], 'open')
def test_annotated_prerequisite_cannot_escape_capture_root(self):
self.execute()
note = self.root / 'annotations.json'
note.write_text(json.dumps({'artifact_roles': {'../secret.mkv': 'prerequisite'}}))
with self.assertRaises(ValueError): self.api().collect(self.run, probe=self.probe, annotations=note)
def test_native_automation_remains_fixture_even_after_collection(self):
self.settings['input_method'] = 'automated'
self.execute()
state = receipts.read_json(self.run / 'session.json')
state['tooling_only'] = False
(self.run / 'session.json').write_bytes(receipts.canonical_bytes(state))
receipt = self.api().collect(self.run, probe=self.probe)
self.assertEqual(receipt['evidence_kind'], 'port_fixture')
self.assertFalse(receipts.validate_receipt(receipt, self.run)['accepted'])
def test_oversized_raw_media_is_preserved_without_unbounded_import(self):
self.execute()
with (self.run / 'raw.mkv').open('r+b') as stream:
stream.truncate(self.api().MAX_MEDIA + 1)
result = self.api().collect(self.run, probe=self.probe)
self.assertEqual(result['status'], 'failed')
self.assertFalse(any(a['path'] == 'raw.mkv' for a in result['artifacts']))
self.assertTrue(any('8 GiB' in reason for reason in result['missing_evidence']))
if __name__ == '__main__':
unittest.main()
+146 -10
View File
@@ -7,6 +7,7 @@ X11/Pulse are the implemented external recorder sources; unsupported sources
produce precise failed receipts rather than dummy fallback recordings.
"""
import argparse
import hashlib
import json
import math
import os
@@ -22,7 +23,8 @@ import time
from fidelity_checkpoint import _package
from fidelity_receipts import (MAX_JSON_BYTES, _id, _write_json, canonical_bytes,
contained_path, identify_file, import_receipt, read_json, separated_roots,
utc_now, validate_receipt, validate_scenario_contract)
utc_now, validate_receipt, validate_scenario_contract, required_events,
_scenario_observation, read_receipt)
PROJECT = Path(__file__).resolve().parents[2]
MAX_SECONDS = 3600
@@ -176,7 +178,8 @@ def run_capture(package, output, session_id, case_id, scope, settings, seconds=1
'input_method': settings['input_method'], 'settings': settings,
'invocation': {'argv': game, 'cwd': str(package)}, 'recorder_argv': recorder,
'clocks': {'monotonic_start': start, 'monotonic_end': start,
'wall_start': wall_start, 'wall_end': wall_start},
'wall_start': wall_start, 'wall_end': wall_start,
'clock_id': session_id + '-monotonic'},
'status': 'failed' if not report['ready'] else 'incomplete',
'missing': list(report['blockers']), 'processes': {}}
_write_json(output / 'started.json', state)
@@ -239,7 +242,9 @@ def collect(run, probe=command_probe, annotations=None):
missing.append('Post-capture package identity failed: ' + str(error))
media = run / 'raw.mkv'
duration = None
if media.exists() and media.stat().st_size:
if media.exists() and media.stat().st_size > MAX_MEDIA:
missing.append('Raw media exceeds 8 GiB capture bound; preserved locally but excluded from import')
elif media.exists() and media.stat().st_size:
media_probe = probe(['ffprobe', '-v', 'error', '-show_streams', '-show_format',
'-of', 'json', str(media)], timeout=20)
_write_json(run / 'media-probe.json', media_probe)
@@ -268,16 +273,47 @@ def collect(run, probe=command_probe, annotations=None):
identity[key] = {'path': 'identity/' + name, **identify_file(identity_dir / name)}
identity['resources'] = {'path': 'identity/resources.json', **identify_file(identity_dir / 'resources.json')}
identity['launcher'] = {'path': 'identity/collector.py', **identify_file(identity_dir / 'collector.py')}
note = read_json(annotations) if annotations else {}
# Explicit local annotations bind actual files; no imported commands execute.
extra_roles = note.get('artifact_roles', {})
if not isinstance(extra_roles, dict) or any(role != 'prerequisite' for role in extra_roles.values()):
raise ValueError('Only explicit prerequisite artifact roles can supplement capture roles')
for name in extra_roles: identify_file(contained_path(run, name))
for key, name in note.get('identity_files', {}).items():
if key not in ('copy_pre', 'retained_copy', 'copy_post'):
raise ValueError('Unsupported annotated copy identity')
identity[key] = {'path': name, **identify_file(contained_path(run, name))}
artifacts = []
for path in sorted(run.rglob('*')):
count = 0
for path in run.rglob('*'):
count += 1
if count > 10000: raise ValueError('Capture exceeds 10000-entry bound')
if path.is_symlink(): raise ValueError('Symlink in capture output')
if not path.is_file() or path.is_relative_to(identity_dir): continue
bound = MAX_MEDIA if path.suffix == '.mkv' else MAX_LOG
if path.stat().st_size > bound:
missing.append('Artifact exceeds import bound: ' + path.relative_to(run).as_posix())
continue
name = path.relative_to(run).as_posix()
role = 'video' if name == 'raw.mkv' else 'input_events' if name.startswith('observer/') else 'diagnostic'
role = extra_roles.get(name, role)
item = {'path': name, 'role': role, **identify_file(path)}
if state['tooling_only']: item['synthetic'] = True
if name == 'raw.mkv' and duration is not None: item['duration_seconds'] = duration
if role == 'prerequisite' and duration is not None:
if item['sha256'] == identify_file(media)['sha256']:
item['duration_seconds'] = duration
else:
inspection = probe(['ffprobe', '-v', 'error', '-show_format', '-of', 'json', str(path)], timeout=20)
try:
seconds = float(json.loads(inspection['stdout'])['format']['duration'])
if inspection['exit_code'] != 0 or inspection['error'] or not math.isfinite(seconds) or seconds <= 0:
raise ValueError('Invalid prerequisite media duration')
item['duration_seconds'] = seconds
except (ValueError, KeyError, TypeError) as error:
raise ValueError('Cannot identify prerequisite recording duration') from error
artifacts.append(item)
artifacts.sort(key=lambda item: item['path'])
# The same hashed MKV contains both streams; audio role uses a distinct
# byte-identical local alias to satisfy the shared validator's unique paths.
if duration is not None:
@@ -285,16 +321,13 @@ def collect(run, probe=command_probe, annotations=None):
artifacts.append({'path': 'raw-audio.mkv', 'role': 'audio',
'duration_seconds': duration, **identify_file(run / 'raw-audio.mkv'),
**({'synthetic': True} if state['tooling_only'] else {})})
events = []
if annotations:
note = read_json(annotations)
events = note.get('events', [])
# Annotations carry observed markers, never review/acceptance promotion.
kind = 'synthetic' if state['tooling_only'] else 'native_port' if state['input_method'] == 'physical' else 'port_fixture'
status = state['status']
if status == 'captured' and missing: status = 'failed'
receipt = {'schema_version': 1, 'receipt_id': state['session_id'] + '-linux',
'session_id': state['session_id'], 'case_id': state['case_id'], 'scope': state['scope'],
'obligation_id': state['scope'],
'evidence_kind': kind, 'historical': False, 'status': status,
'platform': {'os': 'linux', 'execution': 'native' if kind == 'native_port' else kind},
'invocation': state['invocation'], 'clocks': state['clocks'], 'identity': identity,
@@ -306,6 +339,7 @@ def collect(run, probe=command_probe, annotations=None):
'ordinary_input': False, 'human_reviewed': False},
'capture_methods': state['settings'], 'clock_id': state['clock_id'],
'launch_id': state['launch_id'],
'continuity': note.get('continuity', {}),
'acceptance': {'state': 'open', 'remaining': [
'Actual moving-content/crop/clock/physical-input/audibility review',
'Observed scenario markers and lifetime/route continuity validation',
@@ -315,6 +349,84 @@ def collect(run, probe=command_probe, annotations=None):
return receipt
def matrix(protocol, records=(), previous=None, dependency=None):
"""Forty Linux obligations with validated append-only attempt history.
Previous attempts are re-read and hash-checked, never trusted as acceptance.
Distinct failed/incomplete retries require distinct process/session IDs.
A shared environment failure is a dependency, not forty fabricated runs.
"""
validate_scenario_contract(protocol)
cells = {c['id']: c for c in protocol['cells']}
slots = {(c['id'], o): [] for c in protocol['cells'] for o in c['obligations']}
history = []
if previous:
if previous.get('schema_version') != 1 or previous.get('side') != 'port-linux-native':
raise ValueError('Unrecognized previous Linux matrix')
for cell in previous['cells']:
for obligation in cell['obligations']:
for attempt in obligation['attempts']:
path = Path(attempt['receipt_path'])
if identify_file(path)['sha256'] != attempt['receipt_sha256']:
raise ValueError('Historical receipt bytes changed')
history.append((read_receipt(path), path.parent))
seen, sessions = {}, {}
for receipt, root in [*history, *records]:
root = Path(root).resolve(strict=True)
validated = validate_receipt(receipt, root)
key = (receipt['case_id'], receipt.get('obligation_id', receipt['scope']))
if key not in slots: raise ValueError('Unknown Linux matrix slot')
if receipt['platform']['os'] != 'linux': raise ValueError('Linux matrix requires Linux observations')
digest = hashlib.sha256(canonical_bytes(receipt)).hexdigest()
record_id, session_id = receipt['receipt_id'], receipt['session_id']
if record_id in seen:
if seen[record_id] != digest: raise ValueError('Conflicting immutable receipt ID')
continue
if session_id in sessions:
raise ValueError('A new matrix attempt must use a new launch/session ID')
sessions[session_id] = record_id; seen[record_id] = digest
reasons = list(validated['missing_evidence'])
try:
_scenario_observation(protocol, cells[key[0]], receipt)
except (ValueError, KeyError, StopIteration) as error:
reasons.append('Scenario continuity: ' + str(error))
if receipt['evidence_kind'] != 'native_port':
reasons.append('Linux native gameplay requires actual native evidence')
path = root / 'receipt.json'
if read_receipt(path) != receipt: raise ValueError('Matrix input differs from saved receipt')
slots[key].append({'receipt_id': record_id, 'session_id': session_id,
'receipt_path': str(path), 'receipt_sha256': identify_file(path)['sha256'],
'status': receipt['status'], 'evidence_kind': receipt['evidence_kind'],
'accepted': validated['accepted'] and not reasons, 'remaining': list(dict.fromkeys(reasons))})
result_cells, accepted = [], 0
for cell in protocol['cells']:
result = {k: cell[k] for k in ('id', 'lifetime', 'dialogue', 'input')}
result['procedures'] = {
'lifetime': protocol['lifetime_procedures'][cell['lifetime']],
'dialogue': protocol['dialogue_procedures'][cell['dialogue']],
'input': protocol['input_procedures'][cell['input']]}
result['obligations'] = []
for obligation_id in cell['obligations']:
attempts = slots[(cell['id'], obligation_id)]
done = any(a['accepted'] for a in attempts)
accepted += int(done)
dependencies = [] if done else list((dependency or {}).get('reasons', []))
if not done:
dependencies += ['Actual Linux ' + cell['id'] + '/' + obligation_id + ' recording and physical/audible review']
dependencies += [reason for a in attempts for reason in a['remaining']]
result['obligations'].append({'id': obligation_id, 'state': 'accepted' if done else 'open',
'attempts': attempts, 'dependencies': list(dict.fromkeys(dependencies)),
'dependency_receipt_id': (dependency or {}).get('receipt_id'),
'required_events': required_events(protocol, cell, obligation_id),
'route': protocol['obligations'][obligation_id].get('route'),
'procedure': protocol['obligations'][obligation_id]})
result_cells.append(result)
return {'schema_version': 1, 'side': 'port-linux-native', 'required': 40,
'accepted': accepted, 'complete': accepted == 40, 'cells': result_cells,
'environment_dependency': dependency,
'acceptance': {'state': 'open', 'remaining': ['Final combined original/native acceptance audit']}}
def main():
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest='action', required=True)
@@ -332,17 +444,41 @@ def main():
command.add_argument('--run', type=Path, required=True)
command.add_argument('--annotations', type=Path)
command.add_argument('--import-to', type=Path)
command = sub.add_parser('matrix')
command.add_argument('--receipt', type=Path, action='append', default=[])
command.add_argument('--previous', type=Path)
command.add_argument('--dependency-run', type=Path)
command.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
if args.action == 'preflight':
result = preflight(args.package, read_json(args.settings))
elif args.action == 'run':
result = run_capture(args.package, args.output, args.session_id, args.case_id,
args.scope, read_json(args.settings), args.seconds)
else:
elif args.action == 'collect':
result = collect(args.run, annotations=args.annotations)
if args.import_to: import_receipt(args.run / 'receipt.json', args.run, args.import_to)
else:
dependency = None
if args.dependency_run:
path = args.dependency_run / 'receipt.json'
receipt = read_receipt(path)
validate_receipt(receipt, path.parent)
pre = read_json(args.dependency_run / 'preflight.json')
if pre['ready'] or not pre['blockers'] or receipt['status'] != 'failed':
raise ValueError('Environment dependency requires an actual failed-preflight receipt')
dependency = {'receipt_id': receipt['receipt_id'], 'receipt_path': str(path.resolve()),
'receipt_sha256': identify_file(path)['sha256'], 'reasons': pre['blockers']}
result = matrix(read_json(PROJECT / 'docs/evidence/scenarios.json'),
[(read_receipt(p), p.parent) for p in args.receipt],
read_json(args.previous) if args.previous else None, dependency)
separated_roots(PROJECT.parent / 'Original', args.output)
if any(p.is_symlink() for p in (args.output.absolute(), *args.output.absolute().parents)):
raise ValueError('Matrix output cannot use symlink aliases')
args.output.parent.mkdir(parents=True, exist_ok=True)
_write_json(args.output, result)
print(canonical_bytes(result).decode(), end='')
return 0 if result.get('ready', result.get('status') == 'captured') else 1
return 0 if args.action == 'matrix' or result.get('ready', result.get('status') == 'captured') else 1
if __name__ == '__main__':