Files
Neutron/docs/checkpoint/camera-audio-ui.md
T

279 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Camera, audio and UI architecture checkpoint
Scope: playable baseline `69d0187`, lab → VR04 → lab → outside. Read `prompt.MD`
all1281 lines and both AGENTS files. This checkpoint changes no runtime behavior.
Original files were read only. Source-byte checks below are static evidence,
not original-game runtime comparison. Existing Linux GPU/SDL smoke tests and user
Windows reports describe the port, not a recovered original reference recording.
## Evidence identity and reproduction
| Module | SHA-256 |
| --- | --- |
| Neutron.exe | f5281076e1a2492f2d61f87aae3f4544b6d0f535ebe33f9fbb6f78710fe1aea8 |
| OMT2.dll | 3ec41ead011adf56387a6dbd4580e5b5a9721b5f8020c8a56420f42824354943 |
`python3 tools/binary/checkpoint_camera_audio_ui.py ../Original` verifies25
selected instruction sequences and7 constants against those identities and prints
mapped physical offsets. `python3 -m unittest discover -s tests -p
'test_checkpoint_camera_audio_ui.py' -v` also checks ray-adjustment algebra.
Fresh bounded objdump reads corroborate the cached `artifacts/binary/neutron.asm`;
RTTI was checked with `tools/binary/rtti_map.py`.
All EXE code/constant VAs in this report have physical offsets VA−0x400000.
Examples and decisive ranges are listed explicitly. **The globals below occupy
PE zero-fill and have no physical file offset**; subtracting the image base from
their VA does not produce an original-file byte location.
| Runtime owner/global | VA | Meaning supported by consumers |
| --- | --- | --- |
| Controlled game-interface pointer | 0x5099E4 | Input ownership; null also suppresses objective clock/draw |
| Shared viewport pointer | 0x509A50 | Position+44/+48/+4C; signed angle words+50/+52/+54 |
| Current music database | 0x696004 | Separate from current Wave/effects/speech bank |
| Current Wave database | 0x696008 | CAM/MCA replace it with authored speech bank; DEFAULT restores effects |
| Sound engine | 0x69600C | Channel array+40; virtual24 returns channel count |
| Default positional-effect gate | 0x4F8434 | Wrapper4589C0 tests it; cutscenes clear, DEFAULT enables |
| Objective time / selector | 0x4F8188 / 0x4F8194 | Separate transient presentation and persistent selection |
| Objective44 flags | 0x4F7D00 | Independent per-row visibility/selection state |
| Modal active / type | 0x4F8198 / 0x4F8048 | UI continues handling modal while main gameplay is paused |
## Shared camera state and ownership
**Recovered fact.** There is a shared viewport, not independent original
"gameplay eye" and "cutscene eye" objects selected by renderer priority.
CAM/MCA and player callbacks write that same position/orientation. MCA helper
431670 (file31670) writes eye+44/+48/+4C at431677/431683/431694, derives angles
using imported angle routine48D108 at4316F9, writes words+50/+52 and roll0 at
43170D..431717, and mirrors camera globals4F8418..4F842C. CAM formulas are at
4161CD..416585 (file161CD..16585), MCA formulas431027..4311C1
(file31027..311C1). Their authored transforms are already documented in
`../analysis/lab-progression.md` and represented by SceneDirector.UpdateCamera.
CAM activation4165E2..4166FA (file165E2..166FA) resolves PlayerControlled:
`none` preserves, `NULL` clears, a named found actor grants control through game
virtual118. Before clear, it calls controlled actor virtual410/438 and rewrites
transform state. MCA431502..4315A7 (file31502..315A7) similarly prepares prior
controlled actor then clears through game virtual118(NULL). Its PlayerControlled
field is completion ownership at4311F0..431228, not an activation grant.
Control loss does not prove all physics/actors should stop. Main-loop report owns
the enclosing update pass and any per-actor active gates.
**New recovered DEFAULT handoff.** DEFAULT447CF0 has a Jimmy-already-controlled
early branch447D5C..447D9C (file47D5C..47D9C). Even that branch reloads the
default effects bank; it skips camera reposition and music restart. Otherwise
447E1E..447E27 restores control, then447E2D..447EC3
(file47E2D..47EC3) computes actor-local point `(0,500,-500)` using game-interface
virtual384, compares distance from current viewport position against float1000
at48E5AC (file8E5AC), and writes the point to viewport only when distance is
**strictly greater than1000**. At equality the original `fcomp; test ah,41; jne`
skips the snap. Non-Jimmy mode-dependent restore targets also exist447DAC..447E1E;
the port's unconditional Jimmy restore is only the route subset.
**Current implementation.** Program selects LabSimulation.CameraEye then replaces
it with SceneDirector.CameraEye while a shot exists. `RestoreControl` clears the
shot eye; it neither transfers current viewport state into follow state nor
performs the source conditional snap. LabSimulation only advances when Jimmy
controls input. Calling its eye "always advanced" is inaccurate; it remains a
separate stale state during ordinary uncontrolled cutscenes. This is an ownership
mismatch, not a reason to add another transition easing heuristic.
## Follow camera, obstruction and clipping
**Recovered fact.** Jimmy complete-object vtable4A3BDC (fileA3BDC), slot124 →
438BC0 walking, slot128 →439900 airborne, slot16C at4A3D48 (fileA3D48) →
43B820 camera ray callback, slot170 →437740 default camera setup. The separate
game-interface vtable4A373C (fileA373C) has complete-object displacement+C0;
do not apply complete-object offsets directly to an interface pointer.
| Complete-player field | Source initialization/use |
| --- | --- |
| +7C8/+7CC/+7D0 | Eye local0,200,−350 at437740..43776D (file37740..3776D) |
| +7D8/+7DC/+7E0 | Look local0,80,0 at437752..437774 |
| +854/+858/+85C | Axis gains1.6,1,1.6 at43775E..437787 |
| +804..+810 | Cached desired eye when special-state branch retains it439658..4396A5 |
| +714/+718/+71C | Prior camera position used by follow deltas43971E..439748 |
| +828, +830, +84C, +851 | Additional state affects offsets/gains; complete semantic names unresolved |
4395CF..4398E4 (file395CF..398E4) does not simply follow a fixed local offset.
Normal eye height is `eyeY−2*self[84C]`; target height is
`lookY+1.9*self[84C]` at439625..4396DA. Special+828/+830 branches use cached
eye or flip/scale backward distance by3.7 after threshold1.7. Float64 constants
are4AFCE8/4AFCF0/495338 (filesAFCE8/AFCF0/95338). Do not rename84C to ground
height or infer its units without tracing writers.
**New ray contract.** Caller439702..439714 passes look-point by value and
desired-eye pointer through complete-object virtual16C. Callee43B820..43B90C
(file3B820..3B90C) calls ray predicate47C210 at43B874; on hit reads16-byte hit
global5CFC60 and mutates desired eye to
`lookPoint−0.75*(lookPoint−hitPoint)`, i.e. `.25*look+.75*hit`. It returns1.5
on hit (float4A3138,fileA3138), otherwise1.0. Predicate layer/filter/ordering and
its use of actor-query virtual310 still require recovery; this algebra alone
does not authorize replacing its ray with the port capsule sweep.
Normal interpolation4397B1..4397F1 multiplies X/Z delta by axis gain, ray return
gain and dt; Y uses `(gainY+0.2)*rayGain*dt`. +851 selects a0.2 branch and
+828 selects another branch without these axis multipliers. Position is added
to current viewport4397F8..43982D, then look angles derive from prior-position
fields and wrap with mask3FFF at439865..4398E4. This order matters and needs an
isolated trace before replacing the source adapter.
**Current approximation.** LabSimulation.DesiredEye uses root+(yaw-rotated
0,180,−350), target root+99, fixed gains1.6/1.2/1.6 and modern sphere/capsule
obstruction before and after interpolation. 180/99 match only an inferred
selected offset state, not the complete field-driven algorithm above.
**Clipping fact versus policy.** OMT2 layer constructor1004D340
(DLL file4D340) defaults near20/far3200/FOV0xAAA. Source initializers override
geometry-layer far values: LV1B44F5A5/44F5B1 (files4F5A5/4F5B1)=10000,
VR04456C04/456C4A (files56C04/56C4A)=50000, outside45086F
(file5086F) sets layer509A3C to28000 while509A30 retains3200. Full35-scene
table is `../analysis/camera-clipping.md`. SceneProjection's maximum of two
layers is a documented merged-renderer policy; layer masks/clip/fog and dynamic
LV4B clip transition are incomplete. Orbit scene-radius far distance only hid
the earlier wrong gameplay3200 range. Widescreen aspect is a modern policy.
## Audio owns cutscene completion, banks and channel lifetime
RTTI disambiguates the camera callback structures: C3DCutSceneCamera complete
vtable498060 (file98060), slotD8 at498138 →4165A0 activation; its **+C8**
game-interface vtable497BFC (file97BFC), slot3C4 at497FC0 →415F90 update.
Thus update-relative handle+8B4/active+8B8/elapsed+8F0 are complete-object
+97C/+980/+9B8 respectively. C3DMultiCutSceneCamera complete vtable4A93C8
(fileA93C8), slotD8 at4A94A0 →431280; its+C8 game-interface vtable4A8F64
(fileA8F64), slot3C4 at4A9328 →430DA0. MCA update-relative handle+DC0,
active+DC4, elapsed+DE0 are complete+E88/+E8C/+EA8. Camera's interface
displacement is C8, whereas Jimmy's is C0; conflating them shifts every field.
**Recovered fact.** CAM415F90 calls base update463090, increments elapsed+8F0
at415FB2..415FBC, then polls sound handle+8B4 at415FC2..415FD3
(file15FC2..15FD3) with47D890. While sound reports active it updates camera and
skip handling. When inactive it stops/releases handle, changes target animation,
deactivates and invokes next trigger4160A4..4160E2. MCA uses the same predicate
at430DD2 before advancing shot; per-shot elapsed reset43190F. Animation is an
effect of activation/completion, not the recovered cutscene clock owner.
47D890..47D8E6 (file7D890..7D8E6) validates engine, nonnegative index and channel
count, reads voice at `engine[40][handle]`, calls voice virtual14 with two outputs,
and reports active if the second output is nonzero **or voice byte+30 is nonzero**.
Missing engine/index/voice returns false. Follow-up [voice-clock.md](voice-clock.md)
resolves both derived vtables: slot14 is inherited software elapsed/duration,
and+30 is repeat/loop state. This is not a hardware-playing or audibility query.
The ordinary one-shot clock is duration-based, but source integer boundaries,
invalid handles, engine clock pause, play failure and stop still need modeling.
47D7A0..47D849 (file7D7A0..7D849) stops voice virtual0C, optionally releases its
resource from selected database via virtual38, clears handle resource table
695EC0[handle], and clears global music handle4F6B30 when equal. Parameter0
still stops/clears handle association; parameter1 additionally releases database
resource. These are source runtime handles, not original Wave identity numbers.
| Lifecycle | Original transition / offsets | Port boundary |
| --- | --- | --- |
| Jimmy PostLoad |423642..423656 (file23642..23656) loads SoundEffects.omt into696008; selected music stored complete+A4C, handle+A38 | Per-scene prepared files; selectedMusic tuple held by SceneAudio |
| CAM takes audio |416776..4167E1 stops Jimmy music handle with47D7A0(handle,0), unloads selected archive, clears696004 | MusicStopped event disposes stream; tuple retained for restart |
| CAM takes Wave bank |4167FD..416830 (file167FD..16830) unloads default bank, clears696008/gate4F8434, loads authored speech bank | Speech opens explicit independent WAV; SFX default gate absent |
| MCA takes Wave bank |4315AD..4315F5 (file315AD..315F5) performs analogous bank/gate replacement; does not establish universal music stop | Standalone MCA intentionally does not mute all music |
| DEFAULT resumes |447F12..447F45 (file47F12..47F45) reloads selected archive, starts Wave0 and stores new music handle | Restart selected track from start; independent streams |
| DEFAULT restores effects |447F4B..447F6F (file47F4B..47F6F) restores volume/bank/gate; early Jimmy-owned branch447D60..447D76 also restores bank/gate | No explicit bank switch; always SoundEffects.omt Effect(id) |
| Level teardown/reset | Full global sound reset is not proven by these callbacks alone | SceneAudio.Dispose destroys every per-scene stream; RenderSession survives |
458980→47D390 is nonpositional wrapper;4589C0→47D640 checks default gate4F8434.
Therefore "mute every effect during every speech" would be a false generalization.
Recover wrapper call sites before suppressing objective216 or mechanism effects.
Original volume, attenuation, spatial pan, voice limits/allocation and stream
pause ownership remain incomplete. Source general-bank switching is broader than
the narrow fixed default bank in `../analysis/sound-effects.md`.
**Current implementation.** SceneDirector consumes prepared PCM sample duration,
including leftover dt across shots; no SDL voice status participates. SceneAudio
uses independent SDL streams for speech/music/one-shots/owner loops. SdlAudio.Pump
queues loop data below one source-buffer length; bytes queued are not proof all
converted output reached the speaker. Debug SetPaused is a modern menu feature.
The lab16-cue SDL dummy smoke validates port stream bookkeeping only. Physical
speaker output and native Windows end timing remain separate verification.
## UI update, modal timing and layout
**Recovered fact.** UI dispatcher404080 (file4080) calls base473240 first at
40408E, normalizes pointer using original1/640 and1/480 at4040B7/4040C5, processes
cursor repeat state when mode flag4F8181 or modal4F8198 is active, then invokes
UI virtual4E4/4DC at40420B..40421B. Notification timers4F8154/58/5C are separate
from objective timer. After virtual4E0 at4043D3, controlled-pointer condition
4043D9 advances objective time and invokes virtual4D0; null-control branch
40447A..4044BD hides art without advancing. Modal branch4044C2 follows this,
so its forced objective time8.7 overrides any earlier increment that UI tick.
This is a recovered local ordering, not proof of frame-wide input/UI scheduling.
Objective407070 (file7070) has .7 entry, stationary until7, exit until8.7; O
edge toggles timer0/7, not layout. Positive Select406F50 resets timer even when
selection repeats; initialization4018A8 and423EC6 sets20 hidden. Exact timings,
sound216 latch, fields, art identities and byte checks are in
`../analysis/objective-timing.md`. UI row flags are44 independently selected
entries; single selector digit and visible flag set are not interchangeable.
Practice406F90 (file6F90) guards existing modal, shows chosen art and cursor,
then407022..407031 invokes game virtual168(1) and stores type. UI continues to
accept selection while gameplay is paused.4044CE excludes type12 from forced
objective hide. No404D1E..404D97 hides modal, restores main game and resets
objective time0. Yes type0 stages VR04/PHONEBOOTH and departure context;
source mode1/2/3 map VR03/02/01. Exact keyboard/pointer strict bounds are in
`../analysis/ui.md`; controller navigation/highlight/cursor ownership remain open.
**Current implementation.** ObjectivePanel correctly separates timer/selector
and controlled/modal gates for the implemented objective subset. Program.Key
adds Jimmy-owned/inventory/focus/inspection/debug gates; original local clock
checks any nonnull controlled entity. Modern focus/orbit/debug gates are not
recovered source rules. PracticePrompt implements only modes0..3 and no source
highlight/controller state. SceneHud shows only lab objectives1..3, pad/digit and
practice art; gadget inventories and layout callbacks406590/406650 are absent.
Original402CC0/402D70 (files2CC0/2D70) scale horizontal/vertical positions with
width/640 and height/480 separately. Current UiViewport deliberately uses uniform
minimum scale with centered modal/bottom-right anchors and inverse DPI-aware
pointer mapping, following the user's widescreen requirement. This is an
intentional modern policy, not a claim that the original had anchored ultrawide
layout. F1/F2 debug UI is likewise modern; keep its state out of source save/UI
fields. The objective pad extending off reference viewport is authored clipping.
## Route interaction and reset map
| Route point | Camera/audio/UI dependencies and ownership |
| --- | --- |
| Lab NewGame/STARTEXP | Loaded selector does not imply visible pad: timer20. STRT/intro activates CAM/MCA chain, takes shared viewport/control; CAM stops music and replaces effects bank. First CAM bank gate persists through chained MCA until DEFAULT. |
| Intro DEFAULT | Restores actor control, conditionally snaps existing viewport only for distance>1000, restarts music if control was not already Jimmy, restores effects bank/gate. UI objective clock resumes with nonnull control; task selector changes remain separate. |
| Jetpack pickup/practice | Objective changes reset timer; practice modal pauses main game and forces objective hidden. No unpauses/reopens timer0. Yes stages VR04/PHONEBOOTH plus return context; script-state report owns persistent departure root/strings. |
| VR04 | Initial geometry layer far50000; scene-specific camera/speech/music state rebuilt by port. Original full teardown/global reset needs loader recovery; do not infer all fields persist because TSK does. |
| RETURN lab | Source return applies STRT/root/one-time trigger suppression; intro should not replay solely because scene rebuilt. Viewport/music/UI reset ordering depends on PostLoad and retained task state, not generic "resume previous audio". Current port constructs fresh SceneAudio, ObjectivePanel and director. |
| Carl → outside | CAM/MCA null control suppresses input/objective time, not all actor advancement. DEFAULT/LOAD order owns camera/music restoration vs scene destruction. Outside geometry far28000 is layer-specific, while current combined renderer uses28000 globally. |
## Coverage, assumptions and next bounded chunks
| Area | Recovered | Remaining source gap / evidence gate | Current implementation gap |
| --- | --- | --- | --- |
| Script cameras | Local transforms, elapsed, shared viewport writes, control handoff | Vtable caller phase/target transforms after actor motion; full skip/loader ordering | Separate eye state and duration clock |
| Follow | Fields, branches, ray mutation/return gain, viewport/angle writes |47C210 layer masks/nearest-hit semantics;84C writers; cached camera updates | Fixed offsets/gains and capsule clipping |
| DEFAULT | Strict>1000 snap, existing viewport retention, effect/music reset | Mode-dependent control targets outside route | Unconditional Jimmy restore and discarded scripted eye |
| Projection | Source layer defaults/all35 initial limits | Assignment/masks/fog, dynamic transitions | Merged maximum and orbit policy |
| Audio | Handle polling, software duration/repeat, bank/gate ownership, stop/reload | Source duration conversion/allocation/play failures, engine pause | Per-file independent streams, floating-duration/leftover-dt completion |
| UI | Local dispatch order/objective/modal gates and resource coords | Mode4F8181/4F8430 semantics, full flags/gadget/highlights/controller | Lab subset, modern input gating and uniform anchors |
1. C5: trace47C210 filter/caller contract; build shared viewport handoff tests
using source snap999.99/1000/1000.01 and ray-hit weighting. Preserve viewport
at control return, apply source conditional snap, then follow. Do not transplant
gains while leaving its previous-position/angle/collision dependencies guessed.
2. C6: voice-clock follow-up resolves virtual14/+30; recover allocation/failure
and source duration conversion before modeling the channel clock. Model
explicit bank/gate state separately from independent stream transport.
Test invalid/missing handle, delayed start, natural end, skip, paused playback,
DEFAULT already controlled, and level disposal before switching CAM/MCA clock.
3. C8: recover UI mode/pause callback boundaries and44 flags; preserve existing
tested timer rules. Add source gadget/layout/controller state only with call
evidence. Uniform widescreen/debug policy remains intentional and independently
tested. Recover animation end notifications separately from speech end.
These support the primary `coverage.md` and `implementation-plan.md`; they do not
claim all systems faithful or require every level/original GPU capture before a
bounded implementation correction. Stale broad binary.md mode1-no-action and
universal3200 clipping claims are superseded by movement/clipping reports. Broad
"camera recovered"/"audio implemented" labels must retain the boundaries above.