Record original inventory and recover all GAM level entities
This commit is contained in:
commit
d62e2ab4de
16 files changed
+3954
No files matched your search
+12
@@ -0,0 +1,12 @@
|
|||||||
|
bin/
|
||||||
|
obj/
|
||||||
|
dist/
|
||||||
|
artifacts/
|
||||||
|
.tools/
|
||||||
|
.nuget/
|
||||||
|
.dotnet-home/
|
||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
*.user
|
||||||
|
*.log
|
||||||
|
original/
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
# Neutron GamePort
|
||||||
|
|
||||||
|
Follow `../AGENTS.md` and `../prompt.MD`. This is the local Git repository.
|
||||||
|
Do not configure a remote, push, upload originals or use cloud CI.
|
||||||
|
Originals are in `../Original/` and must remain byte-identical; verify using
|
||||||
|
`python3 tools/recon/inventory.py ../Original --verify docs/original-manifest.json`.
|
||||||
|
|
||||||
|
Use .NET 10, Silk.NET.Vulkan Vulkan 1.3+, SDL3 and shared Linux/Windows x64 code.
|
||||||
|
Keep recovered data/algorithms distinct from unverified approximations. Require
|
||||||
|
runtime evidence before claiming working gameplay or platform compatibility.
|
||||||
|
|
||||||
|
## Working map
|
||||||
|
- `src/GamePort/`: runtime, platform, rendering, assets and game systems.
|
||||||
|
- `tools/`: Python analysis/extraction tools (stdlib where practical).
|
||||||
|
- `tests/`: original-data regressions and bounded parser tests.
|
||||||
|
- `scripts/`: local dependency, build, test and packaging entry points.
|
||||||
|
- `docs/`: durable evidence and continuation notes.
|
||||||
|
- `.tools/`, `.nuget/`, `artifacts/`, `dist/`: ignored local generated data.
|
||||||
|
|
||||||
|
Build from this directory with `dotnet build`, `dotnet run`; project-local SDK
|
||||||
|
may require `scripts/dotnet.sh` (consult README for verified invocation).
|
||||||
|
Python tests: `python3 -m unittest discover -s tests -v`.
|
||||||
|
Local builds: `scripts/build-linux.sh`, `scripts/build-windows.sh`;
|
||||||
|
tests: `scripts/test.sh`; packages: `scripts/package.sh`.
|
||||||
|
|
||||||
|
## Coordination
|
||||||
|
Native subagents supported, max three investigators alongside primary. Explicit
|
||||||
|
exclusive file ownership; primary reviews, integrates, tests and commits milestones.
|
||||||
|
Binary agent: `tools/binary/`, binary/graphics analysis. Asset agent: extractors,
|
||||||
|
asset/format docs, extractor tests. Infrastructure agent: runtime/build scripts and
|
||||||
|
infrastructure report. Primary: GAM loader, gameplay/recon/docs and integration.
|
||||||
|
Coordinate any ownership transfer before editing shared files. No agent commits
|
||||||
|
other agents' active changes. Keep updates concise, save detailed evidence to disk.
|
||||||
|
|
||||||
|
Use bounded endian-aware parsers, portable paths, clear C#, deterministic tests,
|
||||||
|
and explicit unmanaged ownership. Build frequently. Update `docs/progress.md`,
|
||||||
|
`docs/tasks.md`, `docs/decisions.md`, and compatibility evidence before transitions.
|
||||||
|
Current priority is exact asset recovery and one reconstructed original scene;
|
||||||
|
read those documents for actual completion status and next actions.
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
# Gameplay evidence
|
||||||
|
|
||||||
|
## Verified data reconstruction
|
||||||
|
|
||||||
|
`tools/levels/gam.py` parses all 35 supplied GAM files and accounts for every byte;
|
||||||
|
re-encoding each parsed file reproduces its original bytes. Corpus: 3,299 entities,
|
||||||
|
93 class FourCCs, 84,044 properties (22,505 string, 26,965 float, 31,275 integer,
|
||||||
|
3,295 opaque type 2, four opaque type 4). Entity type names are inferred from
|
||||||
|
matching object tags/properties, not universally proven class implementations.
|
||||||
|
|
||||||
|
All integer metadata and float32 values in GAM are **big endian** despite the
|
||||||
|
little endian x86 executable. World positions are large unscaled coordinates;
|
||||||
|
the Y coordinate is consistent with height, pending binary confirmation.
|
||||||
|
|
||||||
|
## Initial scene
|
||||||
|
|
||||||
|
`Original/NewGame.tsk` begins with `LV1B` and float32 20.0. This supports LV1B as
|
||||||
|
the new-game scene; it does not prove all save fields. `gam/level1b.gam` has 129
|
||||||
|
objects. Its player record at file offset `0x8`:
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|---|---|
|
||||||
|
| kind | 3JIM |
|
||||||
|
| ObjectTag | JIM1 |
|
||||||
|
| PositionX | 5071.93310546875 |
|
||||||
|
| PositionY | 26.661895751953125 |
|
||||||
|
| PositionZ | 824.1039428710938 |
|
||||||
|
| RotationY | 155 degrees |
|
||||||
|
| StartPoint | STARTEXP |
|
||||||
|
|
||||||
|
The matching STRT record starts at `0x5b0e`. It agrees on player position/yaw;
|
||||||
|
camera position is (4873.83642578125, 235.64761352539062, 373.21588134765625),
|
||||||
|
camera rotations are zero, music is `musicsecretlab.omt` index 0, and StartTrigger
|
||||||
|
is `LABEXP`. Camera look-at behavior and the introductory trigger require code
|
||||||
|
analysis; do not treat zero stored rotation as proof of a zero effective rotation.
|
||||||
|
|
||||||
|
## Small validation scene
|
||||||
|
|
||||||
|
`gam/VR01.gam`: 16 entities, one trophy, four pickups, two phone booths and two
|
||||||
|
LOAD return triggers. Player at `0x8`: (-2678.0380859375, 87.33218383789062,
|
||||||
|
-2783.401123046875), yaw 290 degrees. STRT at `0x45c` stores camera
|
||||||
|
(-2709.25634765625, 287.3321838378906, -2390.133056640625), music `musicvr01.omt`
|
||||||
|
index 0 and start trigger `bu`. Pickups use radius 100.0 and PointValue 5.
|
||||||
|
|
||||||
|
## Controls documented by original readme
|
||||||
|
|
||||||
|
Source: `Original/readme.txt`, section V.
|
||||||
|
|
||||||
|
| Input | Original action |
|
||||||
|
|---|---|
|
||||||
|
| Up / Down | Forward / backup (rearview in scooter mode) |
|
||||||
|
| Left / Right | Turn left / right |
|
||||||
|
| Enter / Ctrl / left mouse | Primary fire |
|
||||||
|
| Shift / Insert / right mouse | Jump or jet-pack burst |
|
||||||
|
| M / O / H / I | Menu / objectives / help / item inventory |
|
||||||
|
| G | Call Goddard |
|
||||||
|
| Tab / Space | Gadget screen |
|
||||||
|
| Escape | Quit / main menu / skip cutscene |
|
||||||
|
| Mouse movement | Forward, backward, turn |
|
||||||
|
|
||||||
|
## Data is not an algorithm
|
||||||
|
|
||||||
|
LOAD exposes LevelName, StartPoint, RequiredTask, RequiredLevel, ExactLevel,
|
||||||
|
Radius, SoundIndex, FadeType and FadeTime. PIC exposes prerequisite fields,
|
||||||
|
PointValue and pickup index. These are faithfully recoverable inputs; exact
|
||||||
|
trigger conditions, distance metric, timing, task comparison and state updates
|
||||||
|
must be established from executable code or observation before implementation is
|
||||||
|
called faithful. Negative integers commonly act as flags/sentinels; their exact
|
||||||
|
meaning depends on each property.
|
||||||
|
|
||||||
|
Save data has a fixed-looking region followed by Pascal task names and big endian
|
||||||
|
values (NewGame.tsk tail includes SCENE = 30). Save writing is not yet recovered.
|
||||||
|
Physics, jumping, collision, AI, RNG, combat, progression and cutscene execution
|
||||||
|
remain under investigation. No original gameplay session has been captured yet.
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# Architecture
|
||||||
|
|
||||||
|
Use a single .NET 10 C# executable, Silk.NET.Vulkan for Vulkan 1.3 rendering,
|
||||||
|
SDL3 for platform windowing, input and audio. Linux x64 and Windows x64 are
|
||||||
|
equally required. Source is `src/GamePort/`; root executable csproj allows
|
||||||
|
`dotnet build` and `dotnet run` from GamePort. No runtime network dependencies.
|
||||||
|
|
||||||
|
Original assets stay in `../Original`. Python tools parse undocumented original
|
||||||
|
formats into generated scene resources under `artifacts/`. The modern application
|
||||||
|
loads these original-derived resources and big endian GAM entity data. Portable
|
||||||
|
asset resolution must handle Windows-authored case differences. Extraction must
|
||||||
|
retain material references, world coordinates, per-corner UVs and animation data;
|
||||||
|
triangulation/render-state approximations require explicit documentation.
|
||||||
|
|
||||||
|
`Assets/GamLevel.cs` independently decodes original gameplay records and exposes
|
||||||
|
player/startpoint/camera and other typed properties. Exact record bytes/unknown
|
||||||
|
types remain accessible; do not normalize unknown flags into invented semantics.
|
||||||
|
`tools/levels/gam.py` provides a lossless forensic reference and JSON export.
|
||||||
|
|
||||||
|
Renderer design follows original fixed-function needs, without PBR or invented
|
||||||
|
visual effects. A headless offscreen rendering path makes actual Vulkan image
|
||||||
|
verification possible without a desktop; this is distinct from SDL presentation.
|
||||||
|
Simulation implementation must follow recovered algorithms and timing. A generic
|
||||||
|
fly camera or rendered model alone does not satisfy the playable-scene requirement.
|
||||||
|
|
||||||
|
Subsystem implementation and validated ownership are documented in
|
||||||
|
`infrastructure.md`, `analysis/graphics.md` and `rendering.md` as they become
|
||||||
|
available. Persistent docs distinguish planned architecture from verified behavior.
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
# Compatibility and validation
|
||||||
|
|
||||||
|
Status is evidence-based; compilation alone is not runtime verification.
|
||||||
|
|
||||||
|
| Feature | Implemented | Tested | Evidence / limits |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Original preservation | Hash baseline | Initial corpus | 539 files in original-manifest.json |
|
||||||
|
| GAM reader/encoder Python | Yes | Linux, all 35 levels | Byte-for-byte round trip, six tests |
|
||||||
|
| GAM C# loader | Yes | Pending compilation | Original scene/player/camera exposed |
|
||||||
|
| OMT geometry recovery | In progress | Original level1 meshes | Asset agent integration pending |
|
||||||
|
| .NET 10 Linux build | In progress | Pending | Project-local SDK installed |
|
||||||
|
| Windows x64 build/package | Pending | Unverified | Cross-build will not prove execution |
|
||||||
|
| Vulkan frame | In progress | Pending | Headless Mesa available; no initial display |
|
||||||
|
| SDL3 window/input/audio | In progress | Pending | Native dependencies being built locally |
|
||||||
|
| Playable scene / original fidelity | No | No | Rendering foundation does not establish gameplay |
|
||||||
|
| Original execution/reference capture | No | No | Wine absent initially; originals not executed |
|
||||||
|
| Animation/AI/combat/progression/save | No | No | Reverse engineering ongoing |
|
||||||
|
|
||||||
|
Current known limitations: no reconstructed complete gameplay loop yet; no native
|
||||||
|
Windows testing or reference captures. Exact camera, physics, fixed-function
|
||||||
|
lighting/blending/fog and task behavior need binary/observational validation.
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
# Decisions
|
||||||
|
|
||||||
|
1. **Preserve input directory.** Keep Original in place, baseline all file hashes,
|
||||||
|
generate outputs elsewhere. No legacy binary execution in the source directory.
|
||||||
|
2. **Nested local repository.** Workspace `.git` is reserved and not a functioning
|
||||||
|
repository. Initialize GamePort/.git, no remotes, commit code/docs/tools only.
|
||||||
|
3. **Evidence-first data recovery.** Native x86 executable does not imply little
|
||||||
|
endian formats: GAM and OMT use big endian metadata. Require bounded readers,
|
||||||
|
full consumption and real original-file regressions.
|
||||||
|
4. **Initial scene.** NewGame.tsk starts LV1B and player GAM references STARTEXP.
|
||||||
|
Prioritize that scene; use smaller VR01 for data/render verification. The LV1B
|
||||||
|
LABEXP start trigger means simple camera spawning is insufficient for fidelity.
|
||||||
|
5. **Local dependencies.** Missing SDK/native tools installed under `.tools` or
|
||||||
|
`/tmp` after approved official downloads, without assuming root or altering host.
|
||||||
|
6. **Useful parallelism.** Three independent agents own binary/rendering, asset
|
||||||
|
recovery and infrastructure, while primary owns gameplay data and integration.
|
||||||
|
Shared runtime/format changes must be coordinated. No external coordination.
|
||||||
|
7. **Honest compatibility.** Build, headless Vulkan frame, SDL presentation,
|
||||||
|
gameplay fidelity and native Windows execution are separate evidence categories.
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# GAM level entities
|
||||||
|
|
||||||
|
Derived from all 35 original files. Implemented reader and lossless encoder:
|
||||||
|
`tools/levels/gam.py`; corpus regression tests: `tests/test_levels.py`.
|
||||||
|
|
||||||
|
All multibyte integers/floats below use big endian byte order.
|
||||||
|
|
||||||
|
```
|
||||||
|
file: char[4] level_code, u32 entity_count, entity[entity_count]
|
||||||
|
entity: char[4] class_code, u32 property_count, property[property_count], byte[4] trailer
|
||||||
|
property: u8 name_length, char[name_length] name, u32 type, u32 payload_length,
|
||||||
|
[u8 string_length if type == 1], byte[payload_length] payload
|
||||||
|
```
|
||||||
|
|
||||||
|
The trailer is opaque: its purpose is unverified. The string prefix must equal
|
||||||
|
payload_length but is excluded from payload_length. Type 1 is Latin-1-compatible
|
||||||
|
text (no terminator); type 3 is float32; type 6 is signed int32. Type 2 has four
|
||||||
|
opaque bytes in this corpus (RotateToDest) and type 4 remains opaque. Retain raw
|
||||||
|
payload bytes and property order to preserve float bit patterns and unknown types.
|
||||||
|
No compression or padding occurs between the fields. There is no file footer.
|
||||||
|
|
||||||
|
Reader rejects truncation, impossible counts, duplicate properties, scalar length
|
||||||
|
mismatches, string length mismatches, non-finite known scalars and trailing bytes.
|
||||||
|
Original corpus passes these constraints; they are not proof of every conceivable
|
||||||
|
GAM version. Never read a Windows binary using host-endian assumptions.
|
||||||
|
|
||||||
|
Examples: `VR01.gam` has header `56 52 30 31 00 00 00 10`; its first object at 8
|
||||||
|
is `3JIM` with 19 properties. `level1b.gam` starts `LV1B`, 129 objects, with the
|
||||||
|
first `3JIM` at 8. All 3,299 object trailers are retained without reinterpretation.
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,25 @@
|
|||||||
|
# Progress
|
||||||
|
|
||||||
|
## Verified so far
|
||||||
|
- Read complete primary specification, inspected originals, established ownership.
|
||||||
|
- Created local-only Git repository under GamePort and persistent project guidance.
|
||||||
|
- Recorded all 539 original-file SHA-256 hashes (224,601,234 bytes).
|
||||||
|
- Identified PE32 x86 MSVC C++ game and Open Media Toolkit 2.0 DR4 engine.
|
||||||
|
- Fully decoded 35 GAM levels, 3,299 entities, 93 class codes, 84,044 properties.
|
||||||
|
- Python GAM reader/encoder round-trips every original byte; six targeted tests pass.
|
||||||
|
- C# GAM reader added with explicit endian handling and original startpoint/camera.
|
||||||
|
- OMT resource index and meshes recovered by asset investigation; integration ongoing.
|
||||||
|
- Scoped dependency download approved; local .NET 10 SDK/Silk/SDL setup ongoing.
|
||||||
|
|
||||||
|
## Current work
|
||||||
|
Binary/rendering investigation, exact material/texture/audio recovery, C# runtime,
|
||||||
|
Vulkan offscreen verification, SDL3 presentation, cross-platform packaging.
|
||||||
|
No claim of full playable reconstruction, original behavior comparison, or Windows
|
||||||
|
runtime validation. See `tasks.md` and `compatibility.md` for remaining work.
|
||||||
|
|
||||||
|
## Continuation
|
||||||
|
Read AGENTS, this file, tasks, architecture and analysis reports. Check local Git
|
||||||
|
status and active agents before editing. Avoid repeating completed inventory/GAM
|
||||||
|
analysis. Original initial-scene evidence points to LV1B / STARTEXP, with VR01 as
|
||||||
|
a smaller resource-validation scene. Exact movement defaults now recovered in
|
||||||
|
binary investigation; do not substitute unverified gameplay guesses.
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
# Initial reconnaissance
|
||||||
|
|
||||||
|
Original game identified from supplied readme: **Jimmy Neutron: Boy Genius**.
|
||||||
|
Root input directory is `../Original/`, preserved in place. The specification
|
||||||
|
is `../prompt.MD` (uppercase extension on disk), read in full before work.
|
||||||
|
|
||||||
|
539 input files, 224,601,234 bytes. Complete per-file sizes, first bytes and
|
||||||
|
SHA-256 hashes: `original-manifest.json`. Reproduce or verify via
|
||||||
|
`tools/recon/inventory.py`. There are 254 ASE, 128 PNG, 100 OMT, 35 GAM,
|
||||||
|
12 TSK, five EXE, three DLL, one readme TXT and one uninstall ISU.
|
||||||
|
|
||||||
|
## Binaries and API evidence
|
||||||
|
|
||||||
|
`file` identifies Neutron.exe and OMT2.dll as PE32 x86 Windows GUI files, each
|
||||||
|
with four sections. Binary analysis identifies MSVC C++ and Open Media Toolkit
|
||||||
|
2.0 DR4. Imported DirectDrawCreateEx, DirectInput8 and DirectSound plus engine
|
||||||
|
renderer symbols establish legacy graphics/audio/input infrastructure. The
|
||||||
|
readme's requirement for DirectX 8 does not alone establish Direct3D version.
|
||||||
|
See `analysis/binary.md` and `analysis/graphics.md` for addresses and confidence.
|
||||||
|
|
||||||
|
## Asset evidence
|
||||||
|
|
||||||
|
ASE files are readable 3ds Max ASCII export version 200, including frame
|
||||||
|
animation. `ASE/jimrun.ASE` names jimmyrun.max and stores 0..4 frames at 10 fps,
|
||||||
|
480 ticks/frame, with position, normals and UV data. PNG files are unmodified
|
||||||
|
original textures. Proprietary OMT databases begin `0MF2` and have a big endian
|
||||||
|
index offset; the asset investigation has recovered the resource table and
|
||||||
|
original world-space meshes. GAM entities are fully decoded and byte-round-trip
|
||||||
|
verified across all 35 files. TSK saves remain partially understood.
|
||||||
|
|
||||||
|
Details: `analysis/assets.md`, `file-formats.md`, `gam-format.md`,
|
||||||
|
`analysis/gameplay.md`. Generated outputs belong under ignored `artifacts/`.
|
||||||
|
|
||||||
|
## Initial environment
|
||||||
|
|
||||||
|
Available initially: git, Python 3.12, file, objdump, strings, gcc, cmake,
|
||||||
|
curl/wget, Vulkan loader and Mesa ICDs including lavapipe. No dotnet in PATH;
|
||||||
|
no Wine, Ghidra/Rizin/radare2, glslang, spirv tools, SDL3 or Xvfb in PATH.
|
||||||
|
No DISPLAY or WAYLAND_DISPLAY was set. No original binary has been executed.
|
||||||
|
|
||||||
|
Native isolated-context subagents, parallel tool orchestration and background
|
||||||
|
commands are supported. Three meaningful independent tasks were delegated
|
||||||
|
(binary/rendering, assets, infrastructure); primary owns gameplay/integration.
|
||||||
|
Git workspace metadata is reserved read-only, so a separate local repository
|
||||||
|
was initialized under GamePort, without remotes. Local worktrees are possible
|
||||||
|
but exclusive directory ownership is sufficient for these initial tasks.
|
||||||
|
|
||||||
|
Dependency downloads initially fail sandbox DNS. Scoped escalation for official
|
||||||
|
SDK/library downloads succeeded; dependencies are installed project-locally.
|
||||||
|
See infrastructure and compatibility reports for final actual versions/checks.
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# Reverse-engineering index
|
||||||
|
|
||||||
|
Authoritative reference is supplied originals. Findings must be supported by file
|
||||||
|
offsets, virtual addresses, API symbols or observed behavior. Decompiled output is
|
||||||
|
evidence, not original source. Runtime behavior has not yet been observed.
|
||||||
|
|
||||||
|
- `original-manifest.json`: complete original SHA-256 baseline.
|
||||||
|
- `reconnaissance.md`: input/tool inventory, game identity and initial environment.
|
||||||
|
- `analysis/binary.md`: PE/compiler, entry points, imports and function mappings.
|
||||||
|
- `analysis/graphics.md`: renderer/API, geometry, camera and Vulkan equivalents.
|
||||||
|
- `analysis/assets.md`, `file-formats.md`: recovered OMT/ASE/texture/audio evidence.
|
||||||
|
- `gam-format.md`: fully accounted GAM wire format and exact round-trip validation.
|
||||||
|
- `analysis/gameplay.md`: original controls, startpoint/camera, entities and limits.
|
||||||
|
|
||||||
|
## Function map under investigation
|
||||||
|
|
||||||
|
Provisional mappings from binary agent (disassembly must accompany final report):
|
||||||
|
C3DJimmy update 0x424600, input 0x426030; player input 0x438A60, walking branch
|
||||||
|
0x438BC0. Player defaults ctor region 0x4372A0 sets acceleration 30, deceleration
|
||||||
|
35, maximum speed 600; increments occur per simulation invocation, while movement
|
||||||
|
uses elapsed time. A future simulation must recover update cadence before calling
|
||||||
|
these coefficients timestep-independent. Jump/gravity values remain provisional.
|
||||||
|
|
||||||
|
## Unresolved
|
||||||
|
|
||||||
|
Original execution under isolated Wine/Windows; complete gameplay algorithms;
|
||||||
|
archive animations and render states; save schema; exact historical timing and
|
||||||
|
Windows runtime compatibility. Follow `tasks.md` for concrete next actions.
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
# Tasks
|
||||||
|
|
||||||
|
## Completed
|
||||||
|
- [x] Full specification read; original inventory, hashes and local Git established.
|
||||||
|
- [x] Capabilities/tool reconnaissance; independent investigators with file ownership.
|
||||||
|
- [x] Executable format and engine identification.
|
||||||
|
- [x] Decode/round-trip all 35 GAM files; reader corruption and original-data tests.
|
||||||
|
- [x] Implement C# GAM asset reader (compilation/runtime checks pending).
|
||||||
|
|
||||||
|
## Active
|
||||||
|
- [ ] Complete binary/graphics report with addresses, rendering state and camera.
|
||||||
|
- [ ] Recover player timing, movement, gravity, collision and input algorithms.
|
||||||
|
- [ ] Extract OMT meshes, textures, materials, animations, sound and music robustly.
|
||||||
|
- [ ] Verify asset schema and integration into native Vulkan renderer.
|
||||||
|
- [ ] Compile .NET 10 runtime and verify real offscreen rendering/readback.
|
||||||
|
- [ ] Integrate SDL3 window/input/audio; exercise local display where possible.
|
||||||
|
- [ ] Local Linux and Windows self-contained builds and dependency packaging.
|
||||||
|
- [ ] Test original preservation after tools; checkpoint stable work locally.
|
||||||
|
|
||||||
|
## Required after foundation
|
||||||
|
- [ ] Run original in isolated copied-data Wine prefix or local Windows VM.
|
||||||
|
- [ ] Capture original startup, controls, camera, animation, audio and gameplay.
|
||||||
|
- [ ] Reconstruct LV1B opening sequence and one complete playable scene.
|
||||||
|
- [ ] Implement/verify collision, interactions, trigger/task system and progression.
|
||||||
|
- [ ] Implement original animation playback and correct rendering states.
|
||||||
|
- [ ] Implement menus, inventory, gadgets, combat/AI and all remaining scenes.
|
||||||
|
- [ ] Recover save/load behavior, configuration, audio mixing and transitions.
|
||||||
|
- [ ] Compare reconstruction to original references; resolve fidelity differences.
|
||||||
|
- [ ] Run native Windows tests; distinguish cross-publishing from runtime evidence.
|
||||||
|
- [ ] Produce verified local release packages and accurate known limitations.
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
using System.Buffers.Binary;
|
||||||
|
using System.Numerics;
|
||||||
|
using System.Text;
|
||||||
|
|
||||||
|
namespace GamePort.Assets;
|
||||||
|
|
||||||
|
/// <summary>Original, big endian level entities. See docs/gam-format.md.</summary>
|
||||||
|
public sealed record GamProperty(string Name, uint Type, byte[] Raw)
|
||||||
|
{
|
||||||
|
public string Text => Type == 1 ? Encoding.Latin1.GetString(Raw)
|
||||||
|
: throw new InvalidDataException($"{Name} is not a string");
|
||||||
|
public float Float => Type == 3 && Raw.Length == 4
|
||||||
|
? BitConverter.Int32BitsToSingle(BinaryPrimitives.ReadInt32BigEndian(Raw))
|
||||||
|
: throw new InvalidDataException($"{Name} is not a float32");
|
||||||
|
public int Integer => Type == 6 && Raw.Length == 4
|
||||||
|
? BinaryPrimitives.ReadInt32BigEndian(Raw)
|
||||||
|
: throw new InvalidDataException($"{Name} is not an int32");
|
||||||
|
}
|
||||||
|
|
||||||
|
public sealed record GamEntity(string Kind, int Offset,
|
||||||
|
IReadOnlyDictionary<string, GamProperty> Properties, byte[] Trailer)
|
||||||
|
{
|
||||||
|
public string? Text(string name) => Properties.TryGetValue(name, out var p) ? p.Text : null;
|
||||||
|
public float Number(string name, float fallback = 0) =>
|
||||||
|
Properties.TryGetValue(name, out var p) ? p.Float : fallback;
|
||||||
|
public int Integer(string name, int fallback = 0) =>
|
||||||
|
Properties.TryGetValue(name, out var p) ? p.Integer : fallback;
|
||||||
|
public string? Tag => Text("ObjectTag");
|
||||||
|
public Vector3 Position => new(Number("PositionX"), Number("PositionY"), Number("PositionZ"));
|
||||||
|
public Vector3 Rotation => new(Number("RotationX"), Number("RotationY"), Number("RotationZ"));
|
||||||
|
public Vector3 CameraPosition => new(Number("ViewportPx"), Number("ViewportPy"), Number("ViewportPz"));
|
||||||
|
}
|
||||||
|
|
||||||
|
public sealed record GamLevel(string Code, IReadOnlyList<GamEntity> Entities)
|
||||||
|
{
|
||||||
|
public GamEntity? Player => Entities.FirstOrDefault(e => e.Kind == "3JIM");
|
||||||
|
|
||||||
|
public GamEntity? FindStart(string? tag = null)
|
||||||
|
{
|
||||||
|
tag ??= Player?.Text("StartPoint");
|
||||||
|
return Entities.FirstOrDefault(e => e.Kind == "STRT" &&
|
||||||
|
string.Equals(e.Tag, tag, StringComparison.OrdinalIgnoreCase));
|
||||||
|
}
|
||||||
|
|
||||||
|
public static GamLevel Load(string path) => Parse(File.ReadAllBytes(path));
|
||||||
|
|
||||||
|
public static GamLevel Parse(ReadOnlySpan<byte> data)
|
||||||
|
{
|
||||||
|
var reader = new GamReader(data);
|
||||||
|
var code = reader.Text(4);
|
||||||
|
if (code.Any(c => !char.IsAsciiLetterOrDigit(c)))
|
||||||
|
throw new InvalidDataException("Invalid level FourCC");
|
||||||
|
var count = reader.Count(12);
|
||||||
|
var entities = new List<GamEntity>(count);
|
||||||
|
for (var i = 0; i < count; i++)
|
||||||
|
{
|
||||||
|
var offset = reader.Offset;
|
||||||
|
var kind = reader.Text(4);
|
||||||
|
var properties = new Dictionary<string, GamProperty>(StringComparer.Ordinal);
|
||||||
|
var propertyCount = reader.Count(10);
|
||||||
|
for (var j = 0; j < propertyCount; j++)
|
||||||
|
{
|
||||||
|
var name = reader.Pascal();
|
||||||
|
var type = reader.U32();
|
||||||
|
var length = reader.U32();
|
||||||
|
if (length > int.MaxValue)
|
||||||
|
throw new InvalidDataException($"Property too long at {reader.Offset:x}");
|
||||||
|
if (type == 1 && reader.Byte() != length)
|
||||||
|
throw new InvalidDataException($"String size mismatch at {reader.Offset:x}");
|
||||||
|
if (type is 3 or 6 && length != 4)
|
||||||
|
throw new InvalidDataException($"Scalar size mismatch at {reader.Offset:x}");
|
||||||
|
var property = new GamProperty(name, type, reader.Take((int)length).ToArray());
|
||||||
|
if (type == 3 && !float.IsFinite(property.Float))
|
||||||
|
throw new InvalidDataException($"Non-finite scalar at {reader.Offset:x}");
|
||||||
|
if (!properties.TryAdd(name, property))
|
||||||
|
throw new InvalidDataException($"Duplicate property {name}");
|
||||||
|
}
|
||||||
|
entities.Add(new GamEntity(kind, offset, properties, reader.Take(4).ToArray()));
|
||||||
|
}
|
||||||
|
if (reader.Remaining != 0)
|
||||||
|
throw new InvalidDataException($"Unexpected bytes at {reader.Offset:x}");
|
||||||
|
return new GamLevel(code, entities);
|
||||||
|
}
|
||||||
|
|
||||||
|
private ref struct GamReader(ReadOnlySpan<byte> data)
|
||||||
|
{
|
||||||
|
private readonly ReadOnlySpan<byte> data = data;
|
||||||
|
public int Offset { get; private set; }
|
||||||
|
public int Remaining => data.Length - Offset;
|
||||||
|
public ReadOnlySpan<byte> Take(int length)
|
||||||
|
{
|
||||||
|
if (length < 0 || length > Remaining)
|
||||||
|
throw new InvalidDataException($"Truncated GAM at {Offset:x}");
|
||||||
|
var result = data.Slice(Offset, length);
|
||||||
|
Offset += length;
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
public byte Byte() => Take(1)[0];
|
||||||
|
public uint U32() => BinaryPrimitives.ReadUInt32BigEndian(Take(4));
|
||||||
|
public string Text(int length) => Encoding.Latin1.GetString(Take(length));
|
||||||
|
public string Pascal() => Text(Byte());
|
||||||
|
public int Count(int minimumItemBytes)
|
||||||
|
{
|
||||||
|
var count = U32();
|
||||||
|
if (count > Remaining / minimumItemBytes)
|
||||||
|
throw new InvalidDataException($"Count exceeds remaining GAM at {Offset:x}");
|
||||||
|
return (int)count;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import struct
|
||||||
|
import sys
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
sys.path.insert(0, str(ROOT / "tools/levels"))
|
||||||
|
import gam
|
||||||
|
|
||||||
|
|
||||||
|
def fixture(properties=b"", count=0):
|
||||||
|
return b"TST1" + struct.pack(">I", 1) + b"3JIM" + struct.pack(">I", count) + properties + b"\x12\x34\x56\x78"
|
||||||
|
|
||||||
|
|
||||||
|
class GamTests(unittest.TestCase):
|
||||||
|
def test_real_levels_roundtrip_and_counts(self):
|
||||||
|
paths = sorted((ROOT.parent / "Original/gam").glob("*.gam"))
|
||||||
|
self.assertEqual(len(paths), 35, "original corpus is required for this integration test")
|
||||||
|
total = 0
|
||||||
|
for path in paths:
|
||||||
|
with self.subTest(path=path.name):
|
||||||
|
raw = path.read_bytes()
|
||||||
|
level = gam.parse(raw)
|
||||||
|
self.assertEqual(gam.encode(level), raw)
|
||||||
|
total += len(level.entities)
|
||||||
|
self.assertEqual(total, 3299)
|
||||||
|
|
||||||
|
def test_exact_original_start(self):
|
||||||
|
level = gam.parse((ROOT.parent / "Original/gam/level1b.gam").read_bytes())
|
||||||
|
self.assertEqual(level.code, "LV1B")
|
||||||
|
player = level.entities[0].values()
|
||||||
|
self.assertEqual(player["StartPoint"], "STARTEXP")
|
||||||
|
self.assertEqual(player["PositionX"], 5071.93310546875)
|
||||||
|
start = next(e.values() for e in level.entities
|
||||||
|
if e.kind == "STRT" and e.values()["ObjectTag"] == "STARTEXP")
|
||||||
|
self.assertEqual(start["StartTrigger"], "LABEXP")
|
||||||
|
self.assertEqual(start["MusicDatabase"], "musicsecretlab.omt")
|
||||||
|
|
||||||
|
def test_endianness_and_string_prefix(self):
|
||||||
|
props = b"\x01x" + struct.pack(">II f", 3, 4, -12.5)
|
||||||
|
props += b"\x01s" + struct.pack(">II", 1, 3) + b"\x03abc"
|
||||||
|
raw = fixture(props, 2)
|
||||||
|
level = gam.parse(raw)
|
||||||
|
self.assertEqual(level.entities[0].values(), {"x": -12.5, "s": "abc"})
|
||||||
|
self.assertEqual(gam.encode(level), raw)
|
||||||
|
|
||||||
|
def test_unknown_type_preserved(self):
|
||||||
|
raw = fixture(b"\x01x" + struct.pack(">II", 999, 3) + b"abc", 1)
|
||||||
|
self.assertEqual(gam.encode(gam.parse(raw)), raw)
|
||||||
|
|
||||||
|
def test_all_truncations_rejected(self):
|
||||||
|
raw = fixture(b"\x01x" + struct.pack(">II f", 3, 4, 12.5), 1)
|
||||||
|
for end in range(len(raw)):
|
||||||
|
with self.subTest(end=end), self.assertRaises(gam.FormatError):
|
||||||
|
gam.parse(raw[:end])
|
||||||
|
|
||||||
|
def test_overflows_trailing_data_and_bad_strings(self):
|
||||||
|
for data in [b"TST1\xff\xff\xff\xff", fixture() + b"x",
|
||||||
|
fixture(b"\x01x" + struct.pack(">II", 1, 2) + b"\x03abc", 1),
|
||||||
|
fixture(b"\x01x" + struct.pack(">II", 3, 3) + b"abc", 1)]:
|
||||||
|
with self.subTest(data=data), self.assertRaises(gam.FormatError):
|
||||||
|
gam.parse(data)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Lossless, bounded reader for the supplied Neutron GAM entity files.
|
||||||
|
|
||||||
|
No original file is modified. Unknown property types stay opaque. The four bytes
|
||||||
|
after each object are retained without assigning an unverified interpretation.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import struct
|
||||||
|
from dataclasses import asdict, dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
class FormatError(ValueError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class Reader:
|
||||||
|
def __init__(self, data: bytes):
|
||||||
|
self.data = data
|
||||||
|
self.offset = 0
|
||||||
|
|
||||||
|
def take(self, count: int) -> bytes:
|
||||||
|
if count < 0 or count > len(self.data) - self.offset:
|
||||||
|
raise FormatError(f"truncated data at 0x{self.offset:x}: need {count} bytes")
|
||||||
|
result = self.data[self.offset:self.offset + count]
|
||||||
|
self.offset += count
|
||||||
|
return result
|
||||||
|
|
||||||
|
def u32(self) -> int:
|
||||||
|
return struct.unpack(">I", self.take(4))[0]
|
||||||
|
|
||||||
|
def text(self, count: int) -> str:
|
||||||
|
return self.take(count).decode("latin-1")
|
||||||
|
|
||||||
|
def pascal(self) -> str:
|
||||||
|
return self.text(self.take(1)[0])
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Property:
|
||||||
|
name: str
|
||||||
|
type: int
|
||||||
|
length: int
|
||||||
|
value: str | int | float
|
||||||
|
raw: str
|
||||||
|
offset: int
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Entity:
|
||||||
|
kind: str
|
||||||
|
offset: int
|
||||||
|
properties: list[Property]
|
||||||
|
trailer: str
|
||||||
|
|
||||||
|
def values(self) -> dict:
|
||||||
|
return {p.name: p.value for p in self.properties}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Level:
|
||||||
|
code: str
|
||||||
|
entities: list[Entity]
|
||||||
|
size: int
|
||||||
|
|
||||||
|
|
||||||
|
def parse(data: bytes) -> Level:
|
||||||
|
r = Reader(data)
|
||||||
|
code = r.text(4)
|
||||||
|
if not code.isascii() or not code.isalnum():
|
||||||
|
raise FormatError("invalid level FourCC")
|
||||||
|
count = r.u32()
|
||||||
|
if count > (len(data) - 8) // 12:
|
||||||
|
raise FormatError("object count exceeds file capacity")
|
||||||
|
entities = []
|
||||||
|
for _ in range(count):
|
||||||
|
offset = r.offset
|
||||||
|
kind, property_count = r.text(4), r.u32()
|
||||||
|
if property_count > (len(data) - r.offset) // 10:
|
||||||
|
raise FormatError(f"property count exceeds capacity at 0x{offset:x}")
|
||||||
|
properties = []
|
||||||
|
names = set()
|
||||||
|
for _ in range(property_count):
|
||||||
|
property_offset = r.offset
|
||||||
|
name, type_id, length = r.pascal(), r.u32(), r.u32()
|
||||||
|
if name in names:
|
||||||
|
raise FormatError(f"duplicate property {name!r} at 0x{property_offset:x}")
|
||||||
|
names.add(name)
|
||||||
|
if type_id == 1:
|
||||||
|
actual = r.take(1)[0]
|
||||||
|
if actual != length:
|
||||||
|
raise FormatError(f"string size mismatch at 0x{property_offset:x}")
|
||||||
|
raw = r.take(length)
|
||||||
|
if type_id == 1:
|
||||||
|
value = raw.decode("latin-1")
|
||||||
|
elif type_id in (3, 6):
|
||||||
|
if length != 4:
|
||||||
|
raise FormatError(f"scalar size {length} at 0x{property_offset:x}")
|
||||||
|
value = struct.unpack(">f" if type_id == 3 else ">i", raw)[0]
|
||||||
|
if isinstance(value, float) and not math.isfinite(value):
|
||||||
|
raise FormatError(f"non-finite scalar at 0x{property_offset:x}")
|
||||||
|
else:
|
||||||
|
value = raw.hex()
|
||||||
|
properties.append(Property(name, type_id, length, value, raw.hex(), property_offset))
|
||||||
|
entities.append(Entity(kind, offset, properties, r.take(4).hex()))
|
||||||
|
if r.offset != len(data):
|
||||||
|
raise FormatError(f"{len(data) - r.offset} unexpected trailing bytes at 0x{r.offset:x}")
|
||||||
|
return Level(code, entities, len(data))
|
||||||
|
|
||||||
|
|
||||||
|
def encode(level: Level) -> bytes:
|
||||||
|
"""Re-encode parsed data to prove every byte has been accounted for."""
|
||||||
|
out = bytearray(level.code.encode("latin-1") + struct.pack(">I", len(level.entities)))
|
||||||
|
for e in level.entities:
|
||||||
|
out += e.kind.encode("latin-1") + struct.pack(">I", len(e.properties))
|
||||||
|
for p in e.properties:
|
||||||
|
name = p.name.encode("latin-1")
|
||||||
|
raw = bytes.fromhex(p.raw)
|
||||||
|
out += bytes([len(name)]) + name + struct.pack(">II", p.type, p.length)
|
||||||
|
if p.type == 1:
|
||||||
|
out += bytes([p.length])
|
||||||
|
out += raw
|
||||||
|
out += bytes.fromhex(e.trailer)
|
||||||
|
return bytes(out)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
cli = argparse.ArgumentParser(description=__doc__)
|
||||||
|
cli.add_argument("input", type=Path)
|
||||||
|
cli.add_argument("--output", type=Path)
|
||||||
|
args = cli.parse_args()
|
||||||
|
raw = args.input.read_bytes()
|
||||||
|
level = parse(raw)
|
||||||
|
if encode(level) != raw:
|
||||||
|
raise FormatError("round trip differs")
|
||||||
|
text = json.dumps(asdict(level), indent=2, ensure_ascii=True, allow_nan=False) + "\n"
|
||||||
|
if args.output:
|
||||||
|
if args.output.resolve() == args.input.resolve():
|
||||||
|
cli.error("output must differ from original input")
|
||||||
|
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
args.output.write_text(text)
|
||||||
|
else:
|
||||||
|
print(text, end="")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Inventory or verify originals with SHA-256; never writes to the source tree."""
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def inventory(root):
|
||||||
|
return [{"path": p.relative_to(root).as_posix(), "size": p.stat().st_size,
|
||||||
|
"sha256": hashlib.sha256(p.read_bytes()).hexdigest(),
|
||||||
|
"header": p.read_bytes()[:16].hex()}
|
||||||
|
for p in sorted(root.rglob("*")) if p.is_file()]
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("root", type=Path)
|
||||||
|
parser.add_argument("--verify", type=Path)
|
||||||
|
args = parser.parse_args()
|
||||||
|
files = inventory(args.root)
|
||||||
|
if args.verify:
|
||||||
|
previous = json.loads(args.verify.read_text())["files"]
|
||||||
|
if files != previous:
|
||||||
|
raise SystemExit("Original inventory differs from recorded baseline")
|
||||||
|
print(f"Verified {len(files)} unchanged original files")
|
||||||
|
else:
|
||||||
|
print(json.dumps({"root": str(args.root), "files": files}, indent=2))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Reference in new issue
Block a user