Recover OMT resources, morph topology, and original engine evidence

This commit is contained in:
UnixNight committed 2026-10-23 17:32:48 -04:00
1 parent d62e2ab4de
commit 3df20ac19a
12 files changed
+1648

No files matched your search

+141
View File
@@ -0,0 +1,141 @@
# Asset recovery evidence
Verified against local originals on 2026-10-23. No original files were modified.
`Original/` is outside the modern local Git repository. Analysis/extraction uses
Python standard library only; nothing is uploaded.
## Verified corpus
`python3 tools/extractors/audit.py ../Original --report artifacts/extracted/audit.json`
successfully parsed all 100 `.omt` databases, 7,027 resource entries, with no
conversion failures:
| Resource | Count | Verification |
|---|---:|---|
| `3DSh` | 2,155 | Every shape v2 consumed exactly; positions finite; corner indices bounded |
| `3DMa` | 2,183 | Every material v1 consumed exactly; original texture links recovered |
| `Canv` | 1,614 | All RLE decoded to exact row lengths and all resource bytes consumed |
| `Wave` | 875 | RIFF/WAVE header and bounded RIFF lengths present |
| `mCTy` | 100 | Indexed metadata preserved; internal interpretation outstanding |
| `0HDR` | 100 | Indexed allocation metadata preserved; not runtime geometry |
Texture counts are 866 RGB555, 736 32-bit RGB, and 12 indexed 8-bit.
All 875 sound resources contain intact RIFF data: 874 PCM and one format-tag 85
(MP3 in WAVE). PCM rates/channels/sample widths vary; do not assume one audio
format. Loop and cue behavior in the surrounding OMT metadata remains unverified.
## Recovered scene outputs
Run from `GamePort/`:
```sh
python3 tools/extractors/prepare.py
python3 -m unittest discover -s tests -p test_extractors.py -v
```
Preparation produces ignored local files below `artifacts/extracted/`:
- `level1b/scene.json`: 8,607 expanded vertices, 2,869 triangles, 33 textures.
- `VR01/scene.json`: 240 expanded vertices, 80 triangles, 7 textures.
- Texture `.rgba` files plus PNG previews, preserving material ID links.
- `sounds/`, `musicjimmyshouse/`, `musicneighborhood/`: exact embedded WAV bytes.
- `animations/{jimstop,jimrun,jimjump}/`: original morph samples and manifests.
- `animations/jimycarl.rgba`: original normal Jimmy atlas, 256 x 256 indexed PNG decoded to RGBA.
Individual archive recovery:
```sh
python3 tools/extractors/omt.py ../Original/omt/level1b.omt --scene --output artifacts/extracted/level1b
python3 tools/extractors/omt.py ../Original/omt/sounds.omt --output artifacts/extracted/sounds --raw
python3 tools/extractors/ase.py ../Original/ASE/jimrun.ASE --base ../Original/ASE/jimstop.ase --output artifacts/extracted/animations/jimrun
```
The static OMT scene is geometry/material recovery, not a complete level state.
GAM entity transforms, object visibility, character meshes, lighting, camera,
gameplay collision masks, triggers and task progression must be applied separately.
All source scene polygons examined in these first scene exports are triangles;
the exporter supports polygon fans and warns for any larger polygons whose
concavity has not been validated.
## Executable evidence used
Preferred virtual addresses in the supplied `OMT2.dll` (image base `0x10000000`):
| Routine | Address | Supporting behavior |
|---|---|---|
| `OMedia3DShape::read_class` | `0x10007b60` | `3DSP` signature, version, vectors and polygon arrays |
| `OMedia3DPolygon::read_class` | `0x10003e20` | Version-dependent corners, UVs, normal indices, material links |
| `OMedia3DMaterial::read_class` | `0x10001490` | Compact v1 styles, color triplets and texture stream link |
| `OMediaCanvas::read_class` | `0x10025b80` | Dispatches OmCv through canvas converter |
| `OMediaOMTConverter::convert` | `0x10027df0` | OmGW dimensions, packing/key/palette header and row RLE |
| RLE unpack helper | `0x1006c230` | Byte/word/dword PackBits variants; LE signed controls |
| `OMediaOMTConverter::create_canvas` | `0x10028620` | Pixel RGB and alpha/color-key conversion |
| `OMediaOMTConverter::read_palette` | `0x10027b40` | OmPa/OPa2 palette with 4,096-byte accelerator |
| `OMediaDBObjectStreamLink::read_class` | `0x10030260` | BE16 present + fourcc + BE32 ID |
| Bool stream operator | `0x10073a80` | Booleans occupy two bytes on disk |
| `OMedia3DMorphAnim::render_draw_shape` | `0x10002df0` | Retains base shape while overriding per-index vectors |
RGB555 channels are expanded by `<< 3`, **not** normalized to 255. Instructions
at `0x10028815` onward verify this; maximum channels therefore remain 248.
The 32-bit path at `0x100288ab` selects RGB bits 16/8/0 and derives alpha from
color-key equality. Stored high-byte alpha is not copied by that converter.
Raw byte-exact resources remain available using `--raw`.
## ASE geometry and animation
The supplied ASE assets include standard ASCII mesh vertex/face tables, per-corner
texture indices, scene timing and vertex morph samples. `jimstop.ase` differs in
filename case from most other ASE files. Resolve filenames case-insensitively.
Two matching assets provide independent coordinate evidence, covered by tests:
- All eight `gate1.ASE` vertices and all eight `yokianpart.ASE` vertices match the
corresponding `objects.omt` resources after `(x,y,z) -> (x,z,y)`.
- Face index order remains identical; the reflection is not accompanied by an
index swap in the stored original.
- Every corner UV matches `(u,v) -> (u,-v)`.
- No additional `NODE_TM` is applied to these already exported vertex positions.
Source ASE normal fields are not assumed authoritative; exported preview normals
currently use flat triangle normals. Original smoothing must still be replicated.
Jimmy uses 426 source vertices and 814 faces. `jimstop.ase` has two samples,
`jimrun.ASE` five, and `jimjump.ASE` three, all with file scene speed 10 FPS and
480 ticks/frame. These establish asset sample timing, not state-specific playback
speed or loop policy in gameplay code.
The EXE stores `jimyshock.png` at file offset `0xef0c8`, followed by `jimstop.ase`
at `0xef0d8`. **Primary integration corrected an early inference from string
proximity:** InitObject at `0x422C96..0x422CB0` binds `jimycarl.png` (VA
`0x4EEAC4`) to texture slot 0 and `jimyshock.png` to slot 1. The latter is the
shock/skeleton effect, not normal appearance. Prepared animation textures and
runtime defaults now use `jimycarl.png`; both have been visually inspected.
The ASE bitmap path `D:\\Jimmy\\jimmy\\jimmylast.bmp` is not a runtime dependency.
Run/jump filenames follow in the same initialization string cluster.
### Morph topology retention
Run/jump/scratch ASE files omit UV tables. Zero UVs would show a single texel and
are rejected by the animation exporter. The original morph renderer validates
the sample vector count against the base shape and interpolates per-index vectors
(`0x10002ed3`, `0x10002eee`, `0x10002f9a..0x10002fd9`), then submits the base
shape plus override vectors (`0x10003396..0x100033ab`). Its topology/UV remain
on the base shape. Therefore the prepared Jimmy animations explicitly use the
textured `jimstop.ase` base and original sample vertex indices. This also matters
because 23 face records in run/jump differ from the later dated stop asset even
though all sample vertex counts remain 426. `morphBase` provenance in each
animation manifest records the source and hash of the retained shape.
## Limits and next work
- Static asset decoding is verified, but visual fidelity needs comparison with an
isolated original game run.
- Legacy material booleans and polygon flags remain numerically preserved rather
than assigned unproven lighting/culling/collision meanings.
- Color triplet roles are conventional labels inferred from the legacy material
layout; exact renderer use should be cross-checked against recovered DX states.
- Animation exporter supports all supplied mesh records, but character bundles
prepared so far cover Jimmy stop/run/jump only. Original smooth normal and
state-driven playback behavior remain incomplete.
- Native runtime renderer/audio integration and package validation belong to the
primary/infrastructure work; successful extraction is not runtime verification.
+181
View File
@@ -0,0 +1,181 @@
# Executable investigation
Static analysis, 2026-10-23. No original code was executed or modified. Addresses
below are preferred-base virtual addresses of **Neutron.exe** or **OMT2.dll**;
patch variants must be checked independently. No Ghidra/Rizin/Wine was available
on PATH initially; the initial investigation uses Python, strings and objdump.
## Verified identity and toolchain
| File | Format | Preferred base | Entry | Header timestamp |
|---|---|---|---|---|
| Neutron.exe | PE32 GUI, native Intel 80386 | 0x00400000 | 0x004802F2 | 2001-09-29 21:07:40 UTC |
| OMT2.dll | PE32 DLL, native Intel 80386 | 0x10000000 | 0x10105753 | 2001-08-27 10:27:41 UTC |
Both have linker 6.0, MSVC decorated C++ exports/imports, C++ RTTI, x87 code and
MSVC exception handling. Together these strongly identify Microsoft Visual C++ 6
era C++; the linker header alone does not prove the exact compiler build. CLR
directories are zero. The normal `.text`, `.rdata`, `.data` sections, extensive
readable symbols and direct instructions support an unpacked build. No packer
was detected; absence is a static inference, not an exhaustive packer test.
The runtime is **Open Media Toolkit 2.0 DR4**. DLL assertion paths literally name
`C:\Open Media Toolkit 2.0 DR4\Sources\OMTClasses\...`, including graphics,
database, compression and DirectX implementations. `OMT2Old.dll` is byte-identical
to `OMT2.dll` (SHA-256
`3ec41ead011adf56387a6dbd4580e5b5a9721b5f8020c8a56420f42824354943`).
Neutron SHA-256:
`f5281076e1a2492f2d61f87aae3f4544b6d0f535ebe33f9fbb6f78710fe1aea8`.
Five EXE variants have distinct hashes; filenames imply software/HD/no-CD behavior
but no behavior should be assigned solely from filenames.
## Dependencies and system boundaries
Neutron imports the OMT DLL and Win32 USER32/KERNEL32/ADVAPI32; consult the
generated PE inventory for full lists. The DLL imports DirectDrawCreateEx and
DirectDrawEnumerateA from DDRAW, DirectInput8Create from DINPUT8, DSOUND ordinal1,
WINMM PlaySoundA, USER32, GDI32, COMDLG32, KERNEL32. Direct3D is obtained through
COM rather than a D3D8 import; see [graphics.md](graphics.md).
The engine has named exports for database objects, shape and morph animation,
collision caches, sphere/triangle picking, light/material state, world updates,
sound channels, event manager and periodical dispatch. These exports are useful
evidence and stable starting points, not complete recovered game logic.
## Initialization and loop map
| Module/address | Identified behavior | Evidence |
|---|---|---|
| EXE 0x4802F2 | CRT entry | PE header, MSVC CRT setup |
| EXE 0x4789B0 | WinMain wrapper | CRT calls it with four WinMain args at 0x4803BB; stores OMT start info, sets current directory, calls 0x46A4A0 |
| EXE 0x46A4A0 | Application bootstrap | Calls window existence/dialog code, creates CAweReal and CGfx objects |
| DLL 0x100218A0 | OMediaApplication::start | Repeatedly invokes OMediaPeriodical::devote_time until application byte +0x18 is true |
| DLL 0x10060720 | OMediaPeriodical::devote_time | Named export; dispatch list must be analyzed further |
| DLL 0x100215C0 | OMediaAnimPeriodical::spend_time | Sends messages -604 for logic, -605 for animation/render dispatch |
| DLL 0x100F9B10 | OMediaWorld::update_logic | Uses timer at world+0x48, pause/max/min-period controls, dispatches update_elements with float elapsed |
| DLL 0x100F9C80 | OMediaWorld::update_elements | Named export, recursively dispatches element logic |
| EXE 0x47E470 | Reset game performance timer | QueryPerformanceCounter into 0x696018 |
| EXE 0x47E490 | Elapsed time in seconds | (counter-lastCounter)/frequency, updates lastCounter when nonzero |
| EXE 0x47E4F0 | Game frame callback | Calls timer via virtual slot 0xD4, stores seconds at 0x696030; only processes `0 < dt < 1` |
QueryPerformanceFrequency IAT is EXE 0x48D0BC, frequency global 0x696020.
QueryPerformanceCounter IAT is 0x48D0B8. Engine GetTickCount exists separately.
The frame callback forwards seconds to 0x479BC0, 0x462A20, 0x4693A0, 0x47D8F0
and 0x477DB0. Their exact roles require further call graph analysis. Timing is
mixed: time-based motion plus per-call speed increments, so a proposed fixed
simulation rate is a compatibility choice until the original cadence is observed.
## Player class and recovered movement evidence
MSVC RTTI identifies C3DJimmy, derived C3DPlayer/C3DAnimated/C3DObject and OMT
classes. C3DJimmy type descriptor: **0x4EEFF8**; main game interface vtable:
**0x4A373C**, object displacement **0xC0**; OMT element vtable at **0x4A3BDC**.
The unusual multiple inheritance means offsets below explicitly distinguish
complete object pointers from game-interface pointers.
| Address | Behavior |
|---|---|
| 0x422AC0 | C3DJimmy::InitObject (embedded exact debug label) |
| 0x422D30 | Jimmy object event/message handler; identified from vtable but exact name unverified |
| 0x424600 | Jimmy per-frame update, game-interface virtual slot 0x3C4 |
| 0x426030 | Jimmy input/movement handler, game-interface slot 0x3CC; calls base 0x438A60 |
| 0x438A60 | Player movement-mode dispatcher; mode0 invokes walking, mode1 no action, mode2 flying, mode3 seated |
| 0x438BC0 | Walking handler; receives complete object pointer, `float dt` |
| 0x437C40 | Applies player speed/turn to world velocity; receives complete object pointer |
| 0x437740 | Sets default follow-camera offsets/scales (complete object pointer) |
| 0x472980 | Rotates local point by object Euler rotation then adds object position |
Complete-object fields from C3DPlayer constructor and walking usage:
| Offset | Initial value | Confirmed use |
|---|---|---|
| +0x6BC | 30.0f | Speed increment per input invocation |
| +0x6C0 | 35.0f | Speed decay per invocation with no movement input |
| +0x6C4 | 600.0f | Speed cap |
| +0x6C8 | 0.0f | Current speed |
| +0x6D4 | 0.0f | Current turn rate, later multiplied by dt |
| +0x7C8,+0x7CC,+0x7D0 | 0,200,-350 | Camera local follow offset |
| +0x7D8,+0x7DC,+0x7E0 | 0,80,0 | Camera local look target offset |
| +0x854,+0x858,+0x85C | 1.6,1,1.6 | Camera interpolation coefficients |
**Correction to a preliminary investigation hypothesis:** -350 at +0x7D0 is
camera offset, not gravity. Do not use it as a physics constant.
Walking input uses global booleans 0x50985D/0x50985F for left/right and
0x50985E/0x509860 for forward/backward, plus analog axes. The precise mapping
is inferred from signed +/- turn behavior and must be checked against input
assignment code. Left/right branches set turn contribution +/-1000, adjust a
local angle by +/-20 and increase speed by30 capped600. Forward also adds30.
No movement input decays speed by35 toward0. These changes omit dt deliberately
in the original. Forward plus turn can increment twice; don't simplify silently.
The application uses 16384 angular units per revolution (mask0x3FFF); conversion
from degrees multiplies by float45.5111122 at EXE 0x48E5E8, followed by x87-to-int
conversion0x47F940. Original trigonometry uses OMediaCosSinTable, 16384 floats per
sin/cos array with cosine base offset0x10004. Position is X/Y/Z with Y vertical,
matching follow-camera local offset and level placement data.
Walking follow-camera logic is in 0x4395CF–0x4398E4. It rotates local offsets,
subtracts prior target positions, applies dt-dependent interpolation, adds to
global viewport position at `[0x509A50]+0x44`, and computes viewport angles. This
is richer than a rigid chase camera; a simple chase camera is an approximation.
Random behavior at0x438D52 uses wrapping uint32 LCG
`seed = seed * 1664525 + 32767; value = seed >> 16`. Verified from instruction
sequence; seed object pointer is global0x509A40. This differs from MSVC rand().
## Reproducible tooling and next work
From GamePort:
```sh
python3 tools/binary/pe_inspect.py ../Original/Neutron.exe
python3 tools/binary/pe_inspect.py ../Original/OMT2.dll --symbols 'render|update_logic'
python3 tools/binary/rtti_map.py ../Original/Neutron.exe C3DJimmy
objdump -d -M intel --start-address=0x438bc0 --stop-address=0x4398f2 ../Original/Neutron.exe
```
Generated complete PE inventories/disassemblies are cached under ignored
`artifacts/binary/`; regenerate with the commands above rather than versioning
the full dumps. Read-only parser output was cross-checked against objdump header,
import and named-export results. Full physics/collision, jump impulse, cadence,
AI, save/load and dynamic initialization remain unresolved. Next investigate the
game-object dispatch0x479BC0, shared physics collision function via game slot0x130
in0x462520, input assignment and compare a copied original in isolated Wine.
## Follow-up evidence preserved by primary integration
The last agent updates below refine earlier unresolved fields. General physics
arithmetic is implemented/tested separately; the full controller is not recovered.
- Generic gravity initially (0,-5000,0) at `0x478D50`; CJimmyGame initialization
`0x44D414` overrides it to **(0,-800,0)**. World physics dispatch `0x479BC0`
calls per-object `0x479D20`. At `0x479EBB..0x479EF2`, queued impulse is added
to velocity and cleared; then object acceleration*dt and gravity*dt are added,
and updated velocity*dt becomes displacement at `0x479F8D` onward.
- Jump branch `0x438397` selects JUMP; `0x4383B3` queues impulse (0,550,0) via
interface slot0x28C. **Gravity -800 alone is not the full player jump model.**
Air handler `0x437F90`, branch `0x438148`, sets extra accelerationY to
`gravityY*1.33f-gravityY` when jump state<3, producing approximately -1064
total acceleration. Constant `0x4AFCE4` is float1.33000004. Other air states
use 0.1/0.3 factors and require further tracing before implementation.
- Jump-held predicate `0x42A730` includes right mouse/Insert/Shift/numpad0 and
controller/gadget gates. Mode dispatch `0x437A8C`: state0 requires grounded;
jump queues impulse and state1; release state1->2; holding0.4s can enter
state3 with jetpack gating. Exact landing/retrigger table is still unresolved.
- Walking +/-20 affects destination yaw through `0x472660`; `0x43A430` smooths
wrapped Y-angle difference by `dt*3.7`. Forward vector at `0x472B10` for
zero pitch/roll is **(-sin(yaw)*speed,0,cos(yaw)*speed)**. Ground motion
`0x437C40` preserves vertical velocity. A right vector (cosYaw,0,sinYaw) is
scaled by `(turnRate*(1-friction)*speed/45 + keyboardStrafe)*dt`, with strafe
+1000 left / -1000 right (`0x4391B1` / `0x439226`). Friction source and full
state gates still require recovery; a simple tank controller would differ.
- Key state array `0x509838` is indexed by Windows key code; handlers `0x46A360`
and `0x46A380` set/clear it; consuming function `0x46A3E0` clears on read.
- C3DJimmy::InitObject `0x422C96..0x422CB0`: texture slot0 uses **jimycarl.png**
(stringVA `0x4EEAC4`), slot1 uses **jimyshock.png**. Inferring normal texture
from neighboring strings alone was incorrect and has been corrected.
Original execution was subsequently attempted in an isolated Wine copy; see
`../reference-testing.md`. No usable original gameplay frames are verified yet.
+124
View File
@@ -0,0 +1,124 @@
# Original rendering evidence
Static analysis of supplied OMT2.dll / Neutron.exe, 2026-10-23. No rendered
reference frame has been captured yet. All DLL addresses use preferred base
0x10000000; EXE addresses use0x400000.
## API and backend
The supplied DLL implements **Direct3D7 fixed-function rendering over DirectDraw7**
and an **OMT software rasterizer**. This is more specific than readme's DirectX8
runtime requirement: input uses DirectInput8 but rendering uses D3D7 interfaces.
Evidence:
- DDRAW imports DirectDrawCreateEx and DirectDrawEnumerateA; there is no D3D8 import.
- IID_IDirect3D7 `{F5049E77-4861-11D2-A407-00A0C90629A8}` occurs at DLL0x10116E0C;
IID_IDirectDraw7 `{15E65EC0-3B9C-11D2-B92F-00609797EA5B}` at0x10116E1C.
- OMediaDXRenderPort::draw at0x100372A0 calls device vtable offset0x64 (D3D7
DrawPrimitive) with FVF0x152. SetTransform uses offset0x2C and SetRenderState0x50.
- Named exports OMediaDXRenderer/DXRenderPort, OMediaOMTRenderer/OMTRenderPort,
software texture16/texture32 rasterization functions and a CPU pipeline coexist.
- Executable contains renderer labels OPENGL, DIRECT3D HAL and DIRECT3D T&L HAL.
The OPENGL label does **not** prove this supplied DLL implements an OpenGL backend:
no matching GL import/backend exports were found. Do not implement one by assumption.
## Function map
| Preferred DLL VA | Named export / behavior |
|---|---|
| 0x10035D80 | OMediaDXRenderPort constructor |
| 0x10036320 | begin_render, initializes/reuses device state |
| 0x10036560 | end_render |
| 0x100372A0 | draw, translates OMediaRenderVertex vector to D3D vertices |
| 0x100374B0 | draw_shape, polygon/material path |
| 0x10037AC0 | draw_surface, canvas quad path |
| 0x10036AF0 | set_model_view, D3D transform1 |
| 0x10036B10 | set_projection, D3D transform3 |
| 0x1004D650 | OMediaLayer::prepare_projection_matrix |
| 0x1004E650 | perspective projection with symmetric -1..1 clip Z |
| 0x1004E6E0 | perspective projection with 0..1 clip Z |
| 0x1003FC00 | OMediaElement::compute_model_matrix overload |
| 0x1003FF70 | compute_model_matrix overload using parent matrix |
| 0x100EE740 | OMediaViewPort::update_render_ports |
## Vertex formats
OMediaDXRenderPort::draw reads input vertices with **68-byte stride**, converts
them to **36-byte D3D vertices**, and passes **FVF0x152**:
| D3D vertex offset | Contents | OMT input offset |
|---|---|---|
| 0x00 | float3 position |0x00 |
| 0x0C | float3 normal |0x34 |
| 0x18 | packed ARGB diffuse |float RGBA0x10,0x14,0x18,0x1C |
| 0x1C | float2 texture UV |0x2C |
FVF = XYZ0x2 + NORMAL0x10 + DIFFUSE0x40 + TEX10x100. It does not contain
pretransformed RHW coordinates. Source color packing multiplies each channel by
DLL constant0x10114A10 and converts through x87 integer conversion. Primitive enum
mapping in this function and exact polygon triangulation still need examination.
## State translations proven by disassembly
| OMT function | Address | D3D state evidence | Vulkan requirement |
|---|---|---|---|
| enable_faceculling |0x10036570 |CULLMODE22=2 (CW) | Match winding after coordinate/viewport transforms |
| disable_faceculling |0x10036590 |CULLMODE22=1 (none) | Per-material culling flag |
| turn_fog_on |0x100365F0 |FOGENABLE28=1; FOGVERTEXMODE140=3 (linear) | Vertex-derived linear fog compatible with reference |
| set_fog_color |0x10036650 |FOGCOLOR34 | Preserve packed color |
| set_fog_range |0x10036670 |FOGSTART36, FOGEND37 | Float fog distances |
| set_zbuffer_write |0x10036910 |ZWRITEENABLE14 | Pipeline depth-write enable |
| set_zbuffer_test |0x10036940 |ZENABLE7 | Pipeline depth-test enable |
| set_zbuffer_func |0x10036970 |ZFUNC23 | Map each original comparison enum |
| set_blend |0x10036A50 |ALPHABLENDENABLE27 | Pipeline blend enable |
| set_blend_func |0x10036AB0 |SRCBLEND19, DESTBLEND20 | Preserve original factors |
begin_render default setup0x10036484 onward sets CW culling, NORMALIZENORMALS142=1,
LIGHTING137=0, texture stage0 COLOROP1=4 (MODULATE), COLORARG12=2 (TEXTURE),
COLORARG23=0 (DIFFUSE). Thus an unlit texture-times-diffuse baseline is evidence
based; game/material overrides still need full tracing. Lighting, fog and alpha
capabilities in the engine do not prove that every game scene enables them.
## Projection and coordinates
OMT angles are16384 units/revolution. Perspective0x1004E6E0 uses
`s=cot(fov/2)`, matrix entries `m00=s*height/width`, `m11=s`,
`m22=far/(far-near)`, `m23=1`, `m32=-near*far/(far-near)` with other perspective
entries zero except unused m33 initialized1. This is the positive-Z-forward,
0..1 depth form. The alternative0x1004E650 uses `(far+near)/(far-near)` and
`-2*near*far/(far-near)` for -1..1 depth. Matrix storage includes a trailing type
word at+0x40 beyond the16 floats. These are engine constructors; game-selected FOV,
near/far and projection mode remain to be recovered from viewport/layer setup.
The game uses **Y-up** world placement, while exported ASE files are3ds Max
coordinates; don't apply guessed axis changes to OMT geometry. The exact ASE
converter's rotation/UV policy must be confirmed against OMT geometry, not inferred
from application conventions. Vulkan requires handling viewport Y and winding
consistently. No claim of full handedness equivalence is made without a frame test.
## Implementation priorities and remaining evidence
Implement original geometry/UV/material colors, D3D0..1 projection, texture modulation,
depth and material culling/blending first. Add recovered fog and fixed-function
lights as actually required. Do not replace with PBR. Reuse original morph animation
data, not invented skeletal animation. Validate triangle order, UV orientation,
alpha, depth, filtering and camera against original before claiming visual fidelity.
Not yet recovered: complete material enum maps, game-selected sampler modes, light
values, FOV/clips, render sorting, alpha thresholds, animation timing and visual
reference captures. See asset report for archive model/material recovery.
## Projection follow-up
OMediaLayer ctor `0x1004D340`: projection mode3, FOV word+0x78=`0x0AAA`
(59.9853515625 degrees), near+0xC4=20, far+0xC0=3200. Game creates five layers
at `0x47664C` onward. Sky layer global0x509A38 far1,000,000 at `0x47682C`;
geometry globals0x509A30/0x509A3C. CLevel01BGame setup `0x44F5A0` sets far
10000 on both geometry layers. No game FOV writes were found by initial static
search; that is evidence for inherited FOV, not a proof of every possible path.
Primary renderer review additionally recovered null-material skipping and v2
polygon culling flags from DX draw_shape; see `../rendering.md`. The previous
white fallback for unresolved material18 incorrectly rendered invisible lab
proxy surfaces and is fixed. Full material state mapping remains incomplete.
+138
View File
@@ -0,0 +1,138 @@
# Original asset formats
This describes formats recovered from the supplied files and OMT2.dll serializers.
All multibyte OMT metadata is big-endian unless explicitly stated otherwise.
Paths below are relative to `Original/`. See [asset evidence](analysis/assets.md)
for executable addresses, confidence, test results and limitations.
## 0MF2 OMT database
| Offset / field | Representation |
|---|---|
| `0` | ASCII `0MF2` |
| `4` | BE32 absolute resource-index offset |
| Index start | BE32 resource-type count |
| Each type | fourcc, BE32 entry count |
| Each entry | BE32 ID, absolute offset, length; byte Pascal-name length + Latin-1 bytes |
IDs are unsigned on disk; some metadata IDs represent negative signed values.
Offsets point to live resource payloads. Resources can occur in arbitrary order
and an archive can contain obsolete/unindexed allocations. Use the index instead
of blindly scanning magic strings. `0HDR` describes allocation/index bookkeeping
and can overlap the index region. It is preserved, not decoded as a game object.
`mCTy` records per-object metadata; interpretation remains incomplete.
Examples: `omt/level1.omt` index at `0x1ee333`; type counts are 47 `Canv`,
195 `3DSh`, 181 `3DMa`, one `mCTy`, one `0HDR`.
Object links serialize as BE16 boolean; if nonzero, fourcc + BE32 resource ID
follow. All booleans used by these serializers occupy **two** bytes.
## 3DSh / 3DSP version 2
All 2,155 supplied shape payloads use this exact version:
1. `3DSP`, BE32 version (2).
2. Eight BE32 floats (bounds/settings); additional legacy float + BE16 boolean.
3. BE32 vertex count; each vertex three BE32 IEEE-754 floats.
4. BE32 polygon count; variable polygon records below.
5. BE32 color count; each color four BE32 floats.
6. BE32 normal count; each normal three BE32 floats.
Polygon v2:
1. Two BE16 booleans, BE16 legacy field, BE32 ID, two BE32 floats.
2. BE16 legacy field, BE16 corner count.
3. Each corner: BE32 vertex index, float U, float V, BE32 normal index.
4. Three float polygon normal coordinates.
5. BE16 material-present boolean; if true, serialized object link.
Vertices preserve original scale and engine world coordinates. Do not normalize
individual scene shapes or reapply arbitrary transforms. Names can identify
already placed world objects. `3DMa` IDs are local to the source database.
Collision meaning of legacy polygon flags has not yet been recovered.
## 3DMa compact version 1
Every supplied material uses BE16 version 1, followed by BE16 legacy style 0..2,
a float scalar and BE16 flag. Styles 1/2 store four unsigned16 RGB triplets and
another boolean; style 0 stores one triplet. Style 2 adds a boolean, BE16 texture
presence enum and, for enum 1, an object link. Final BE16 field completes record.
Typical textured material length is 52 bytes; untextured variants can be 38/42.
RGB triplets are converted to floats by dividing by 65,535. The output keys
`emission`, `diffuse`, `specular`, `ambient` are inferred conventional roles;
legacy flags are retained with neutral names pending renderer validation.
## Canv / OmCv / OmGW
| Field | Representation |
|---|---|
| Signature | `OmCv` |
| Flags | BE32 |
| Pixel header | `OmGW` |
| Width, height, bit depth | BE16 each |
| Packing unit enum, color-key kind | byte each |
| Decoded row byte stride | BE16 |
| Has color key | BE16 boolean |
| Key | RGB16 triplet for key-kind 0; otherwise BE32 only when key present |
| Has palette | BE16 boolean |
| Optional palette | described below |
| Compressed data size | BE32 |
| Data | one BE16 encoded row length + encoded bytes per image row |
Packing enum 0/1/2 selects RLE units of 1/2/4 bytes. This is a PackBits variant:
control values are **little-endian** signed values of the unit width (the dword
variant tests low signed16 bits). Nonnegative N copies N+1 literal units; negative
N repeats the next unit 1-N times; -128 is a no-op. Encoded row must consume
exactly its advertised bytes and produce exactly the decoded stride.
Pixel values remain big-endian in encoded literal/repeat payloads. RGB555 selects
bits 10..14, 5..9, 0..4 and shifts each channel left three, matching original
conversion. The 32-bit format selects RGB at bits 16/8/0. Both derive alpha from
full stored-pixel equality with the color key; without a key alpha becomes 255.
Palette signature `OPa2` or `OmPa`, Pascal-name, BE32 color count, count RGB16
triplets. `OPa2` additionally stores a 4,096-byte palette accelerator. The original
converter takes low bytes of palette16 values after endian conversion. The twelve
indexed textures all use 8-bit pixels and decode with this layout.
## Wave resources
A `Wave` database entry contains RIFF/WAVE bytes either directly or after an OMT
object prefix. RIFF chunk sizes and WAVE audio format are little-endian. Extraction
finds the RIFF inside its indexed resource and preserves exactly `size+8` bytes;
it checks that the result is WAVE and never crosses the indexed resource boundary.
PCM is reusable by cross-platform audio. One MP3-in-WAVE is retained unchanged
and requires decoding support if that particular resource is played. Surrounding
OMT object metadata may contain playback settings; extracting a WAV does not
recover loop policy automatically.
## ASE source models
`ASE/*.ASE` and `ASE/jimstop.ase` are 3DS Max ASCII export files. Source object
positions are already exported vertex coordinates. Shared original OMT/ASE
objects prove the conversion `(x,y,z) -> (x,z,y)` and UV `(u,v) -> (u,-v)` while
retaining face winding. `MESH_ANIMATION` contains complete vertex samples tagged
with `TIMEVALUE`; scene FPS and ticks/frame establish source sample times.
Some morph-only files omit UVs and can have a different face table from the
original base shape. The original renderer keeps base topology and texture
coordinates while substituting morph vectors by vertex index. Use `--base`
explicitly rather than rendering missing UVs as `(0,0)`. First prepared player
base is `jimstop.ase`, normal runtime atlas is `png/jimycarl.png` (texture slot 0);
`png/jimyshock.png` is a separate effect in slot 1. See InitObject evidence.
## Prepared scene interchange
`scene.json` has `vertices:[[x,y,z]]`, `uvs:[[u,v]]`, `normals:[[x,y,z]]`,
`indices:[uint...]`, and `meshes:[{name,shapeId,firstIndex,indexCount,material}]`.
Vertices are expanded per polygon corner to retain discontinuous UVs and normals.
`materials` retain source IDs, texture links and numeric fields. `textures` give
ID/name/width/height and relative paths to tightly packed row-major RGBA8 bytes
and PNG preview. Source archive filename and SHA-256 establish provenance.
No coordinate or UV convention conversion is applied to already native OMT data.
`animation.json` records source hash, FPS, ticks/frame, frame JSON relative paths,
original TIMEVALUE, morph base provenance and optional original texture metadata.
Frame JSON uses the scene vertex/UV/index schema. Preview normals are currently
flat normals, explicitly marked in the manifest; original smooth shading is pending.
+146
View File
@@ -0,0 +1,146 @@
import json
from pathlib import Path
import struct
import sys
import tempfile
import unittest
import zlib
ROOT = Path(__file__).resolve().parents[1]
ORIGINAL = ROOT.parent / 'Original'
sys.path.insert(0, str(ROOT / 'tools' / 'extractors'))
from omt import Database, FormatError, Reader, export_scene, read_canvas, read_material, read_shape, unpack_row, validate_output, write_png
from ase import read_ase, export_ase
from png import read_png
class BoundedReaders(unittest.TestCase):
def test_read_bounds_and_negative_size(self):
for size in (-1, 2):
with self.assertRaises(FormatError): Reader(b'x').take(size)
def test_serialized_bool_is_two_bytes(self):
r = Reader(b'\x00\x01\x00\x00')
self.assertTrue(r.boolean())
self.assertFalse(r.boolean())
r.end()
def test_packbits_bytes_and_pixel_words(self):
self.assertEqual(unpack_row(b'\x02abc\xfeZ', 6, 1), b'abcZZZ')
self.assertEqual(unpack_row(b'\x01\x00\x7c\x00\x03\xe0\xfe\xff\x00\x1f', 10, 2),
b'\x7c\x00\x03\xe0\x00\x1f\x00\x1f\x00\x1f')
def test_packbits_rejects_overrun_truncation_and_leftovers(self):
for encoded, expected, unit in ((b'\x7fX', 1, 1), (b'\xff', 2, 1), (b'\x00XY', 1, 1)):
with self.assertRaises(FormatError): unpack_row(encoded, expected, unit)
def test_canvas_rgb555_matches_original_shift_and_key(self):
# Original pixel converter uses channel<<3, yielding 248 at full intensity.
row = b'\x02\x00\x7c\x00\x03\xe0\x00\x1f'
pixel_data = struct.pack('>H', len(row)) + row
header = b'OmCv'+struct.pack('>I', 0)+b'OmGW'+struct.pack('>HHHBBH', 3, 1, 16, 1, 1, 6)
header += struct.pack('>HIHI', 1, 0x03e0, 0, len(pixel_data))
canvas = read_canvas(header+pixel_data)
self.assertEqual(canvas['rgba'], bytes((248,0,0,255,0,248,0,0,0,0,248,255)))
def test_png_roundtrip_payload(self):
rgba = bytes((4,5,6,255,10,20,30,0))
with tempfile.TemporaryDirectory() as t:
path = Path(t)/'test.png'
write_png(path, 2, 1, rgba)
data = path.read_bytes()
self.assertEqual(data[:8], b'\x89PNG\r\n\x1a\n')
p, compressed = 8, bytearray()
while p < len(data):
size = struct.unpack_from('>I', data, p)[0]
tag, payload = data[p+4:p+8], data[p+8:p+8+size]
self.assertEqual(zlib.crc32(tag+payload), struct.unpack_from('>I',data,p+8+size)[0])
if tag == b'IDAT': compressed.extend(payload)
p += size+12
self.assertEqual(zlib.decompress(compressed), b'\x00'+rgba)
self.assertEqual(read_png(path)['rgba'],rgba)
@unittest.skipUnless(ORIGINAL.exists(), 'Original game files are local integration fixtures')
class OriginalResources(unittest.TestCase):
def test_every_database_index_is_bounded(self):
paths = list(ORIGINAL.rglob('*.omt'))
self.assertEqual(len(paths), 100)
self.assertEqual(sum(len(Database(p).entries) for p in paths), 7027)
def test_first_scene_geometry_materials_and_textures(self):
db = Database(ORIGINAL/'omt'/'level1b.omt')
counts = {}
for entry in db.entries:
if entry.type == '3DSh': read_shape(db.payload(entry))
elif entry.type == '3DMa': read_material(db.payload(entry))
elif entry.type == 'Canv':
canvas = read_canvas(db.payload(entry))
self.assertEqual(len(canvas['rgba']), canvas['width']*canvas['height']*4)
counts[entry.type] = counts.get(entry.type, 0)+1
self.assertEqual(counts['Canv'], 33)
def test_scene_export_contains_original_geometry_and_texture_links(self):
db = Database(ORIGINAL/'omt'/'VR01.omt')
with tempfile.TemporaryDirectory() as t:
output = Path(t)/'scene'
scene = export_scene(db, output)
self.assertEqual(len(scene['indices']), 240)
self.assertEqual(len(scene['vertices']), 240)
self.assertEqual(len(scene['textures']), 7)
self.assertTrue(all(0 <= i < len(scene['vertices']) for i in scene['indices']))
self.assertEqual([list(v) for v in scene['vertices']], json.loads((output/'scene.json').read_text())['vertices'])
for texture in scene['textures']:
self.assertEqual((output/texture['path']).stat().st_size, texture['width']*texture['height']*4)
def test_extraction_cannot_target_original_directory(self):
db = Database(ORIGINAL/'omt'/'VR01.omt')
for target in (ORIGINAL, ORIGINAL/'omt', ORIGINAL/'omt'/'converted'):
with self.assertRaises(ValueError): validate_output(db, target)
def test_indexed_texture_recovery(self):
db = Database(ORIGINAL/'omt'/'sprites.omt')
entry = next(e for e in db.entries if e.type=='Canv' and e.name=='shadow2')
canvas = read_canvas(db.payload(entry))
self.assertEqual((canvas['width'],canvas['height'],canvas['depth']), (64,32,8))
self.assertIn(0, canvas['rgba'][3::4])
def test_ase_coordinate_uv_and_winding_match_original_omt(self):
db = Database(ORIGINAL/'omt'/'objects.omt')
for name in ('gate1','yokianpart'):
entry = next(e for e in db.entries if e.type=='3DSh' and e.name==name)
omt = read_shape(db.payload(entry))
ase = read_ase(ORIGINAL/'ASE'/(name+'.ASE'))['objects'][0]['samples'][0]
self.assertEqual(len(omt['vertices']),len(ase['vertices']))
self.assertEqual(len(omt['polygons']),len(ase['faces']))
for original,recovered in zip(omt['vertices'],ase['vertices']):
for a,b in zip(original,recovered): self.assertAlmostEqual(a,b,places=3)
for polygon,face,tf in zip(omt['polygons'],ase['faces'],ase['textureFaces']):
self.assertEqual(tuple(c['vertex'] for c in polygon['corners']),face)
for corner,uv_index in zip(polygon['corners'],tf):
for a,b in zip(corner['uv'],ase['uvs'][uv_index]): self.assertAlmostEqual(a,b,places=6)
def test_jimmy_morph_animation_samples_and_texture(self):
for name,count in (('jimstop.ase',2),('jimrun.ASE',5),('jimjump.ASE',3)):
animation=read_ase(ORIGINAL/'ASE'/name)
self.assertEqual((animation['framesPerSecond'],animation['ticksPerFrame']),(10,480))
self.assertEqual(len(animation['objects'][0]['samples']),count)
self.assertTrue(all(len(m['faces'])==814 for m in animation['objects'][0]['samples']))
texture=read_png(ORIGINAL/'png'/'jimycarl.png')
self.assertEqual((texture['width'],texture['height'],len(texture['rgba'])),(256,256,256*256*4))
def test_morph_reuses_textured_base_topology(self):
with tempfile.TemporaryDirectory() as t:
output=Path(t)
with self.assertRaises(FormatError):
export_ase(ORIGINAL/'ASE'/'jimrun.ASE',output/'invalid')
export_ase(ORIGINAL/'ASE'/'jimrun.ASE',output/'run',ORIGINAL/'ASE'/'jimstop.ase')
export_ase(ORIGINAL/'ASE'/'jimstop.ase',output/'base')
frame=json.loads((output/'run'/'frame_0000.json').read_text())
base=json.loads((output/'base'/'frame_0000.json').read_text())
self.assertEqual(frame['uvs'],base['uvs'])
self.assertTrue(any(uv != [0,0] for uv in frame['uvs']))
self.assertNotEqual(frame['vertices'],base['vertices'])
if __name__ == '__main__': unittest.main()
+140
View File
@@ -0,0 +1,140 @@
#!/usr/bin/env python3
"""Read-only PE32 import/export/RTTI evidence inventory, no third-party modules.
Addresses are preferred-base VAs, not relocation-adjusted runtime addresses.
Usage: python3 tools/binary/pe_inspect.py ../Original/OMT2.dll --symbols 'render|timer'
"""
import argparse
import hashlib
import json
from pathlib import Path
import re
import struct
class PE:
def __init__(self, path):
self.path = Path(path)
self.data = self.path.read_bytes()
if self.data[:2] != b'MZ':
raise ValueError('Missing MZ signature')
nt = self.u32(0x3c)
if self.data[nt:nt+4] != b'PE\0\0':
raise ValueError('Missing PE signature')
self.machine, count = struct.unpack_from('<HH', self.data, nt+4)
self.timestamp = self.u32(nt+8)
optsize = self.u16(nt+20)
opt = nt+24
if self.u16(opt) != 0x10b:
raise ValueError('This inspector supports PE32 only')
self.base = self.u32(opt+28)
self.entry = self.base+self.u32(opt+16)
self.header_size = self.u32(opt+60)
self.linker = list(self.data[opt+2:opt+4])
self.directories = [struct.unpack_from('<II', self.data, opt+96+8*i)
for i in range(min(16, self.u32(opt+92)))]
self.sections = []
for i in range(count):
off = opt+optsize+i*40
name = self.data[off:off+8].split(b'\0', 1)[0].decode('ascii')
vs, rva, size, raw = struct.unpack_from('<IIII', self.data, off+8)
self.sections.append(dict(name=name, virtual_size=vs, rva=rva,
raw_size=size, file_offset=raw))
def u16(self, off):
return struct.unpack_from('<H', self.data, off)[0]
def u32(self, off):
return struct.unpack_from('<I', self.data, off)[0]
def offset(self, rva):
if 0 <= rva < self.header_size:
return rva
for s in self.sections:
delta = rva-s['rva']
if 0 <= delta < s['raw_size']:
return s['file_offset']+delta
raise ValueError(f'RVA {rva:#x} has no file data')
def va(self, off):
if 0 <= off < self.header_size:
return self.base+off
for s in self.sections:
delta = off-s['file_offset']
if 0 <= delta < s['raw_size']:
return self.base+s['rva']+delta
raise ValueError(f'Offset {off:#x} has no virtual address')
def cstr(self, off):
end = self.data.index(0, off)
return self.data[off:end].decode('ascii', 'replace')
def exports(self):
rva, size = self.directories[0]
if not rva:
return []
off = self.offset(rva)
ordinal_base, count, names, functions, nameptrs, ordinals = struct.unpack_from(
'<IIIIII', self.data, off+16)
result = []
for i in range(names):
idx = self.u16(self.offset(ordinals)+2*i)
target = self.u32(self.offset(functions)+4*idx)
name = self.cstr(self.offset(self.u32(self.offset(nameptrs)+4*i)))
result.append(dict(name=name, ordinal=ordinal_base+idx,
va=f'0x{self.base+target:08x}', rva=target))
return result
def imports(self):
rva, _ = self.directories[1]
result = []
if not rva:
return result
off = self.offset(rva)
while any(self.data[off:off+20]):
original, _, _, name, first = struct.unpack_from('<IIIII', self.data, off)
dll = self.cstr(self.offset(name))
thunk = self.offset(original or first)
i = 0
while (v := self.u32(thunk+4*i)):
n = f'ordinal:{v & 0xffff}' if v & 0x80000000 else self.cstr(self.offset(v)+2)
result.append(dict(dll=dll, name=n, iat=f'0x{self.base+first+4*i:08x}'))
i += 1
off += 20
return result
def strings(self, pattern):
result = []
regex = re.compile(pattern, re.I)
for m in re.finditer(rb'[\x20-\x7e]{4,}', self.data):
value = m.group().decode('ascii')
if regex.search(value):
result.append(dict(value=value, offset=f'0x{m.start():x}',
va=f'0x{self.va(m.start()):08x}'))
return result
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('binary', type=Path)
parser.add_argument('--symbols', help='Regex filter for imports and exports')
parser.add_argument('--strings', help='Regex filter for ASCII strings with VAs')
args = parser.parse_args()
pe = PE(args.binary)
imports, exports = pe.imports(), pe.exports()
if args.symbols:
regex = re.compile(args.symbols, re.I)
imports = [x for x in imports if regex.search(x['name'])]
exports = [x for x in exports if regex.search(x['name'])]
out = dict(path=str(args.binary), sha256=hashlib.sha256(pe.data).hexdigest(),
machine=hex(pe.machine), linker=pe.linker, timestamp=pe.timestamp,
image_base=hex(pe.base), entry_point=hex(pe.entry),
clr_directory=pe.directories[14], sections=pe.sections,
imports=imports, exports=exports)
if args.strings:
out['strings'] = pe.strings(args.strings)
print(json.dumps(out, indent=2))
if __name__ == '__main__':
main()
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env python3
"""Find MSVC x86 RTTI complete-object locators and candidate vtables.
Output is evidence, not a claim that every candidate is an actual vtable.
Usage: python3 tools/binary/rtti_map.py ../Original/Neutron.exe C3DJimmy
"""
import argparse
import json
import re
import struct
from pe_inspect import PE
def rtti_map(pe, class_name):
marker = ('.?AV'+class_name+'@@').encode('ascii')
at = pe.data.find(marker)
if at < 8:
raise ValueError(f'No type descriptor for {class_name}')
type_va = pe.va(at-8)
result = dict(class_name=class_name, type_descriptor=hex(type_va), vtables=[])
for m in re.finditer(re.escape(struct.pack('<I', type_va)), pe.data):
off = m.start()-12
if off < 0 or off % 4:
continue
sig, displacement, ctor_displacement, _, hierarchy = struct.unpack_from('<IIIII', pe.data, off)
if sig != 0 or displacement > 0x100000:
continue
locator_va = pe.va(off)
for ref in re.finditer(re.escape(struct.pack('<I', locator_va)), pe.data):
if ref.start() % 4:
continue
methods = []
for i in range(512):
pos = ref.start()+4+i*4
if pos+4 > len(pe.data):
break
va = pe.u32(pos)
try:
function_off = pe.offset(va-pe.base)
section = next(s for s in pe.sections if s['file_offset'] <= function_off < s['file_offset']+s['raw_size'])
if section['name'] != '.text':
break
except (ValueError, StopIteration):
break
methods.append(dict(slot=hex(i*4), va=hex(va)))
if methods:
result['vtables'].append(dict(va=hex(pe.va(ref.start()+4)),
complete_object_locator=hex(locator_va), object_displacement=hex(displacement),
hierarchy=hex(hierarchy), methods=methods))
return result
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('binary')
parser.add_argument('class_name')
args = parser.parse_args()
print(json.dumps(rtti_map(PE(args.binary), args.class_name), indent=2))
+151
View File
@@ -0,0 +1,151 @@
#!/usr/bin/env python3
"""Recover supplied 3DS Max ASE mesh animation samples into OMT coordinates."""
from __future__ import annotations
import argparse
import hashlib
import json
import math
from pathlib import Path
import re
from omt import FormatError
NUMBER = r'[-+\d.eE]+'
def blocks(text: str, tag: str):
for match in re.finditer(r'\*'+re.escape(tag)+r'\s*\{', text):
start, depth = match.end(), 1
cursor = start
while cursor < len(text) and depth:
if text[cursor] == '{': depth += 1
elif text[cursor] == '}': depth -= 1
cursor += 1
if depth: raise FormatError(f'Unclosed {tag} block')
yield match.start(), text[start:cursor-1]
def vector_table(text: str, tag: str):
values = {}
pattern = r'\*'+tag+r'\s+(\d+)\s+('+NUMBER+r')\s+('+NUMBER+r')\s+('+NUMBER+r')'
for match in re.finditer(pattern, text):
index = int(match[1])
if index in values: raise FormatError(f'Duplicate {tag} index {index}')
values[index] = tuple(float(match[i]) for i in (2,3,4))
if set(values) != set(range(len(values))): raise FormatError(f'Noncontiguous {tag} indices')
if any(not math.isfinite(v) for p in values.values() for v in p): raise FormatError('Nonfinite ASE value')
return [values[i] for i in range(len(values))]
def read_mesh(text: str) -> dict:
vertices = vector_table(text, 'MESH_VERTEX')
tvertices = vector_table(text, 'MESH_TVERT')
faces, tfaces = {}, {}
for match in re.finditer(r'\*MESH_FACE\s+(\d+):\s+A:\s*(\d+)\s+B:\s*(\d+)\s+C:\s*(\d+)', text):
faces[int(match[1])] = tuple(int(match[i]) for i in (2,3,4))
for match in re.finditer(r'\*MESH_TFACE\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)', text):
tfaces[int(match[1])] = tuple(int(match[i]) for i in (2,3,4))
for key, count in (('MESH_NUMVERTEX',len(vertices)),('MESH_NUMFACES',len(faces))):
match = re.search(r'\*'+key+r'\s+(\d+)', text)
if not match or int(match[1]) != count: raise FormatError(f'{key} does not match data')
if set(faces) != set(range(len(faces))): raise FormatError('Noncontiguous face indices')
if tfaces and set(tfaces) != set(faces): raise FormatError('Texture faces do not match faces')
if any(i>=len(vertices) for face in faces.values() for i in face): raise FormatError('ASE vertex outside table')
if any(i>=len(tvertices) for face in tfaces.values() for i in face): raise FormatError('ASE UV outside table')
time = re.search(r'\*TIMEVALUE\s+(-?\d+)',text)
return {'time': int(time[1]) if time else 0,
'vertices': [(x,z,y) for x,y,z in vertices],
'uvs': [(u,-v) for u,v,_ in tvertices],
'faces': [faces[i] for i in range(len(faces))],
'textureFaces': [tfaces[i] for i in range(len(tfaces))] if tfaces else []}
def read_ase(path: Path | str) -> dict:
path = Path(path)
data = path.read_bytes()
text = data.decode('latin1')
if not text.lstrip().startswith('*3DSMAX_ASCIIEXPORT'): raise FormatError('Expected ASE signature')
objects = []
for _, geom in blocks(text, 'GEOMOBJECT'):
name = re.search(r'\*NODE_NAME\s+"([^"]+)"', geom)
meshes = []
animations = list(blocks(geom, 'MESH_ANIMATION'))
if animations:
for _, animation in animations: meshes.extend(read_mesh(body) for _,body in blocks(animation,'MESH'))
else: meshes = [read_mesh(body) for _,body in blocks(geom,'MESH')]
objects.append({'name': name[1] if name else '', 'samples': meshes})
if not objects: raise FormatError('No geometry in ASE')
def setting(name):
match = re.search(r'\*'+name+r'\s+(\d+)',text)
if not match: raise FormatError(f'Missing ASE scene setting {name}')
return int(match[1])
return {'source':path.name, 'sha256':hashlib.sha256(data).hexdigest(),
'framesPerSecond':setting('SCENE_FRAMESPEED'),
'ticksPerFrame':setting('SCENE_TICKSPERFRAME'), 'objects':objects,
'coordinates':'(ASE x,z,y), texture (u,-v); matched OMT duplicate assets'}
def expand_mesh(mesh: dict) -> dict:
result = {'vertices':[], 'uvs':[], 'indices':[], 'normals':[]}
for face_index, face in enumerate(mesh['faces']):
a,b,c = [mesh['vertices'][i] for i in face]
ab,ac = [b[i]-a[i] for i in range(3)], [c[i]-a[i] for i in range(3)]
normal = [ab[1]*ac[2]-ab[2]*ac[1], ab[2]*ac[0]-ab[0]*ac[2], ab[0]*ac[1]-ab[1]*ac[0]]
length = math.sqrt(sum(v*v for v in normal))
normal = [v/length for v in normal] if length else [0.,0.,0.]
for corner,index in enumerate(face):
result['indices'].append(len(result['vertices']))
result['vertices'].append(mesh['vertices'][index])
result['uvs'].append(mesh['uvs'][mesh['textureFaces'][face_index][corner]] if mesh['textureFaces'] else [0.,0.])
result['normals'].append(normal)
return result
def export_ase(path: Path, output: Path, base_path: Path | None = None):
if output.resolve().is_relative_to(path.parent.resolve()) or path.resolve().is_relative_to(output.resolve()):
raise ValueError('Output must be separate from original inputs')
animation = read_ase(path)
base_animation = read_ase(base_path) if base_path else None
if base_animation and len(base_animation['objects']) != len(animation['objects']):
raise FormatError('Morph base and animation object counts differ')
output.mkdir(parents=True, exist_ok=True)
frames = []
for object_index,item in enumerate(animation['objects']):
for sample in item['samples']:
if base_animation:
base = base_animation['objects'][object_index]['samples'][0]
if len(base['vertices']) != len(sample['vertices']):
raise FormatError('Morph sample vertex count differs from base shape')
# Original OMedia3DMorphAnim supplies only per-index vectors to a
# base shape; its polygon/UV topology remains unchanged.
sample = {**sample, 'faces':base['faces'], 'textureFaces':base['textureFaces'],
'uvs':base['uvs']}
if not sample['textureFaces']:
raise FormatError('Mesh has no UVs; supply --base with the original textured morph base')
filename = f"frame_{len(frames):04d}.json"
mesh = expand_mesh(sample)
mesh['meshes'] = [{'name': item['name'],'firstIndex':0,'indexCount':len(mesh['indices']),'material':-1}]
(output/filename).write_text(json.dumps(mesh,separators=(',',':'))+'\n', encoding='utf-8')
frames.append({'node':item['name'],'time':sample['time'], 'path':filename})
manifest = {key:value for key,value in animation.items() if key != 'objects'}
manifest['frames'] = frames
if base_animation:
manifest['morphBase'] = {'source':base_animation['source'], 'sha256':base_animation['sha256'],
'mode':'Original base topology and UV, sample positions by vertex index'}
manifest['normalMode'] = 'Flat normals reconstructed from geometry; original smoothing not yet replicated'
(output/'animation.json').write_text(json.dumps(manifest,indent=2)+'\n', encoding='utf-8')
return manifest
def main():
p = argparse.ArgumentParser(description=__doc__)
p.add_argument('ase',type=Path)
p.add_argument('--output',type=Path,required=True)
p.add_argument('--base',type=Path,help='Original textured shape whose topology a morph retains')
args = p.parse_args()
manifest = export_ase(args.ase,args.output,args.base)
print(f"{manifest['source']}: {len(manifest['frames'])} samples at {manifest['framesPerSecond']} fps")
if __name__ == '__main__': main()
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env python3
"""Validate every original OMT index, mesh, material, canvas and RIFF boundary."""
import argparse
from collections import Counter
import json
from pathlib import Path
import struct
from omt import Database, FormatError, read_canvas, read_material, read_shape
def audit(original: Path) -> dict:
counts, conversions, failures = Counter(), Counter(), []
archives = sorted(original.rglob('*.omt'))
for path in archives:
try:
db = Database(path)
except (OSError, FormatError) as ex:
failures.append({'source': str(path), 'error': str(ex)})
continue
for entry in db.entries:
counts[entry.type] += 1
function = {'3DSh': read_shape, '3DMa': read_material, 'Canv': read_canvas}.get(entry.type)
try:
payload = db.payload(entry)
if function:
function(payload)
conversions[entry.type] += 1
elif entry.type == 'Wave':
start = payload.find(b'RIFF')
if start < 0 or payload[start+8:start+12] != b'WAVE':
raise FormatError('No RIFF WAVE header')
size = struct.unpack_from('<I', payload, start+4)[0]+8
if size < 12 or start+size > len(payload): raise FormatError('RIFF size exceeds resource')
conversions['Wave'] += 1
except (FormatError, struct.error) as ex:
failures.append({'source': str(path), 'type': entry.type,
'id': entry.id, 'name': entry.name, 'error': str(ex)})
return {'archives': len(archives), 'counts': dict(counts),
'decoded': dict(conversions), 'failures': failures}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('original', type=Path)
parser.add_argument('--report', type=Path)
args = parser.parse_args()
result = audit(args.original)
if args.report:
args.report.parent.mkdir(parents=True, exist_ok=True)
args.report.write_text(json.dumps(result, indent=2)+'\n', encoding='utf-8')
print(json.dumps(result, indent=2))
raise SystemExit(bool(result['failures']))
if __name__ == '__main__': main()
+390
View File
@@ -0,0 +1,390 @@
#!/usr/bin/env python3
"""Read-only Open Media Toolkit 0MF2 database recovery. No third-party modules."""
from __future__ import annotations
import argparse
from dataclasses import asdict, dataclass
import hashlib
import json
import math
from pathlib import Path
import re
import struct
import wave
import zlib
class FormatError(ValueError):
"""Malformed or unsupported original resource."""
class Reader:
def __init__(self, data: bytes, position: int = 0):
self.data = data
self.position = position
def take(self, size: int) -> bytes:
if size < 0 or self.position < 0 or self.position + size > len(self.data):
raise FormatError(f"Read outside resource at 0x{self.position:x}, size {size}")
result = self.data[self.position:self.position + size]
self.position += size
return result
def unpack(self, fmt: str):
return struct.unpack(fmt, self.take(struct.calcsize(fmt)))
def u8(self): return self.unpack('>B')[0]
def u16(self): return self.unpack('>H')[0]
def u32(self): return self.unpack('>I')[0]
def f32(self): return self.unpack('>f')[0]
def boolean(self): return self.u16() != 0
def vec3(self): return self.unpack('>3f')
def string(self): return self.take(self.u8()).decode('latin1')
def count(self, minimum_bytes: int):
count = self.u32()
if count > (len(self.data) - self.position) // minimum_bytes:
raise FormatError(f"Impossible count {count} at 0x{self.position - 4:x}")
return count
def link(self):
if not self.boolean(): return None
return {'type': self.take(4).decode('latin1'), 'id': self.u32()}
def end(self):
if self.position != len(self.data):
raise FormatError(f"Unconsumed resource bytes: {len(self.data)-self.position}")
@dataclass(frozen=True)
class Entry:
type: str
id: int
offset: int
size: int
name: str
class Database:
def __init__(self, path: Path | str):
self.path = Path(path)
self.data = self.path.read_bytes()
reader = Reader(self.data)
if reader.take(4) != b'0MF2':
raise FormatError('Expected 0MF2 database signature')
self.table_offset = reader.u32()
if not 8 <= self.table_offset < len(self.data):
raise FormatError('Database table offset is outside file')
reader.position = self.table_offset
groups = reader.count(8)
self.entries = []
keys = set()
for _ in range(groups):
tag = reader.take(4).decode('latin1')
for _ in range(reader.count(13)):
identity, offset, size = reader.unpack('>III')
name = reader.string()
if offset < 8 or offset + size > len(self.data):
raise FormatError(f'Resource {tag}:{identity} outside file')
if (tag, identity) in keys:
raise FormatError(f'Duplicate resource {tag}:{identity}')
keys.add((tag, identity))
self.entries.append(Entry(tag, identity, offset, size, name))
self.table_end = reader.position
def payload(self, entry: Entry) -> bytes:
return self.data[entry.offset:entry.offset + entry.size]
def read_shape(data: bytes) -> dict:
"""Decode 3DSP versions 0..4 using OMT2.dll read_class evidence."""
r = Reader(data)
if r.take(4) != b'3DSP': raise FormatError('Expected 3DSP shape')
version = r.u32()
if version > 4: raise FormatError(f'Unsupported 3DSP version {version}')
bounds = r.unpack('>8f')
legacy = {'unused': r.f32(), 'flag': r.boolean()} if 0 < version < 3 else None
vertices = [r.vec3() for _ in range(r.count(12))]
polygons = []
for _ in range(r.count(1)):
if version <= 2:
flags = {'first': r.boolean(), 'second': r.boolean(), 'legacy': r.u16()}
identifier = r.u32()
extra = r.unpack('>2f') if version else None
legacy_flag = r.u16()
corners = r.u16()
if corners > (len(data)-r.position)//12: raise FormatError('Impossible polygon corner count')
points = []
for _ in range(corners):
index = r.u32()
uv = r.unpack('>2f')
normal_index = r.u32() if version > 1 else index
if index >= len(vertices): raise FormatError(f'Invalid vertex index {index}')
points.append({'vertex': index, 'uv': uv, 'normal': normal_index})
normal = r.vec3()
# Old polygon format has an explicit material-present bool around link.
material = r.link() if r.boolean() else None
polygons.append({'flags': flags, 'id': identifier, 'extra': extra,
'legacy_flag': legacy_flag, 'corners': points,
'normal': normal, 'material': material})
else:
flags = r.u16()
identifier = r.u32()
points = []
for _ in range(r.count(20)):
index, normal_index, colour_index = r.unpack('>3I')
uv = r.unpack('>2f')
if index >= len(vertices): raise FormatError(f'Invalid vertex index {index}')
points.append({'vertex': index, 'normal': normal_index,
'colour': colour_index, 'uv': uv})
polygons.append({'flags': flags, 'id': identifier, 'corners': points,
'normal': r.vec3(), 'material': r.link()})
colours = [r.unpack('>4f') for _ in range(r.count(16))]
normals = [r.vec3() for _ in range(r.count(12))]
additional = [r.unpack('>4f') for _ in range(r.count(16))] if version > 2 else []
r.end()
if not all(math.isfinite(v) for point in vertices for v in point):
raise FormatError('Nonfinite shape coordinate')
return {'version': version, 'bounds': bounds, 'legacy': legacy, 'vertices': vertices,
'polygons': polygons, 'colours': colours, 'normals': normals,
'additional': additional}
def safe_name(value: str) -> str:
return re.sub(r'[^a-zA-Z0-9._-]+', '_', value).strip('._')[:100] or 'unnamed'
def read_material(data: bytes) -> dict:
r = Reader(data)
version = r.u16()
if version != 1: raise FormatError(f'Unsupported material version {version}')
style = r.u16()
if style > 2: raise FormatError(f'Unsupported material style {style}')
shininess, legacy_flag = r.f32(), r.boolean()
colours = {}
if style == 0:
colours['diffuse'] = [v/65535 for v in r.unpack('>3H')]
else:
for name in ('emission', 'diffuse', 'specular', 'ambient'):
colours[name] = [v/65535 for v in r.unpack('>3H')]
legacy_boolean = r.boolean()
texture, texture_override_flag = None, False
if style == 2:
texture_override_flag = r.boolean()
if r.u16() == 1: texture = r.link()
trailing = r.u16()
r.end()
return {'version': version, 'style': style, 'shininess': shininess,
'legacyFlag': legacy_flag, 'colours': colours,
'legacyBoolean': legacy_boolean if style else False, 'textureOverrideFlag': texture_override_flag,
'texture': texture, 'trailing': trailing}
def unpack_row(encoded: bytes, expected_size: int, unit: int) -> bytes:
"""OMT PackBits: LE signed control of unit width, payload in source pixel order."""
r = Reader(encoded)
output = bytearray()
while len(output) < expected_size:
control_bytes = r.take(unit)
# 32-bit variant tests the low signed 16 bits, as original routine does.
control = int.from_bytes(control_bytes[:min(unit, 2)], 'little', signed=True)
if control == -128: continue
count = control + 1 if control >= 0 else 1 - control
if len(output) + count*unit > expected_size:
raise FormatError('RLE run exceeds decoded row')
output.extend(r.take(count*unit) if control >= 0 else r.take(unit)*count)
r.end()
return bytes(output)
def read_canvas(data: bytes) -> dict:
r = Reader(data)
magic = r.take(4)
if magic != b'OmCv': raise FormatError(f'Unsupported canvas signature {magic!r}')
flags = r.u32()
if r.take(4) != b'OmGW': raise FormatError('Missing OmGW pixel header')
width, height, depth = r.unpack('>3H')
packing, key_type, row_bytes = r.u8(), r.u8(), r.u16()
if not width or not height or width*height > 16777216: raise FormatError('Invalid canvas dimensions')
if packing not in (0, 1, 2): raise FormatError('Unknown canvas packing')
if depth not in (8, 16, 32): raise FormatError(f'Unsupported canvas depth {depth}')
has_key = r.boolean()
key = None
if key_type == 0:
rgb = r.unpack('>3H')
if has_key:
channels = [v*(31 if depth == 16 else 255)//65535 for v in rgb]
key = ((channels[0]<<10)|(channels[1]<<5)|channels[2]) if depth == 16 else ((channels[0]<<16)|(channels[1]<<8)|channels[2])
elif has_key: key = r.u32()
palette = None
if r.boolean():
signature = r.take(4)
if signature not in (b'OPa2', b'OmPa'): raise FormatError('Invalid canvas palette signature')
r.string() # Palette name; absent for these originals.
count = r.u32()
if count > 256: raise FormatError('Palette exceeds 256 colours')
palette = [tuple(v & 255 for v in r.unpack('>3H')) for _ in range(count)]
if signature == b'OPa2': r.take(4096) # Legacy palette matching accelerator.
if count <= 2: palette = [(255,255,255), (0,0,0)]
if depth == 8 and palette is None: raise FormatError('Indexed canvas has no palette')
size = r.u32()
compressed = Reader(r.take(size))
r.end()
if row_bytes < width*(depth//8): raise FormatError('Canvas row stride too small')
pixels = bytearray()
for _ in range(height):
row = unpack_row(compressed.take(compressed.u16()), row_bytes, 1 << packing)
for x in range(width):
v = int.from_bytes(row[x*(depth//8):(x+1)*(depth//8)], 'big')
if depth == 8:
if v >= len(palette): raise FormatError('Pixel index outside palette')
rgb = palette[v]
elif depth == 16:
rgb = (((v>>10)&31)<<3, ((v>>5)&31)<<3, (v&31)<<3)
else: rgb = ((v>>16)&255, (v>>8)&255, v&255)
pixels.extend((*rgb, 0 if key is not None and v == key else 255))
compressed.end()
return {'width': width, 'height': height, 'depth': depth, 'flags': flags,
'packing': packing, 'rowBytes': row_bytes, 'colourKey': key,
'rgba': bytes(pixels)}
def write_png(path: Path, width: int, height: int, rgba: bytes):
if len(rgba) != width*height*4: raise ValueError('Incorrect RGBA byte count')
def chunk(tag, payload):
return struct.pack('>I', len(payload)) + tag + payload + struct.pack('>I', zlib.crc32(tag+payload))
scanlines = b''.join(b'\0' + rgba[y*width*4:(y+1)*width*4] for y in range(height))
path.write_bytes(b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', struct.pack('>IIBBBBB', width, height, 8, 6, 0, 0, 0))
+ chunk(b'IDAT', zlib.compress(scanlines)) + chunk(b'IEND', b''))
def export_scene(database: Database, output: Path):
"""Expanded original corner data; triangle fans preserve source winding."""
validate_output(database, output)
output.mkdir(parents=True, exist_ok=True)
scene = {'source': database.path.name, 'sha256': hashlib.sha256(database.data).hexdigest(),
'coordinates': 'Original OMT world coordinates, unchanged',
'vertices': [], 'uvs': [], 'normals': [], 'indices': [], 'meshes': [],
'materials': [], 'textures': [], 'warnings': []}
for entry in database.entries:
payload = database.payload(entry)
if entry.type == '3DSh':
shape = read_shape(payload)
for poly in shape['polygons']:
first_index = len(scene['indices'])
first_vertex = len(scene['vertices'])
for corner in poly['corners']:
scene['vertices'].append(shape['vertices'][corner['vertex']])
scene['uvs'].append(corner['uv'])
ni = corner['normal']
scene['normals'].append(shape['normals'][ni] if ni < len(shape['normals']) else poly['normal'])
for i in range(1, len(poly['corners'])-1):
scene['indices'].extend((first_vertex, first_vertex+i, first_vertex+i+1))
if len(poly['corners']) > 3:
scene['warnings'].append(f'{entry.id}: polygon with {len(poly["corners"])} corners fan triangulated; concavity not validated')
material = poly['material']['id'] if poly['material'] else -1
if scene['meshes'] and scene['meshes'][-1]['shapeId'] == entry.id and scene['meshes'][-1]['material'] == material:
scene['meshes'][-1]['indexCount'] += len(scene['indices'])-first_index
else:
scene['meshes'].append({'name': entry.name, 'shapeId': entry.id,
'firstIndex': first_index, 'indexCount': len(scene['indices'])-first_index, 'material': material})
elif entry.type == '3DMa':
material = read_material(payload)
scene['materials'].append({'id': entry.id, 'name': entry.name, **material})
elif entry.type == 'Canv':
canvas = read_canvas(payload)
base = f'texture_{entry.id:08x}'
(output / (base + '.rgba')).write_bytes(canvas.pop('rgba'))
rgba = (output / (base + '.rgba')).read_bytes()
write_png(output / (base + '.png'), canvas['width'], canvas['height'], rgba)
scene['textures'].append({'id': entry.id, 'name': entry.name,
'path': base + '.rgba', 'png': base + '.png', **canvas})
(output / 'scene.json').write_text(json.dumps(scene, separators=(',', ':')) + '\n', encoding='utf-8')
return scene
def write_obj(shape: dict, path: Path):
"""Preserve original positions/UVs/polygon normals in a standard interchange file."""
lines = ['# Original OMT coordinates; no transform or guessed smoothing applied']
lines.extend('v ' + ' '.join(format(v, '.9g') for v in p) for p in shape['vertices'])
for poly in shape['polygons']:
lines.extend('vt ' + ' '.join(format(v, '.9g') for v in c['uv']) for c in poly['corners'])
lines.extend('vn ' + ' '.join(format(v, '.9g') for v in p['normal']) for p in shape['polygons'])
uv_index = 1
for normal_index, poly in enumerate(shape['polygons'], 1):
if poly['material']: lines.append(f"usemtl material_{poly['material']['id']}")
corners = []
for corner in poly['corners']:
corners.append(f"{corner['vertex']+1}/{uv_index}/{normal_index}")
uv_index += 1
if len(corners) >= 3: lines.append('f ' + ' '.join(corners))
path.write_text('\n'.join(lines) + '\n', encoding='utf-8')
def extract(database: Database, output: Path, raw: bool = False):
output = output.resolve()
validate_output(database, output)
output.mkdir(parents=True, exist_ok=True)
report = {'source': str(database.path), 'sha256': hashlib.sha256(database.data).hexdigest(),
'table_offset': database.table_offset, 'entries': []}
for entry in database.entries:
info = asdict(entry)
payload = database.payload(entry)
base = f'{safe_name(entry.type)}_{entry.id:08x}_{safe_name(entry.name)}'
if raw and entry.type != '0HDR':
(output / (base + '.bin')).write_bytes(payload)
if entry.type == '3DSh':
try:
shape = read_shape(payload)
write_obj(shape, output / (base + '.obj'))
(output / (base + '.json')).write_text(json.dumps(shape), encoding='utf-8')
info.update(vertices=len(shape['vertices']), polygons=len(shape['polygons']))
except FormatError as ex: info['unsupported'] = str(ex)
if entry.type == 'Wave':
start = payload.find(b'RIFF')
if start >= 0 and payload[start+8:start+12] == b'WAVE':
size = struct.unpack_from('<I', payload, start+4)[0] + 8
if start + size <= len(payload):
target = output / (base + '.wav')
target.write_bytes(payload[start:start+size])
try:
with wave.open(str(target)) as wav:
info['audio'] = {'channels': wav.getnchannels(), 'rate': wav.getframerate(),
'sample_width': wav.getsampwidth(), 'frames': wav.getnframes()}
except wave.Error:
info['audio'] = {'encoding': 'non-PCM RIFF preserved unchanged'}
else: info['unsupported'] = 'RIFF size outside resource'
report['entries'].append(info)
(output / 'manifest.json').write_text(json.dumps(report, indent=2) + '\n', encoding='utf-8')
return report
def validate_output(database: Database, output: Path):
source_directory = database.path.parent.resolve()
target = output.resolve()
if target.is_relative_to(source_directory) or database.path.resolve().is_relative_to(target):
raise ValueError('Output must be separate from original input directory')
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('database', type=Path)
parser.add_argument('--output', type=Path)
parser.add_argument('--raw', action='store_true', help='Also extract exact raw resources')
parser.add_argument('--scene', action='store_true', help='Export runtime scene.json with decoded textures')
args = parser.parse_args()
db = Database(args.database)
if args.output:
if args.scene:
scene = export_scene(db, args.output)
print(f"{len(scene['vertices'])} vertices, {len(scene['indices'])//3} triangles, {len(scene['textures'])} textures; {args.output}")
return
report = extract(db, args.output, args.raw)
failures = [e for e in report['entries'] if 'unsupported' in e]
print(f'{len(db.entries)} resources; {len(failures)} unsupported conversions; {args.output}')
for fail in failures[:10]: print(f" {fail['type']}:{fail['id']} {fail['unsupported']}")
else:
for e in db.entries: print(f'{e.type} {e.id:10d} {e.offset:10d} {e.size:10d} {e.name}')
if __name__ == '__main__': main()
+68
View File
@@ -0,0 +1,68 @@
"""Small bounded decoder for the supplied non-interlaced 8-bit PNG textures."""
from pathlib import Path
import struct
import zlib
from omt import FormatError, Reader
def read_png(path: Path) -> dict:
r = Reader(path.read_bytes())
if r.take(8) != b'\x89PNG\r\n\x1a\n': raise FormatError('Invalid PNG signature')
header, palette, transparency = None, None, b''
compressed = bytearray()
ended = False
while r.position < len(r.data):
size = r.u32()
tag, data = r.take(4), r.take(size)
if r.u32() != zlib.crc32(tag+data): raise FormatError('PNG checksum mismatch')
if tag == b'IHDR': header = struct.unpack('>IIBBBBB',data)
elif tag == b'PLTE': palette = data
elif tag == b'tRNS': transparency = data
elif tag == b'IDAT': compressed.extend(data)
elif tag == b'IEND': ended = True; break
elif tag[0] & 32 == 0: raise FormatError(f'Unsupported critical PNG chunk {tag!r}')
if not ended or not header: raise FormatError('Incomplete PNG')
r.end()
width,height,depth,colour,compression,filter_method,interlace = header
if depth != 8 or compression or filter_method or interlace: raise FormatError('Only non-interlaced 8-bit PNG is supported')
channels = {0:1,2:3,3:1,4:2,6:4}.get(colour)
if not channels or not width or not height or width*height > 16777216: raise FormatError('Invalid PNG dimensions/colour')
stride = width*channels
expected = (stride+1)*height
decoder = zlib.decompressobj()
raw = decoder.decompress(bytes(compressed),expected+1)
if len(raw)!=expected or not decoder.eof or decoder.unused_data or decoder.unconsumed_tail:
raise FormatError('Invalid PNG decoded length')
rgba, previous = bytearray(), bytearray(stride)
for y in range(height):
mode = raw[y*(stride+1)]
scan = bytearray(raw[y*(stride+1)+1:(y+1)*(stride+1)])
if mode > 4: raise FormatError('Invalid PNG scanline filter')
for x in range(stride):
a = scan[x-channels] if x>=channels else 0
b = previous[x]
c = previous[x-channels] if x>=channels else 0
if mode==1: predict = a
elif mode==2: predict = b
elif mode==3: predict = (a+b)//2
elif mode==4:
p=a+b-c;pa,pb,pc=abs(p-a),abs(p-b),abs(p-c)
predict = a if pa<=pb and pa<=pc else b if pb<=pc else c
else: predict=0
scan[x] = (scan[x]+predict)&255
for x in range(width):
pixel = scan[x*channels:(x+1)*channels]
if colour==3:
index=pixel[0]
if palette is None or index*3+3>len(palette): raise FormatError('PNG pixel outside palette')
rgba.extend((*palette[index*3:index*3+3],transparency[index] if index<len(transparency) else 255))
elif colour==6: rgba.extend(pixel)
elif colour==2:
alpha=0 if transparency and tuple(pixel)==struct.unpack('>3H',transparency) else 255
rgba.extend((*pixel,alpha))
else:
alpha=pixel[1] if colour==4 else 0 if transparency and pixel[0]==struct.unpack('>H',transparency)[0] else 255
rgba.extend((pixel[0],pixel[0],pixel[0],alpha))
previous=scan
return {'width':width,'height':height,'rgba':bytes(rgba)}
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""Prepare local scene/audio assets without modifying the original installation."""
import argparse
import json
from pathlib import Path
from omt import Database, export_scene, extract
from ase import export_ase
from png import read_png
def find_archive(original: Path, name: str) -> Path:
matches = [p for p in (original / 'omt').iterdir()
if p.name.casefold() == (name if name.casefold().endswith('.omt') else name + '.omt').casefold()]
if len(matches) != 1:
raise ValueError(f'Expected exactly one archive for {name}, found {len(matches)}')
return matches[0]
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--original', type=Path, default=Path('../Original'))
parser.add_argument('--output', type=Path, default=Path('artifacts/extracted'))
parser.add_argument('--scenes', nargs='+', default=['level1b', 'VR01'])
parser.add_argument('--audio', nargs='*', default=['sounds', 'musicjimmyshouse', 'musicneighborhood'])
parser.add_argument('--animations', nargs='*', default=['jimstop', 'jimrun', 'jimjump'])
args = parser.parse_args()
if args.output.resolve().is_relative_to(args.original.resolve()):
parser.error('Output must be outside the original installation')
for name in args.scenes:
source = find_archive(args.original, name)
scene = export_scene(Database(source), args.output / source.stem)
print(f"{source.name}: {len(scene['indices'])//3} triangles, {len(scene['textures'])} textures")
for name in args.audio:
source = find_archive(args.original, name)
report = extract(Database(source), args.output / source.stem)
count = sum(1 for e in report['entries'] if 'audio' in e)
print(f'{source.name}: {count} audio resources')
if args.animations:
texture_source = next(p for p in (args.original/'png').iterdir() if p.name.casefold()=='jimycarl.png')
texture = read_png(texture_source)
target = args.output/'animations'
target.mkdir(parents=True,exist_ok=True)
(target/'jimycarl.rgba').write_bytes(texture.pop('rgba'))
base = next(p for p in (args.original/'ASE').iterdir() if p.name.casefold()=='jimstop.ase')
for name in args.animations:
matches = [p for p in (args.original/'ASE').iterdir() if p.stem.casefold()==name.casefold() and p.suffix.casefold()=='.ase']
if len(matches)!=1: raise ValueError(f'Expected one ASE for {name}')
manifest = export_ase(matches[0],target/name,base)
manifest['texture'] = {'path':'../jimycarl.rgba', **texture}
(target/name/'animation.json').write_text(json.dumps(manifest,indent=2)+'\n',encoding='utf-8')
print(f"{matches[0].name}: {len(manifest['frames'])} animation samples")
if __name__ == '__main__': main()