Recover OMT resources, morph topology, and original engine evidence
This commit is contained in:
1 parent
d62e2ab4de
commit
3df20ac19a
12 files changed
+1648
No files matched your search
@@ -0,0 +1,141 @@
|
|||||||
|
# Asset recovery evidence
|
||||||
|
|
||||||
|
Verified against local originals on 2026-10-23. No original files were modified.
|
||||||
|
`Original/` is outside the modern local Git repository. Analysis/extraction uses
|
||||||
|
Python standard library only; nothing is uploaded.
|
||||||
|
|
||||||
|
## Verified corpus
|
||||||
|
|
||||||
|
`python3 tools/extractors/audit.py ../Original --report artifacts/extracted/audit.json`
|
||||||
|
successfully parsed all 100 `.omt` databases, 7,027 resource entries, with no
|
||||||
|
conversion failures:
|
||||||
|
|
||||||
|
| Resource | Count | Verification |
|
||||||
|
|---|---:|---|
|
||||||
|
| `3DSh` | 2,155 | Every shape v2 consumed exactly; positions finite; corner indices bounded |
|
||||||
|
| `3DMa` | 2,183 | Every material v1 consumed exactly; original texture links recovered |
|
||||||
|
| `Canv` | 1,614 | All RLE decoded to exact row lengths and all resource bytes consumed |
|
||||||
|
| `Wave` | 875 | RIFF/WAVE header and bounded RIFF lengths present |
|
||||||
|
| `mCTy` | 100 | Indexed metadata preserved; internal interpretation outstanding |
|
||||||
|
| `0HDR` | 100 | Indexed allocation metadata preserved; not runtime geometry |
|
||||||
|
|
||||||
|
Texture counts are 866 RGB555, 736 32-bit RGB, and 12 indexed 8-bit.
|
||||||
|
All 875 sound resources contain intact RIFF data: 874 PCM and one format-tag 85
|
||||||
|
(MP3 in WAVE). PCM rates/channels/sample widths vary; do not assume one audio
|
||||||
|
format. Loop and cue behavior in the surrounding OMT metadata remains unverified.
|
||||||
|
|
||||||
|
## Recovered scene outputs
|
||||||
|
|
||||||
|
Run from `GamePort/`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 tools/extractors/prepare.py
|
||||||
|
python3 -m unittest discover -s tests -p test_extractors.py -v
|
||||||
|
```
|
||||||
|
|
||||||
|
Preparation produces ignored local files below `artifacts/extracted/`:
|
||||||
|
|
||||||
|
- `level1b/scene.json`: 8,607 expanded vertices, 2,869 triangles, 33 textures.
|
||||||
|
- `VR01/scene.json`: 240 expanded vertices, 80 triangles, 7 textures.
|
||||||
|
- Texture `.rgba` files plus PNG previews, preserving material ID links.
|
||||||
|
- `sounds/`, `musicjimmyshouse/`, `musicneighborhood/`: exact embedded WAV bytes.
|
||||||
|
- `animations/{jimstop,jimrun,jimjump}/`: original morph samples and manifests.
|
||||||
|
- `animations/jimycarl.rgba`: original normal Jimmy atlas, 256 x 256 indexed PNG decoded to RGBA.
|
||||||
|
|
||||||
|
Individual archive recovery:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 tools/extractors/omt.py ../Original/omt/level1b.omt --scene --output artifacts/extracted/level1b
|
||||||
|
python3 tools/extractors/omt.py ../Original/omt/sounds.omt --output artifacts/extracted/sounds --raw
|
||||||
|
python3 tools/extractors/ase.py ../Original/ASE/jimrun.ASE --base ../Original/ASE/jimstop.ase --output artifacts/extracted/animations/jimrun
|
||||||
|
```
|
||||||
|
|
||||||
|
The static OMT scene is geometry/material recovery, not a complete level state.
|
||||||
|
GAM entity transforms, object visibility, character meshes, lighting, camera,
|
||||||
|
gameplay collision masks, triggers and task progression must be applied separately.
|
||||||
|
All source scene polygons examined in these first scene exports are triangles;
|
||||||
|
the exporter supports polygon fans and warns for any larger polygons whose
|
||||||
|
concavity has not been validated.
|
||||||
|
|
||||||
|
## Executable evidence used
|
||||||
|
|
||||||
|
Preferred virtual addresses in the supplied `OMT2.dll` (image base `0x10000000`):
|
||||||
|
|
||||||
|
| Routine | Address | Supporting behavior |
|
||||||
|
|---|---|---|
|
||||||
|
| `OMedia3DShape::read_class` | `0x10007b60` | `3DSP` signature, version, vectors and polygon arrays |
|
||||||
|
| `OMedia3DPolygon::read_class` | `0x10003e20` | Version-dependent corners, UVs, normal indices, material links |
|
||||||
|
| `OMedia3DMaterial::read_class` | `0x10001490` | Compact v1 styles, color triplets and texture stream link |
|
||||||
|
| `OMediaCanvas::read_class` | `0x10025b80` | Dispatches OmCv through canvas converter |
|
||||||
|
| `OMediaOMTConverter::convert` | `0x10027df0` | OmGW dimensions, packing/key/palette header and row RLE |
|
||||||
|
| RLE unpack helper | `0x1006c230` | Byte/word/dword PackBits variants; LE signed controls |
|
||||||
|
| `OMediaOMTConverter::create_canvas` | `0x10028620` | Pixel RGB and alpha/color-key conversion |
|
||||||
|
| `OMediaOMTConverter::read_palette` | `0x10027b40` | OmPa/OPa2 palette with 4,096-byte accelerator |
|
||||||
|
| `OMediaDBObjectStreamLink::read_class` | `0x10030260` | BE16 present + fourcc + BE32 ID |
|
||||||
|
| Bool stream operator | `0x10073a80` | Booleans occupy two bytes on disk |
|
||||||
|
| `OMedia3DMorphAnim::render_draw_shape` | `0x10002df0` | Retains base shape while overriding per-index vectors |
|
||||||
|
|
||||||
|
RGB555 channels are expanded by `<< 3`, **not** normalized to 255. Instructions
|
||||||
|
at `0x10028815` onward verify this; maximum channels therefore remain 248.
|
||||||
|
The 32-bit path at `0x100288ab` selects RGB bits 16/8/0 and derives alpha from
|
||||||
|
color-key equality. Stored high-byte alpha is not copied by that converter.
|
||||||
|
Raw byte-exact resources remain available using `--raw`.
|
||||||
|
|
||||||
|
## ASE geometry and animation
|
||||||
|
|
||||||
|
The supplied ASE assets include standard ASCII mesh vertex/face tables, per-corner
|
||||||
|
texture indices, scene timing and vertex morph samples. `jimstop.ase` differs in
|
||||||
|
filename case from most other ASE files. Resolve filenames case-insensitively.
|
||||||
|
|
||||||
|
Two matching assets provide independent coordinate evidence, covered by tests:
|
||||||
|
|
||||||
|
- All eight `gate1.ASE` vertices and all eight `yokianpart.ASE` vertices match the
|
||||||
|
corresponding `objects.omt` resources after `(x,y,z) -> (x,z,y)`.
|
||||||
|
- Face index order remains identical; the reflection is not accompanied by an
|
||||||
|
index swap in the stored original.
|
||||||
|
- Every corner UV matches `(u,v) -> (u,-v)`.
|
||||||
|
- No additional `NODE_TM` is applied to these already exported vertex positions.
|
||||||
|
|
||||||
|
Source ASE normal fields are not assumed authoritative; exported preview normals
|
||||||
|
currently use flat triangle normals. Original smoothing must still be replicated.
|
||||||
|
|
||||||
|
Jimmy uses 426 source vertices and 814 faces. `jimstop.ase` has two samples,
|
||||||
|
`jimrun.ASE` five, and `jimjump.ASE` three, all with file scene speed 10 FPS and
|
||||||
|
480 ticks/frame. These establish asset sample timing, not state-specific playback
|
||||||
|
speed or loop policy in gameplay code.
|
||||||
|
|
||||||
|
The EXE stores `jimyshock.png` at file offset `0xef0c8`, followed by `jimstop.ase`
|
||||||
|
at `0xef0d8`. **Primary integration corrected an early inference from string
|
||||||
|
proximity:** InitObject at `0x422C96..0x422CB0` binds `jimycarl.png` (VA
|
||||||
|
`0x4EEAC4`) to texture slot 0 and `jimyshock.png` to slot 1. The latter is the
|
||||||
|
shock/skeleton effect, not normal appearance. Prepared animation textures and
|
||||||
|
runtime defaults now use `jimycarl.png`; both have been visually inspected.
|
||||||
|
The ASE bitmap path `D:\\Jimmy\\jimmy\\jimmylast.bmp` is not a runtime dependency.
|
||||||
|
Run/jump filenames follow in the same initialization string cluster.
|
||||||
|
|
||||||
|
### Morph topology retention
|
||||||
|
|
||||||
|
Run/jump/scratch ASE files omit UV tables. Zero UVs would show a single texel and
|
||||||
|
are rejected by the animation exporter. The original morph renderer validates
|
||||||
|
the sample vector count against the base shape and interpolates per-index vectors
|
||||||
|
(`0x10002ed3`, `0x10002eee`, `0x10002f9a..0x10002fd9`), then submits the base
|
||||||
|
shape plus override vectors (`0x10003396..0x100033ab`). Its topology/UV remain
|
||||||
|
on the base shape. Therefore the prepared Jimmy animations explicitly use the
|
||||||
|
textured `jimstop.ase` base and original sample vertex indices. This also matters
|
||||||
|
because 23 face records in run/jump differ from the later dated stop asset even
|
||||||
|
though all sample vertex counts remain 426. `morphBase` provenance in each
|
||||||
|
animation manifest records the source and hash of the retained shape.
|
||||||
|
|
||||||
|
## Limits and next work
|
||||||
|
|
||||||
|
- Static asset decoding is verified, but visual fidelity needs comparison with an
|
||||||
|
isolated original game run.
|
||||||
|
- Legacy material booleans and polygon flags remain numerically preserved rather
|
||||||
|
than assigned unproven lighting/culling/collision meanings.
|
||||||
|
- Color triplet roles are conventional labels inferred from the legacy material
|
||||||
|
layout; exact renderer use should be cross-checked against recovered DX states.
|
||||||
|
- Animation exporter supports all supplied mesh records, but character bundles
|
||||||
|
prepared so far cover Jimmy stop/run/jump only. Original smooth normal and
|
||||||
|
state-driven playback behavior remain incomplete.
|
||||||
|
- Native runtime renderer/audio integration and package validation belong to the
|
||||||
|
primary/infrastructure work; successful extraction is not runtime verification.
|
||||||
@@ -0,0 +1,181 @@
|
|||||||
|
# Executable investigation
|
||||||
|
|
||||||
|
Static analysis, 2026-10-23. No original code was executed or modified. Addresses
|
||||||
|
below are preferred-base virtual addresses of **Neutron.exe** or **OMT2.dll**;
|
||||||
|
patch variants must be checked independently. No Ghidra/Rizin/Wine was available
|
||||||
|
on PATH initially; the initial investigation uses Python, strings and objdump.
|
||||||
|
|
||||||
|
## Verified identity and toolchain
|
||||||
|
|
||||||
|
| File | Format | Preferred base | Entry | Header timestamp |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| Neutron.exe | PE32 GUI, native Intel 80386 | 0x00400000 | 0x004802F2 | 2001-09-29 21:07:40 UTC |
|
||||||
|
| OMT2.dll | PE32 DLL, native Intel 80386 | 0x10000000 | 0x10105753 | 2001-08-27 10:27:41 UTC |
|
||||||
|
|
||||||
|
Both have linker 6.0, MSVC decorated C++ exports/imports, C++ RTTI, x87 code and
|
||||||
|
MSVC exception handling. Together these strongly identify Microsoft Visual C++ 6
|
||||||
|
era C++; the linker header alone does not prove the exact compiler build. CLR
|
||||||
|
directories are zero. The normal `.text`, `.rdata`, `.data` sections, extensive
|
||||||
|
readable symbols and direct instructions support an unpacked build. No packer
|
||||||
|
was detected; absence is a static inference, not an exhaustive packer test.
|
||||||
|
|
||||||
|
The runtime is **Open Media Toolkit 2.0 DR4**. DLL assertion paths literally name
|
||||||
|
`C:\Open Media Toolkit 2.0 DR4\Sources\OMTClasses\...`, including graphics,
|
||||||
|
database, compression and DirectX implementations. `OMT2Old.dll` is byte-identical
|
||||||
|
to `OMT2.dll` (SHA-256
|
||||||
|
`3ec41ead011adf56387a6dbd4580e5b5a9721b5f8020c8a56420f42824354943`).
|
||||||
|
|
||||||
|
Neutron SHA-256:
|
||||||
|
`f5281076e1a2492f2d61f87aae3f4544b6d0f535ebe33f9fbb6f78710fe1aea8`.
|
||||||
|
Five EXE variants have distinct hashes; filenames imply software/HD/no-CD behavior
|
||||||
|
but no behavior should be assigned solely from filenames.
|
||||||
|
|
||||||
|
## Dependencies and system boundaries
|
||||||
|
|
||||||
|
Neutron imports the OMT DLL and Win32 USER32/KERNEL32/ADVAPI32; consult the
|
||||||
|
generated PE inventory for full lists. The DLL imports DirectDrawCreateEx and
|
||||||
|
DirectDrawEnumerateA from DDRAW, DirectInput8Create from DINPUT8, DSOUND ordinal1,
|
||||||
|
WINMM PlaySoundA, USER32, GDI32, COMDLG32, KERNEL32. Direct3D is obtained through
|
||||||
|
COM rather than a D3D8 import; see [graphics.md](graphics.md).
|
||||||
|
|
||||||
|
The engine has named exports for database objects, shape and morph animation,
|
||||||
|
collision caches, sphere/triangle picking, light/material state, world updates,
|
||||||
|
sound channels, event manager and periodical dispatch. These exports are useful
|
||||||
|
evidence and stable starting points, not complete recovered game logic.
|
||||||
|
|
||||||
|
## Initialization and loop map
|
||||||
|
|
||||||
|
| Module/address | Identified behavior | Evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| EXE 0x4802F2 | CRT entry | PE header, MSVC CRT setup |
|
||||||
|
| EXE 0x4789B0 | WinMain wrapper | CRT calls it with four WinMain args at 0x4803BB; stores OMT start info, sets current directory, calls 0x46A4A0 |
|
||||||
|
| EXE 0x46A4A0 | Application bootstrap | Calls window existence/dialog code, creates CAweReal and CGfx objects |
|
||||||
|
| DLL 0x100218A0 | OMediaApplication::start | Repeatedly invokes OMediaPeriodical::devote_time until application byte +0x18 is true |
|
||||||
|
| DLL 0x10060720 | OMediaPeriodical::devote_time | Named export; dispatch list must be analyzed further |
|
||||||
|
| DLL 0x100215C0 | OMediaAnimPeriodical::spend_time | Sends messages -604 for logic, -605 for animation/render dispatch |
|
||||||
|
| DLL 0x100F9B10 | OMediaWorld::update_logic | Uses timer at world+0x48, pause/max/min-period controls, dispatches update_elements with float elapsed |
|
||||||
|
| DLL 0x100F9C80 | OMediaWorld::update_elements | Named export, recursively dispatches element logic |
|
||||||
|
| EXE 0x47E470 | Reset game performance timer | QueryPerformanceCounter into 0x696018 |
|
||||||
|
| EXE 0x47E490 | Elapsed time in seconds | (counter-lastCounter)/frequency, updates lastCounter when nonzero |
|
||||||
|
| EXE 0x47E4F0 | Game frame callback | Calls timer via virtual slot 0xD4, stores seconds at 0x696030; only processes `0 < dt < 1` |
|
||||||
|
|
||||||
|
QueryPerformanceFrequency IAT is EXE 0x48D0BC, frequency global 0x696020.
|
||||||
|
QueryPerformanceCounter IAT is 0x48D0B8. Engine GetTickCount exists separately.
|
||||||
|
The frame callback forwards seconds to 0x479BC0, 0x462A20, 0x4693A0, 0x47D8F0
|
||||||
|
and 0x477DB0. Their exact roles require further call graph analysis. Timing is
|
||||||
|
mixed: time-based motion plus per-call speed increments, so a proposed fixed
|
||||||
|
simulation rate is a compatibility choice until the original cadence is observed.
|
||||||
|
|
||||||
|
## Player class and recovered movement evidence
|
||||||
|
|
||||||
|
MSVC RTTI identifies C3DJimmy, derived C3DPlayer/C3DAnimated/C3DObject and OMT
|
||||||
|
classes. C3DJimmy type descriptor: **0x4EEFF8**; main game interface vtable:
|
||||||
|
**0x4A373C**, object displacement **0xC0**; OMT element vtable at **0x4A3BDC**.
|
||||||
|
The unusual multiple inheritance means offsets below explicitly distinguish
|
||||||
|
complete object pointers from game-interface pointers.
|
||||||
|
|
||||||
|
| Address | Behavior |
|
||||||
|
|---|---|
|
||||||
|
| 0x422AC0 | C3DJimmy::InitObject (embedded exact debug label) |
|
||||||
|
| 0x422D30 | Jimmy object event/message handler; identified from vtable but exact name unverified |
|
||||||
|
| 0x424600 | Jimmy per-frame update, game-interface virtual slot 0x3C4 |
|
||||||
|
| 0x426030 | Jimmy input/movement handler, game-interface slot 0x3CC; calls base 0x438A60 |
|
||||||
|
| 0x438A60 | Player movement-mode dispatcher; mode0 invokes walking, mode1 no action, mode2 flying, mode3 seated |
|
||||||
|
| 0x438BC0 | Walking handler; receives complete object pointer, `float dt` |
|
||||||
|
| 0x437C40 | Applies player speed/turn to world velocity; receives complete object pointer |
|
||||||
|
| 0x437740 | Sets default follow-camera offsets/scales (complete object pointer) |
|
||||||
|
| 0x472980 | Rotates local point by object Euler rotation then adds object position |
|
||||||
|
|
||||||
|
Complete-object fields from C3DPlayer constructor and walking usage:
|
||||||
|
|
||||||
|
| Offset | Initial value | Confirmed use |
|
||||||
|
|---|---|---|
|
||||||
|
| +0x6BC | 30.0f | Speed increment per input invocation |
|
||||||
|
| +0x6C0 | 35.0f | Speed decay per invocation with no movement input |
|
||||||
|
| +0x6C4 | 600.0f | Speed cap |
|
||||||
|
| +0x6C8 | 0.0f | Current speed |
|
||||||
|
| +0x6D4 | 0.0f | Current turn rate, later multiplied by dt |
|
||||||
|
| +0x7C8,+0x7CC,+0x7D0 | 0,200,-350 | Camera local follow offset |
|
||||||
|
| +0x7D8,+0x7DC,+0x7E0 | 0,80,0 | Camera local look target offset |
|
||||||
|
| +0x854,+0x858,+0x85C | 1.6,1,1.6 | Camera interpolation coefficients |
|
||||||
|
|
||||||
|
**Correction to a preliminary investigation hypothesis:** -350 at +0x7D0 is
|
||||||
|
camera offset, not gravity. Do not use it as a physics constant.
|
||||||
|
|
||||||
|
Walking input uses global booleans 0x50985D/0x50985F for left/right and
|
||||||
|
0x50985E/0x509860 for forward/backward, plus analog axes. The precise mapping
|
||||||
|
is inferred from signed +/- turn behavior and must be checked against input
|
||||||
|
assignment code. Left/right branches set turn contribution +/-1000, adjust a
|
||||||
|
local angle by +/-20 and increase speed by30 capped600. Forward also adds30.
|
||||||
|
No movement input decays speed by35 toward0. These changes omit dt deliberately
|
||||||
|
in the original. Forward plus turn can increment twice; don't simplify silently.
|
||||||
|
|
||||||
|
The application uses 16384 angular units per revolution (mask0x3FFF); conversion
|
||||||
|
from degrees multiplies by float45.5111122 at EXE 0x48E5E8, followed by x87-to-int
|
||||||
|
conversion0x47F940. Original trigonometry uses OMediaCosSinTable, 16384 floats per
|
||||||
|
sin/cos array with cosine base offset0x10004. Position is X/Y/Z with Y vertical,
|
||||||
|
matching follow-camera local offset and level placement data.
|
||||||
|
|
||||||
|
Walking follow-camera logic is in 0x4395CF–0x4398E4. It rotates local offsets,
|
||||||
|
subtracts prior target positions, applies dt-dependent interpolation, adds to
|
||||||
|
global viewport position at `[0x509A50]+0x44`, and computes viewport angles. This
|
||||||
|
is richer than a rigid chase camera; a simple chase camera is an approximation.
|
||||||
|
|
||||||
|
Random behavior at0x438D52 uses wrapping uint32 LCG
|
||||||
|
`seed = seed * 1664525 + 32767; value = seed >> 16`. Verified from instruction
|
||||||
|
sequence; seed object pointer is global0x509A40. This differs from MSVC rand().
|
||||||
|
|
||||||
|
## Reproducible tooling and next work
|
||||||
|
|
||||||
|
From GamePort:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 tools/binary/pe_inspect.py ../Original/Neutron.exe
|
||||||
|
python3 tools/binary/pe_inspect.py ../Original/OMT2.dll --symbols 'render|update_logic'
|
||||||
|
python3 tools/binary/rtti_map.py ../Original/Neutron.exe C3DJimmy
|
||||||
|
objdump -d -M intel --start-address=0x438bc0 --stop-address=0x4398f2 ../Original/Neutron.exe
|
||||||
|
```
|
||||||
|
|
||||||
|
Generated complete PE inventories/disassemblies are cached under ignored
|
||||||
|
`artifacts/binary/`; regenerate with the commands above rather than versioning
|
||||||
|
the full dumps. Read-only parser output was cross-checked against objdump header,
|
||||||
|
import and named-export results. Full physics/collision, jump impulse, cadence,
|
||||||
|
AI, save/load and dynamic initialization remain unresolved. Next investigate the
|
||||||
|
game-object dispatch0x479BC0, shared physics collision function via game slot0x130
|
||||||
|
in0x462520, input assignment and compare a copied original in isolated Wine.
|
||||||
|
|
||||||
|
## Follow-up evidence preserved by primary integration
|
||||||
|
|
||||||
|
The last agent updates below refine earlier unresolved fields. General physics
|
||||||
|
arithmetic is implemented/tested separately; the full controller is not recovered.
|
||||||
|
|
||||||
|
- Generic gravity initially (0,-5000,0) at `0x478D50`; CJimmyGame initialization
|
||||||
|
`0x44D414` overrides it to **(0,-800,0)**. World physics dispatch `0x479BC0`
|
||||||
|
calls per-object `0x479D20`. At `0x479EBB..0x479EF2`, queued impulse is added
|
||||||
|
to velocity and cleared; then object acceleration*dt and gravity*dt are added,
|
||||||
|
and updated velocity*dt becomes displacement at `0x479F8D` onward.
|
||||||
|
- Jump branch `0x438397` selects JUMP; `0x4383B3` queues impulse (0,550,0) via
|
||||||
|
interface slot0x28C. **Gravity -800 alone is not the full player jump model.**
|
||||||
|
Air handler `0x437F90`, branch `0x438148`, sets extra accelerationY to
|
||||||
|
`gravityY*1.33f-gravityY` when jump state<3, producing approximately -1064
|
||||||
|
total acceleration. Constant `0x4AFCE4` is float1.33000004. Other air states
|
||||||
|
use 0.1/0.3 factors and require further tracing before implementation.
|
||||||
|
- Jump-held predicate `0x42A730` includes right mouse/Insert/Shift/numpad0 and
|
||||||
|
controller/gadget gates. Mode dispatch `0x437A8C`: state0 requires grounded;
|
||||||
|
jump queues impulse and state1; release state1->2; holding0.4s can enter
|
||||||
|
state3 with jetpack gating. Exact landing/retrigger table is still unresolved.
|
||||||
|
- Walking +/-20 affects destination yaw through `0x472660`; `0x43A430` smooths
|
||||||
|
wrapped Y-angle difference by `dt*3.7`. Forward vector at `0x472B10` for
|
||||||
|
zero pitch/roll is **(-sin(yaw)*speed,0,cos(yaw)*speed)**. Ground motion
|
||||||
|
`0x437C40` preserves vertical velocity. A right vector (cosYaw,0,sinYaw) is
|
||||||
|
scaled by `(turnRate*(1-friction)*speed/45 + keyboardStrafe)*dt`, with strafe
|
||||||
|
+1000 left / -1000 right (`0x4391B1` / `0x439226`). Friction source and full
|
||||||
|
state gates still require recovery; a simple tank controller would differ.
|
||||||
|
- Key state array `0x509838` is indexed by Windows key code; handlers `0x46A360`
|
||||||
|
and `0x46A380` set/clear it; consuming function `0x46A3E0` clears on read.
|
||||||
|
- C3DJimmy::InitObject `0x422C96..0x422CB0`: texture slot0 uses **jimycarl.png**
|
||||||
|
(stringVA `0x4EEAC4`), slot1 uses **jimyshock.png**. Inferring normal texture
|
||||||
|
from neighboring strings alone was incorrect and has been corrected.
|
||||||
|
|
||||||
|
Original execution was subsequently attempted in an isolated Wine copy; see
|
||||||
|
`../reference-testing.md`. No usable original gameplay frames are verified yet.
|
||||||
@@ -0,0 +1,124 @@
|
|||||||
|
# Original rendering evidence
|
||||||
|
|
||||||
|
Static analysis of supplied OMT2.dll / Neutron.exe, 2026-10-23. No rendered
|
||||||
|
reference frame has been captured yet. All DLL addresses use preferred base
|
||||||
|
0x10000000; EXE addresses use0x400000.
|
||||||
|
|
||||||
|
## API and backend
|
||||||
|
|
||||||
|
The supplied DLL implements **Direct3D7 fixed-function rendering over DirectDraw7**
|
||||||
|
and an **OMT software rasterizer**. This is more specific than readme's DirectX8
|
||||||
|
runtime requirement: input uses DirectInput8 but rendering uses D3D7 interfaces.
|
||||||
|
|
||||||
|
Evidence:
|
||||||
|
|
||||||
|
- DDRAW imports DirectDrawCreateEx and DirectDrawEnumerateA; there is no D3D8 import.
|
||||||
|
- IID_IDirect3D7 `{F5049E77-4861-11D2-A407-00A0C90629A8}` occurs at DLL0x10116E0C;
|
||||||
|
IID_IDirectDraw7 `{15E65EC0-3B9C-11D2-B92F-00609797EA5B}` at0x10116E1C.
|
||||||
|
- OMediaDXRenderPort::draw at0x100372A0 calls device vtable offset0x64 (D3D7
|
||||||
|
DrawPrimitive) with FVF0x152. SetTransform uses offset0x2C and SetRenderState0x50.
|
||||||
|
- Named exports OMediaDXRenderer/DXRenderPort, OMediaOMTRenderer/OMTRenderPort,
|
||||||
|
software texture16/texture32 rasterization functions and a CPU pipeline coexist.
|
||||||
|
- Executable contains renderer labels OPENGL, DIRECT3D HAL and DIRECT3D T&L HAL.
|
||||||
|
The OPENGL label does **not** prove this supplied DLL implements an OpenGL backend:
|
||||||
|
no matching GL import/backend exports were found. Do not implement one by assumption.
|
||||||
|
|
||||||
|
## Function map
|
||||||
|
|
||||||
|
| Preferred DLL VA | Named export / behavior |
|
||||||
|
|---|---|
|
||||||
|
| 0x10035D80 | OMediaDXRenderPort constructor |
|
||||||
|
| 0x10036320 | begin_render, initializes/reuses device state |
|
||||||
|
| 0x10036560 | end_render |
|
||||||
|
| 0x100372A0 | draw, translates OMediaRenderVertex vector to D3D vertices |
|
||||||
|
| 0x100374B0 | draw_shape, polygon/material path |
|
||||||
|
| 0x10037AC0 | draw_surface, canvas quad path |
|
||||||
|
| 0x10036AF0 | set_model_view, D3D transform1 |
|
||||||
|
| 0x10036B10 | set_projection, D3D transform3 |
|
||||||
|
| 0x1004D650 | OMediaLayer::prepare_projection_matrix |
|
||||||
|
| 0x1004E650 | perspective projection with symmetric -1..1 clip Z |
|
||||||
|
| 0x1004E6E0 | perspective projection with 0..1 clip Z |
|
||||||
|
| 0x1003FC00 | OMediaElement::compute_model_matrix overload |
|
||||||
|
| 0x1003FF70 | compute_model_matrix overload using parent matrix |
|
||||||
|
| 0x100EE740 | OMediaViewPort::update_render_ports |
|
||||||
|
|
||||||
|
## Vertex formats
|
||||||
|
|
||||||
|
OMediaDXRenderPort::draw reads input vertices with **68-byte stride**, converts
|
||||||
|
them to **36-byte D3D vertices**, and passes **FVF0x152**:
|
||||||
|
|
||||||
|
| D3D vertex offset | Contents | OMT input offset |
|
||||||
|
|---|---|---|
|
||||||
|
| 0x00 | float3 position |0x00 |
|
||||||
|
| 0x0C | float3 normal |0x34 |
|
||||||
|
| 0x18 | packed ARGB diffuse |float RGBA0x10,0x14,0x18,0x1C |
|
||||||
|
| 0x1C | float2 texture UV |0x2C |
|
||||||
|
|
||||||
|
FVF = XYZ0x2 + NORMAL0x10 + DIFFUSE0x40 + TEX10x100. It does not contain
|
||||||
|
pretransformed RHW coordinates. Source color packing multiplies each channel by
|
||||||
|
DLL constant0x10114A10 and converts through x87 integer conversion. Primitive enum
|
||||||
|
mapping in this function and exact polygon triangulation still need examination.
|
||||||
|
|
||||||
|
## State translations proven by disassembly
|
||||||
|
|
||||||
|
| OMT function | Address | D3D state evidence | Vulkan requirement |
|
||||||
|
|---|---|---|---|
|
||||||
|
| enable_faceculling |0x10036570 |CULLMODE22=2 (CW) | Match winding after coordinate/viewport transforms |
|
||||||
|
| disable_faceculling |0x10036590 |CULLMODE22=1 (none) | Per-material culling flag |
|
||||||
|
| turn_fog_on |0x100365F0 |FOGENABLE28=1; FOGVERTEXMODE140=3 (linear) | Vertex-derived linear fog compatible with reference |
|
||||||
|
| set_fog_color |0x10036650 |FOGCOLOR34 | Preserve packed color |
|
||||||
|
| set_fog_range |0x10036670 |FOGSTART36, FOGEND37 | Float fog distances |
|
||||||
|
| set_zbuffer_write |0x10036910 |ZWRITEENABLE14 | Pipeline depth-write enable |
|
||||||
|
| set_zbuffer_test |0x10036940 |ZENABLE7 | Pipeline depth-test enable |
|
||||||
|
| set_zbuffer_func |0x10036970 |ZFUNC23 | Map each original comparison enum |
|
||||||
|
| set_blend |0x10036A50 |ALPHABLENDENABLE27 | Pipeline blend enable |
|
||||||
|
| set_blend_func |0x10036AB0 |SRCBLEND19, DESTBLEND20 | Preserve original factors |
|
||||||
|
|
||||||
|
begin_render default setup0x10036484 onward sets CW culling, NORMALIZENORMALS142=1,
|
||||||
|
LIGHTING137=0, texture stage0 COLOROP1=4 (MODULATE), COLORARG12=2 (TEXTURE),
|
||||||
|
COLORARG23=0 (DIFFUSE). Thus an unlit texture-times-diffuse baseline is evidence
|
||||||
|
based; game/material overrides still need full tracing. Lighting, fog and alpha
|
||||||
|
capabilities in the engine do not prove that every game scene enables them.
|
||||||
|
|
||||||
|
## Projection and coordinates
|
||||||
|
|
||||||
|
OMT angles are16384 units/revolution. Perspective0x1004E6E0 uses
|
||||||
|
`s=cot(fov/2)`, matrix entries `m00=s*height/width`, `m11=s`,
|
||||||
|
`m22=far/(far-near)`, `m23=1`, `m32=-near*far/(far-near)` with other perspective
|
||||||
|
entries zero except unused m33 initialized1. This is the positive-Z-forward,
|
||||||
|
0..1 depth form. The alternative0x1004E650 uses `(far+near)/(far-near)` and
|
||||||
|
`-2*near*far/(far-near)` for -1..1 depth. Matrix storage includes a trailing type
|
||||||
|
word at+0x40 beyond the16 floats. These are engine constructors; game-selected FOV,
|
||||||
|
near/far and projection mode remain to be recovered from viewport/layer setup.
|
||||||
|
|
||||||
|
The game uses **Y-up** world placement, while exported ASE files are3ds Max
|
||||||
|
coordinates; don't apply guessed axis changes to OMT geometry. The exact ASE
|
||||||
|
converter's rotation/UV policy must be confirmed against OMT geometry, not inferred
|
||||||
|
from application conventions. Vulkan requires handling viewport Y and winding
|
||||||
|
consistently. No claim of full handedness equivalence is made without a frame test.
|
||||||
|
|
||||||
|
## Implementation priorities and remaining evidence
|
||||||
|
|
||||||
|
Implement original geometry/UV/material colors, D3D0..1 projection, texture modulation,
|
||||||
|
depth and material culling/blending first. Add recovered fog and fixed-function
|
||||||
|
lights as actually required. Do not replace with PBR. Reuse original morph animation
|
||||||
|
data, not invented skeletal animation. Validate triangle order, UV orientation,
|
||||||
|
alpha, depth, filtering and camera against original before claiming visual fidelity.
|
||||||
|
|
||||||
|
Not yet recovered: complete material enum maps, game-selected sampler modes, light
|
||||||
|
values, FOV/clips, render sorting, alpha thresholds, animation timing and visual
|
||||||
|
reference captures. See asset report for archive model/material recovery.
|
||||||
|
|
||||||
|
## Projection follow-up
|
||||||
|
|
||||||
|
OMediaLayer ctor `0x1004D340`: projection mode3, FOV word+0x78=`0x0AAA`
|
||||||
|
(59.9853515625 degrees), near+0xC4=20, far+0xC0=3200. Game creates five layers
|
||||||
|
at `0x47664C` onward. Sky layer global0x509A38 far1,000,000 at `0x47682C`;
|
||||||
|
geometry globals0x509A30/0x509A3C. CLevel01BGame setup `0x44F5A0` sets far
|
||||||
|
10000 on both geometry layers. No game FOV writes were found by initial static
|
||||||
|
search; that is evidence for inherited FOV, not a proof of every possible path.
|
||||||
|
|
||||||
|
Primary renderer review additionally recovered null-material skipping and v2
|
||||||
|
polygon culling flags from DX draw_shape; see `../rendering.md`. The previous
|
||||||
|
white fallback for unresolved material18 incorrectly rendered invisible lab
|
||||||
|
proxy surfaces and is fixed. Full material state mapping remains incomplete.
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
# Original asset formats
|
||||||
|
|
||||||
|
This describes formats recovered from the supplied files and OMT2.dll serializers.
|
||||||
|
All multibyte OMT metadata is big-endian unless explicitly stated otherwise.
|
||||||
|
Paths below are relative to `Original/`. See [asset evidence](analysis/assets.md)
|
||||||
|
for executable addresses, confidence, test results and limitations.
|
||||||
|
|
||||||
|
## 0MF2 OMT database
|
||||||
|
|
||||||
|
| Offset / field | Representation |
|
||||||
|
|---|---|
|
||||||
|
| `0` | ASCII `0MF2` |
|
||||||
|
| `4` | BE32 absolute resource-index offset |
|
||||||
|
| Index start | BE32 resource-type count |
|
||||||
|
| Each type | fourcc, BE32 entry count |
|
||||||
|
| Each entry | BE32 ID, absolute offset, length; byte Pascal-name length + Latin-1 bytes |
|
||||||
|
|
||||||
|
IDs are unsigned on disk; some metadata IDs represent negative signed values.
|
||||||
|
Offsets point to live resource payloads. Resources can occur in arbitrary order
|
||||||
|
and an archive can contain obsolete/unindexed allocations. Use the index instead
|
||||||
|
of blindly scanning magic strings. `0HDR` describes allocation/index bookkeeping
|
||||||
|
and can overlap the index region. It is preserved, not decoded as a game object.
|
||||||
|
`mCTy` records per-object metadata; interpretation remains incomplete.
|
||||||
|
|
||||||
|
Examples: `omt/level1.omt` index at `0x1ee333`; type counts are 47 `Canv`,
|
||||||
|
195 `3DSh`, 181 `3DMa`, one `mCTy`, one `0HDR`.
|
||||||
|
|
||||||
|
Object links serialize as BE16 boolean; if nonzero, fourcc + BE32 resource ID
|
||||||
|
follow. All booleans used by these serializers occupy **two** bytes.
|
||||||
|
|
||||||
|
## 3DSh / 3DSP version 2
|
||||||
|
|
||||||
|
All 2,155 supplied shape payloads use this exact version:
|
||||||
|
|
||||||
|
1. `3DSP`, BE32 version (2).
|
||||||
|
2. Eight BE32 floats (bounds/settings); additional legacy float + BE16 boolean.
|
||||||
|
3. BE32 vertex count; each vertex three BE32 IEEE-754 floats.
|
||||||
|
4. BE32 polygon count; variable polygon records below.
|
||||||
|
5. BE32 color count; each color four BE32 floats.
|
||||||
|
6. BE32 normal count; each normal three BE32 floats.
|
||||||
|
|
||||||
|
Polygon v2:
|
||||||
|
|
||||||
|
1. Two BE16 booleans, BE16 legacy field, BE32 ID, two BE32 floats.
|
||||||
|
2. BE16 legacy field, BE16 corner count.
|
||||||
|
3. Each corner: BE32 vertex index, float U, float V, BE32 normal index.
|
||||||
|
4. Three float polygon normal coordinates.
|
||||||
|
5. BE16 material-present boolean; if true, serialized object link.
|
||||||
|
|
||||||
|
Vertices preserve original scale and engine world coordinates. Do not normalize
|
||||||
|
individual scene shapes or reapply arbitrary transforms. Names can identify
|
||||||
|
already placed world objects. `3DMa` IDs are local to the source database.
|
||||||
|
Collision meaning of legacy polygon flags has not yet been recovered.
|
||||||
|
|
||||||
|
## 3DMa compact version 1
|
||||||
|
|
||||||
|
Every supplied material uses BE16 version 1, followed by BE16 legacy style 0..2,
|
||||||
|
a float scalar and BE16 flag. Styles 1/2 store four unsigned16 RGB triplets and
|
||||||
|
another boolean; style 0 stores one triplet. Style 2 adds a boolean, BE16 texture
|
||||||
|
presence enum and, for enum 1, an object link. Final BE16 field completes record.
|
||||||
|
Typical textured material length is 52 bytes; untextured variants can be 38/42.
|
||||||
|
RGB triplets are converted to floats by dividing by 65,535. The output keys
|
||||||
|
`emission`, `diffuse`, `specular`, `ambient` are inferred conventional roles;
|
||||||
|
legacy flags are retained with neutral names pending renderer validation.
|
||||||
|
|
||||||
|
## Canv / OmCv / OmGW
|
||||||
|
|
||||||
|
| Field | Representation |
|
||||||
|
|---|---|
|
||||||
|
| Signature | `OmCv` |
|
||||||
|
| Flags | BE32 |
|
||||||
|
| Pixel header | `OmGW` |
|
||||||
|
| Width, height, bit depth | BE16 each |
|
||||||
|
| Packing unit enum, color-key kind | byte each |
|
||||||
|
| Decoded row byte stride | BE16 |
|
||||||
|
| Has color key | BE16 boolean |
|
||||||
|
| Key | RGB16 triplet for key-kind 0; otherwise BE32 only when key present |
|
||||||
|
| Has palette | BE16 boolean |
|
||||||
|
| Optional palette | described below |
|
||||||
|
| Compressed data size | BE32 |
|
||||||
|
| Data | one BE16 encoded row length + encoded bytes per image row |
|
||||||
|
|
||||||
|
Packing enum 0/1/2 selects RLE units of 1/2/4 bytes. This is a PackBits variant:
|
||||||
|
control values are **little-endian** signed values of the unit width (the dword
|
||||||
|
variant tests low signed16 bits). Nonnegative N copies N+1 literal units; negative
|
||||||
|
N repeats the next unit 1-N times; -128 is a no-op. Encoded row must consume
|
||||||
|
exactly its advertised bytes and produce exactly the decoded stride.
|
||||||
|
|
||||||
|
Pixel values remain big-endian in encoded literal/repeat payloads. RGB555 selects
|
||||||
|
bits 10..14, 5..9, 0..4 and shifts each channel left three, matching original
|
||||||
|
conversion. The 32-bit format selects RGB at bits 16/8/0. Both derive alpha from
|
||||||
|
full stored-pixel equality with the color key; without a key alpha becomes 255.
|
||||||
|
|
||||||
|
Palette signature `OPa2` or `OmPa`, Pascal-name, BE32 color count, count RGB16
|
||||||
|
triplets. `OPa2` additionally stores a 4,096-byte palette accelerator. The original
|
||||||
|
converter takes low bytes of palette16 values after endian conversion. The twelve
|
||||||
|
indexed textures all use 8-bit pixels and decode with this layout.
|
||||||
|
|
||||||
|
## Wave resources
|
||||||
|
|
||||||
|
A `Wave` database entry contains RIFF/WAVE bytes either directly or after an OMT
|
||||||
|
object prefix. RIFF chunk sizes and WAVE audio format are little-endian. Extraction
|
||||||
|
finds the RIFF inside its indexed resource and preserves exactly `size+8` bytes;
|
||||||
|
it checks that the result is WAVE and never crosses the indexed resource boundary.
|
||||||
|
PCM is reusable by cross-platform audio. One MP3-in-WAVE is retained unchanged
|
||||||
|
and requires decoding support if that particular resource is played. Surrounding
|
||||||
|
OMT object metadata may contain playback settings; extracting a WAV does not
|
||||||
|
recover loop policy automatically.
|
||||||
|
|
||||||
|
## ASE source models
|
||||||
|
|
||||||
|
`ASE/*.ASE` and `ASE/jimstop.ase` are 3DS Max ASCII export files. Source object
|
||||||
|
positions are already exported vertex coordinates. Shared original OMT/ASE
|
||||||
|
objects prove the conversion `(x,y,z) -> (x,z,y)` and UV `(u,v) -> (u,-v)` while
|
||||||
|
retaining face winding. `MESH_ANIMATION` contains complete vertex samples tagged
|
||||||
|
with `TIMEVALUE`; scene FPS and ticks/frame establish source sample times.
|
||||||
|
|
||||||
|
Some morph-only files omit UVs and can have a different face table from the
|
||||||
|
original base shape. The original renderer keeps base topology and texture
|
||||||
|
coordinates while substituting morph vectors by vertex index. Use `--base`
|
||||||
|
explicitly rather than rendering missing UVs as `(0,0)`. First prepared player
|
||||||
|
base is `jimstop.ase`, normal runtime atlas is `png/jimycarl.png` (texture slot 0);
|
||||||
|
`png/jimyshock.png` is a separate effect in slot 1. See InitObject evidence.
|
||||||
|
|
||||||
|
## Prepared scene interchange
|
||||||
|
|
||||||
|
`scene.json` has `vertices:[[x,y,z]]`, `uvs:[[u,v]]`, `normals:[[x,y,z]]`,
|
||||||
|
`indices:[uint...]`, and `meshes:[{name,shapeId,firstIndex,indexCount,material}]`.
|
||||||
|
Vertices are expanded per polygon corner to retain discontinuous UVs and normals.
|
||||||
|
`materials` retain source IDs, texture links and numeric fields. `textures` give
|
||||||
|
ID/name/width/height and relative paths to tightly packed row-major RGBA8 bytes
|
||||||
|
and PNG preview. Source archive filename and SHA-256 establish provenance.
|
||||||
|
No coordinate or UV convention conversion is applied to already native OMT data.
|
||||||
|
|
||||||
|
`animation.json` records source hash, FPS, ticks/frame, frame JSON relative paths,
|
||||||
|
original TIMEVALUE, morph base provenance and optional original texture metadata.
|
||||||
|
Frame JSON uses the scene vertex/UV/index schema. Preview normals are currently
|
||||||
|
flat normals, explicitly marked in the manifest; original smooth shading is pending.
|
||||||
@@ -0,0 +1,146 @@
|
|||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import struct
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
import zlib
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
ORIGINAL = ROOT.parent / 'Original'
|
||||||
|
sys.path.insert(0, str(ROOT / 'tools' / 'extractors'))
|
||||||
|
from omt import Database, FormatError, Reader, export_scene, read_canvas, read_material, read_shape, unpack_row, validate_output, write_png
|
||||||
|
from ase import read_ase, export_ase
|
||||||
|
from png import read_png
|
||||||
|
|
||||||
|
|
||||||
|
class BoundedReaders(unittest.TestCase):
|
||||||
|
def test_read_bounds_and_negative_size(self):
|
||||||
|
for size in (-1, 2):
|
||||||
|
with self.assertRaises(FormatError): Reader(b'x').take(size)
|
||||||
|
|
||||||
|
def test_serialized_bool_is_two_bytes(self):
|
||||||
|
r = Reader(b'\x00\x01\x00\x00')
|
||||||
|
self.assertTrue(r.boolean())
|
||||||
|
self.assertFalse(r.boolean())
|
||||||
|
r.end()
|
||||||
|
|
||||||
|
def test_packbits_bytes_and_pixel_words(self):
|
||||||
|
self.assertEqual(unpack_row(b'\x02abc\xfeZ', 6, 1), b'abcZZZ')
|
||||||
|
self.assertEqual(unpack_row(b'\x01\x00\x7c\x00\x03\xe0\xfe\xff\x00\x1f', 10, 2),
|
||||||
|
b'\x7c\x00\x03\xe0\x00\x1f\x00\x1f\x00\x1f')
|
||||||
|
|
||||||
|
def test_packbits_rejects_overrun_truncation_and_leftovers(self):
|
||||||
|
for encoded, expected, unit in ((b'\x7fX', 1, 1), (b'\xff', 2, 1), (b'\x00XY', 1, 1)):
|
||||||
|
with self.assertRaises(FormatError): unpack_row(encoded, expected, unit)
|
||||||
|
|
||||||
|
def test_canvas_rgb555_matches_original_shift_and_key(self):
|
||||||
|
# Original pixel converter uses channel<<3, yielding 248 at full intensity.
|
||||||
|
row = b'\x02\x00\x7c\x00\x03\xe0\x00\x1f'
|
||||||
|
pixel_data = struct.pack('>H', len(row)) + row
|
||||||
|
header = b'OmCv'+struct.pack('>I', 0)+b'OmGW'+struct.pack('>HHHBBH', 3, 1, 16, 1, 1, 6)
|
||||||
|
header += struct.pack('>HIHI', 1, 0x03e0, 0, len(pixel_data))
|
||||||
|
canvas = read_canvas(header+pixel_data)
|
||||||
|
self.assertEqual(canvas['rgba'], bytes((248,0,0,255,0,248,0,0,0,0,248,255)))
|
||||||
|
|
||||||
|
def test_png_roundtrip_payload(self):
|
||||||
|
rgba = bytes((4,5,6,255,10,20,30,0))
|
||||||
|
with tempfile.TemporaryDirectory() as t:
|
||||||
|
path = Path(t)/'test.png'
|
||||||
|
write_png(path, 2, 1, rgba)
|
||||||
|
data = path.read_bytes()
|
||||||
|
self.assertEqual(data[:8], b'\x89PNG\r\n\x1a\n')
|
||||||
|
p, compressed = 8, bytearray()
|
||||||
|
while p < len(data):
|
||||||
|
size = struct.unpack_from('>I', data, p)[0]
|
||||||
|
tag, payload = data[p+4:p+8], data[p+8:p+8+size]
|
||||||
|
self.assertEqual(zlib.crc32(tag+payload), struct.unpack_from('>I',data,p+8+size)[0])
|
||||||
|
if tag == b'IDAT': compressed.extend(payload)
|
||||||
|
p += size+12
|
||||||
|
self.assertEqual(zlib.decompress(compressed), b'\x00'+rgba)
|
||||||
|
self.assertEqual(read_png(path)['rgba'],rgba)
|
||||||
|
|
||||||
|
|
||||||
|
@unittest.skipUnless(ORIGINAL.exists(), 'Original game files are local integration fixtures')
|
||||||
|
class OriginalResources(unittest.TestCase):
|
||||||
|
def test_every_database_index_is_bounded(self):
|
||||||
|
paths = list(ORIGINAL.rglob('*.omt'))
|
||||||
|
self.assertEqual(len(paths), 100)
|
||||||
|
self.assertEqual(sum(len(Database(p).entries) for p in paths), 7027)
|
||||||
|
|
||||||
|
def test_first_scene_geometry_materials_and_textures(self):
|
||||||
|
db = Database(ORIGINAL/'omt'/'level1b.omt')
|
||||||
|
counts = {}
|
||||||
|
for entry in db.entries:
|
||||||
|
if entry.type == '3DSh': read_shape(db.payload(entry))
|
||||||
|
elif entry.type == '3DMa': read_material(db.payload(entry))
|
||||||
|
elif entry.type == 'Canv':
|
||||||
|
canvas = read_canvas(db.payload(entry))
|
||||||
|
self.assertEqual(len(canvas['rgba']), canvas['width']*canvas['height']*4)
|
||||||
|
counts[entry.type] = counts.get(entry.type, 0)+1
|
||||||
|
self.assertEqual(counts['Canv'], 33)
|
||||||
|
|
||||||
|
def test_scene_export_contains_original_geometry_and_texture_links(self):
|
||||||
|
db = Database(ORIGINAL/'omt'/'VR01.omt')
|
||||||
|
with tempfile.TemporaryDirectory() as t:
|
||||||
|
output = Path(t)/'scene'
|
||||||
|
scene = export_scene(db, output)
|
||||||
|
self.assertEqual(len(scene['indices']), 240)
|
||||||
|
self.assertEqual(len(scene['vertices']), 240)
|
||||||
|
self.assertEqual(len(scene['textures']), 7)
|
||||||
|
self.assertTrue(all(0 <= i < len(scene['vertices']) for i in scene['indices']))
|
||||||
|
self.assertEqual([list(v) for v in scene['vertices']], json.loads((output/'scene.json').read_text())['vertices'])
|
||||||
|
for texture in scene['textures']:
|
||||||
|
self.assertEqual((output/texture['path']).stat().st_size, texture['width']*texture['height']*4)
|
||||||
|
|
||||||
|
def test_extraction_cannot_target_original_directory(self):
|
||||||
|
db = Database(ORIGINAL/'omt'/'VR01.omt')
|
||||||
|
for target in (ORIGINAL, ORIGINAL/'omt', ORIGINAL/'omt'/'converted'):
|
||||||
|
with self.assertRaises(ValueError): validate_output(db, target)
|
||||||
|
|
||||||
|
def test_indexed_texture_recovery(self):
|
||||||
|
db = Database(ORIGINAL/'omt'/'sprites.omt')
|
||||||
|
entry = next(e for e in db.entries if e.type=='Canv' and e.name=='shadow2')
|
||||||
|
canvas = read_canvas(db.payload(entry))
|
||||||
|
self.assertEqual((canvas['width'],canvas['height'],canvas['depth']), (64,32,8))
|
||||||
|
self.assertIn(0, canvas['rgba'][3::4])
|
||||||
|
|
||||||
|
def test_ase_coordinate_uv_and_winding_match_original_omt(self):
|
||||||
|
db = Database(ORIGINAL/'omt'/'objects.omt')
|
||||||
|
for name in ('gate1','yokianpart'):
|
||||||
|
entry = next(e for e in db.entries if e.type=='3DSh' and e.name==name)
|
||||||
|
omt = read_shape(db.payload(entry))
|
||||||
|
ase = read_ase(ORIGINAL/'ASE'/(name+'.ASE'))['objects'][0]['samples'][0]
|
||||||
|
self.assertEqual(len(omt['vertices']),len(ase['vertices']))
|
||||||
|
self.assertEqual(len(omt['polygons']),len(ase['faces']))
|
||||||
|
for original,recovered in zip(omt['vertices'],ase['vertices']):
|
||||||
|
for a,b in zip(original,recovered): self.assertAlmostEqual(a,b,places=3)
|
||||||
|
for polygon,face,tf in zip(omt['polygons'],ase['faces'],ase['textureFaces']):
|
||||||
|
self.assertEqual(tuple(c['vertex'] for c in polygon['corners']),face)
|
||||||
|
for corner,uv_index in zip(polygon['corners'],tf):
|
||||||
|
for a,b in zip(corner['uv'],ase['uvs'][uv_index]): self.assertAlmostEqual(a,b,places=6)
|
||||||
|
|
||||||
|
def test_jimmy_morph_animation_samples_and_texture(self):
|
||||||
|
for name,count in (('jimstop.ase',2),('jimrun.ASE',5),('jimjump.ASE',3)):
|
||||||
|
animation=read_ase(ORIGINAL/'ASE'/name)
|
||||||
|
self.assertEqual((animation['framesPerSecond'],animation['ticksPerFrame']),(10,480))
|
||||||
|
self.assertEqual(len(animation['objects'][0]['samples']),count)
|
||||||
|
self.assertTrue(all(len(m['faces'])==814 for m in animation['objects'][0]['samples']))
|
||||||
|
texture=read_png(ORIGINAL/'png'/'jimycarl.png')
|
||||||
|
self.assertEqual((texture['width'],texture['height'],len(texture['rgba'])),(256,256,256*256*4))
|
||||||
|
|
||||||
|
def test_morph_reuses_textured_base_topology(self):
|
||||||
|
with tempfile.TemporaryDirectory() as t:
|
||||||
|
output=Path(t)
|
||||||
|
with self.assertRaises(FormatError):
|
||||||
|
export_ase(ORIGINAL/'ASE'/'jimrun.ASE',output/'invalid')
|
||||||
|
export_ase(ORIGINAL/'ASE'/'jimrun.ASE',output/'run',ORIGINAL/'ASE'/'jimstop.ase')
|
||||||
|
export_ase(ORIGINAL/'ASE'/'jimstop.ase',output/'base')
|
||||||
|
frame=json.loads((output/'run'/'frame_0000.json').read_text())
|
||||||
|
base=json.loads((output/'base'/'frame_0000.json').read_text())
|
||||||
|
self.assertEqual(frame['uvs'],base['uvs'])
|
||||||
|
self.assertTrue(any(uv != [0,0] for uv in frame['uvs']))
|
||||||
|
self.assertNotEqual(frame['vertices'],base['vertices'])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__': unittest.main()
|
||||||
@@ -0,0 +1,140 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Read-only PE32 import/export/RTTI evidence inventory, no third-party modules.
|
||||||
|
|
||||||
|
Addresses are preferred-base VAs, not relocation-adjusted runtime addresses.
|
||||||
|
Usage: python3 tools/binary/pe_inspect.py ../Original/OMT2.dll --symbols 'render|timer'
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
import struct
|
||||||
|
|
||||||
|
|
||||||
|
class PE:
|
||||||
|
def __init__(self, path):
|
||||||
|
self.path = Path(path)
|
||||||
|
self.data = self.path.read_bytes()
|
||||||
|
if self.data[:2] != b'MZ':
|
||||||
|
raise ValueError('Missing MZ signature')
|
||||||
|
nt = self.u32(0x3c)
|
||||||
|
if self.data[nt:nt+4] != b'PE\0\0':
|
||||||
|
raise ValueError('Missing PE signature')
|
||||||
|
self.machine, count = struct.unpack_from('<HH', self.data, nt+4)
|
||||||
|
self.timestamp = self.u32(nt+8)
|
||||||
|
optsize = self.u16(nt+20)
|
||||||
|
opt = nt+24
|
||||||
|
if self.u16(opt) != 0x10b:
|
||||||
|
raise ValueError('This inspector supports PE32 only')
|
||||||
|
self.base = self.u32(opt+28)
|
||||||
|
self.entry = self.base+self.u32(opt+16)
|
||||||
|
self.header_size = self.u32(opt+60)
|
||||||
|
self.linker = list(self.data[opt+2:opt+4])
|
||||||
|
self.directories = [struct.unpack_from('<II', self.data, opt+96+8*i)
|
||||||
|
for i in range(min(16, self.u32(opt+92)))]
|
||||||
|
self.sections = []
|
||||||
|
for i in range(count):
|
||||||
|
off = opt+optsize+i*40
|
||||||
|
name = self.data[off:off+8].split(b'\0', 1)[0].decode('ascii')
|
||||||
|
vs, rva, size, raw = struct.unpack_from('<IIII', self.data, off+8)
|
||||||
|
self.sections.append(dict(name=name, virtual_size=vs, rva=rva,
|
||||||
|
raw_size=size, file_offset=raw))
|
||||||
|
|
||||||
|
def u16(self, off):
|
||||||
|
return struct.unpack_from('<H', self.data, off)[0]
|
||||||
|
|
||||||
|
def u32(self, off):
|
||||||
|
return struct.unpack_from('<I', self.data, off)[0]
|
||||||
|
|
||||||
|
def offset(self, rva):
|
||||||
|
if 0 <= rva < self.header_size:
|
||||||
|
return rva
|
||||||
|
for s in self.sections:
|
||||||
|
delta = rva-s['rva']
|
||||||
|
if 0 <= delta < s['raw_size']:
|
||||||
|
return s['file_offset']+delta
|
||||||
|
raise ValueError(f'RVA {rva:#x} has no file data')
|
||||||
|
|
||||||
|
def va(self, off):
|
||||||
|
if 0 <= off < self.header_size:
|
||||||
|
return self.base+off
|
||||||
|
for s in self.sections:
|
||||||
|
delta = off-s['file_offset']
|
||||||
|
if 0 <= delta < s['raw_size']:
|
||||||
|
return self.base+s['rva']+delta
|
||||||
|
raise ValueError(f'Offset {off:#x} has no virtual address')
|
||||||
|
|
||||||
|
def cstr(self, off):
|
||||||
|
end = self.data.index(0, off)
|
||||||
|
return self.data[off:end].decode('ascii', 'replace')
|
||||||
|
|
||||||
|
def exports(self):
|
||||||
|
rva, size = self.directories[0]
|
||||||
|
if not rva:
|
||||||
|
return []
|
||||||
|
off = self.offset(rva)
|
||||||
|
ordinal_base, count, names, functions, nameptrs, ordinals = struct.unpack_from(
|
||||||
|
'<IIIIII', self.data, off+16)
|
||||||
|
result = []
|
||||||
|
for i in range(names):
|
||||||
|
idx = self.u16(self.offset(ordinals)+2*i)
|
||||||
|
target = self.u32(self.offset(functions)+4*idx)
|
||||||
|
name = self.cstr(self.offset(self.u32(self.offset(nameptrs)+4*i)))
|
||||||
|
result.append(dict(name=name, ordinal=ordinal_base+idx,
|
||||||
|
va=f'0x{self.base+target:08x}', rva=target))
|
||||||
|
return result
|
||||||
|
|
||||||
|
def imports(self):
|
||||||
|
rva, _ = self.directories[1]
|
||||||
|
result = []
|
||||||
|
if not rva:
|
||||||
|
return result
|
||||||
|
off = self.offset(rva)
|
||||||
|
while any(self.data[off:off+20]):
|
||||||
|
original, _, _, name, first = struct.unpack_from('<IIIII', self.data, off)
|
||||||
|
dll = self.cstr(self.offset(name))
|
||||||
|
thunk = self.offset(original or first)
|
||||||
|
i = 0
|
||||||
|
while (v := self.u32(thunk+4*i)):
|
||||||
|
n = f'ordinal:{v & 0xffff}' if v & 0x80000000 else self.cstr(self.offset(v)+2)
|
||||||
|
result.append(dict(dll=dll, name=n, iat=f'0x{self.base+first+4*i:08x}'))
|
||||||
|
i += 1
|
||||||
|
off += 20
|
||||||
|
return result
|
||||||
|
|
||||||
|
def strings(self, pattern):
|
||||||
|
result = []
|
||||||
|
regex = re.compile(pattern, re.I)
|
||||||
|
for m in re.finditer(rb'[\x20-\x7e]{4,}', self.data):
|
||||||
|
value = m.group().decode('ascii')
|
||||||
|
if regex.search(value):
|
||||||
|
result.append(dict(value=value, offset=f'0x{m.start():x}',
|
||||||
|
va=f'0x{self.va(m.start()):08x}'))
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument('binary', type=Path)
|
||||||
|
parser.add_argument('--symbols', help='Regex filter for imports and exports')
|
||||||
|
parser.add_argument('--strings', help='Regex filter for ASCII strings with VAs')
|
||||||
|
args = parser.parse_args()
|
||||||
|
pe = PE(args.binary)
|
||||||
|
imports, exports = pe.imports(), pe.exports()
|
||||||
|
if args.symbols:
|
||||||
|
regex = re.compile(args.symbols, re.I)
|
||||||
|
imports = [x for x in imports if regex.search(x['name'])]
|
||||||
|
exports = [x for x in exports if regex.search(x['name'])]
|
||||||
|
out = dict(path=str(args.binary), sha256=hashlib.sha256(pe.data).hexdigest(),
|
||||||
|
machine=hex(pe.machine), linker=pe.linker, timestamp=pe.timestamp,
|
||||||
|
image_base=hex(pe.base), entry_point=hex(pe.entry),
|
||||||
|
clr_directory=pe.directories[14], sections=pe.sections,
|
||||||
|
imports=imports, exports=exports)
|
||||||
|
if args.strings:
|
||||||
|
out['strings'] = pe.strings(args.strings)
|
||||||
|
print(json.dumps(out, indent=2))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Find MSVC x86 RTTI complete-object locators and candidate vtables.
|
||||||
|
|
||||||
|
Output is evidence, not a claim that every candidate is an actual vtable.
|
||||||
|
Usage: python3 tools/binary/rtti_map.py ../Original/Neutron.exe C3DJimmy
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import struct
|
||||||
|
from pe_inspect import PE
|
||||||
|
|
||||||
|
|
||||||
|
def rtti_map(pe, class_name):
|
||||||
|
marker = ('.?AV'+class_name+'@@').encode('ascii')
|
||||||
|
at = pe.data.find(marker)
|
||||||
|
if at < 8:
|
||||||
|
raise ValueError(f'No type descriptor for {class_name}')
|
||||||
|
type_va = pe.va(at-8)
|
||||||
|
result = dict(class_name=class_name, type_descriptor=hex(type_va), vtables=[])
|
||||||
|
for m in re.finditer(re.escape(struct.pack('<I', type_va)), pe.data):
|
||||||
|
off = m.start()-12
|
||||||
|
if off < 0 or off % 4:
|
||||||
|
continue
|
||||||
|
sig, displacement, ctor_displacement, _, hierarchy = struct.unpack_from('<IIIII', pe.data, off)
|
||||||
|
if sig != 0 or displacement > 0x100000:
|
||||||
|
continue
|
||||||
|
locator_va = pe.va(off)
|
||||||
|
for ref in re.finditer(re.escape(struct.pack('<I', locator_va)), pe.data):
|
||||||
|
if ref.start() % 4:
|
||||||
|
continue
|
||||||
|
methods = []
|
||||||
|
for i in range(512):
|
||||||
|
pos = ref.start()+4+i*4
|
||||||
|
if pos+4 > len(pe.data):
|
||||||
|
break
|
||||||
|
va = pe.u32(pos)
|
||||||
|
try:
|
||||||
|
function_off = pe.offset(va-pe.base)
|
||||||
|
section = next(s for s in pe.sections if s['file_offset'] <= function_off < s['file_offset']+s['raw_size'])
|
||||||
|
if section['name'] != '.text':
|
||||||
|
break
|
||||||
|
except (ValueError, StopIteration):
|
||||||
|
break
|
||||||
|
methods.append(dict(slot=hex(i*4), va=hex(va)))
|
||||||
|
if methods:
|
||||||
|
result['vtables'].append(dict(va=hex(pe.va(ref.start()+4)),
|
||||||
|
complete_object_locator=hex(locator_va), object_displacement=hex(displacement),
|
||||||
|
hierarchy=hex(hierarchy), methods=methods))
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument('binary')
|
||||||
|
parser.add_argument('class_name')
|
||||||
|
args = parser.parse_args()
|
||||||
|
print(json.dumps(rtti_map(PE(args.binary), args.class_name), indent=2))
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Recover supplied 3DS Max ASE mesh animation samples into OMT coordinates."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
|
||||||
|
from omt import FormatError
|
||||||
|
|
||||||
|
NUMBER = r'[-+\d.eE]+'
|
||||||
|
|
||||||
|
|
||||||
|
def blocks(text: str, tag: str):
|
||||||
|
for match in re.finditer(r'\*'+re.escape(tag)+r'\s*\{', text):
|
||||||
|
start, depth = match.end(), 1
|
||||||
|
cursor = start
|
||||||
|
while cursor < len(text) and depth:
|
||||||
|
if text[cursor] == '{': depth += 1
|
||||||
|
elif text[cursor] == '}': depth -= 1
|
||||||
|
cursor += 1
|
||||||
|
if depth: raise FormatError(f'Unclosed {tag} block')
|
||||||
|
yield match.start(), text[start:cursor-1]
|
||||||
|
|
||||||
|
|
||||||
|
def vector_table(text: str, tag: str):
|
||||||
|
values = {}
|
||||||
|
pattern = r'\*'+tag+r'\s+(\d+)\s+('+NUMBER+r')\s+('+NUMBER+r')\s+('+NUMBER+r')'
|
||||||
|
for match in re.finditer(pattern, text):
|
||||||
|
index = int(match[1])
|
||||||
|
if index in values: raise FormatError(f'Duplicate {tag} index {index}')
|
||||||
|
values[index] = tuple(float(match[i]) for i in (2,3,4))
|
||||||
|
if set(values) != set(range(len(values))): raise FormatError(f'Noncontiguous {tag} indices')
|
||||||
|
if any(not math.isfinite(v) for p in values.values() for v in p): raise FormatError('Nonfinite ASE value')
|
||||||
|
return [values[i] for i in range(len(values))]
|
||||||
|
|
||||||
|
|
||||||
|
def read_mesh(text: str) -> dict:
|
||||||
|
vertices = vector_table(text, 'MESH_VERTEX')
|
||||||
|
tvertices = vector_table(text, 'MESH_TVERT')
|
||||||
|
faces, tfaces = {}, {}
|
||||||
|
for match in re.finditer(r'\*MESH_FACE\s+(\d+):\s+A:\s*(\d+)\s+B:\s*(\d+)\s+C:\s*(\d+)', text):
|
||||||
|
faces[int(match[1])] = tuple(int(match[i]) for i in (2,3,4))
|
||||||
|
for match in re.finditer(r'\*MESH_TFACE\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)', text):
|
||||||
|
tfaces[int(match[1])] = tuple(int(match[i]) for i in (2,3,4))
|
||||||
|
for key, count in (('MESH_NUMVERTEX',len(vertices)),('MESH_NUMFACES',len(faces))):
|
||||||
|
match = re.search(r'\*'+key+r'\s+(\d+)', text)
|
||||||
|
if not match or int(match[1]) != count: raise FormatError(f'{key} does not match data')
|
||||||
|
if set(faces) != set(range(len(faces))): raise FormatError('Noncontiguous face indices')
|
||||||
|
if tfaces and set(tfaces) != set(faces): raise FormatError('Texture faces do not match faces')
|
||||||
|
if any(i>=len(vertices) for face in faces.values() for i in face): raise FormatError('ASE vertex outside table')
|
||||||
|
if any(i>=len(tvertices) for face in tfaces.values() for i in face): raise FormatError('ASE UV outside table')
|
||||||
|
time = re.search(r'\*TIMEVALUE\s+(-?\d+)',text)
|
||||||
|
return {'time': int(time[1]) if time else 0,
|
||||||
|
'vertices': [(x,z,y) for x,y,z in vertices],
|
||||||
|
'uvs': [(u,-v) for u,v,_ in tvertices],
|
||||||
|
'faces': [faces[i] for i in range(len(faces))],
|
||||||
|
'textureFaces': [tfaces[i] for i in range(len(tfaces))] if tfaces else []}
|
||||||
|
|
||||||
|
|
||||||
|
def read_ase(path: Path | str) -> dict:
|
||||||
|
path = Path(path)
|
||||||
|
data = path.read_bytes()
|
||||||
|
text = data.decode('latin1')
|
||||||
|
if not text.lstrip().startswith('*3DSMAX_ASCIIEXPORT'): raise FormatError('Expected ASE signature')
|
||||||
|
objects = []
|
||||||
|
for _, geom in blocks(text, 'GEOMOBJECT'):
|
||||||
|
name = re.search(r'\*NODE_NAME\s+"([^"]+)"', geom)
|
||||||
|
meshes = []
|
||||||
|
animations = list(blocks(geom, 'MESH_ANIMATION'))
|
||||||
|
if animations:
|
||||||
|
for _, animation in animations: meshes.extend(read_mesh(body) for _,body in blocks(animation,'MESH'))
|
||||||
|
else: meshes = [read_mesh(body) for _,body in blocks(geom,'MESH')]
|
||||||
|
objects.append({'name': name[1] if name else '', 'samples': meshes})
|
||||||
|
if not objects: raise FormatError('No geometry in ASE')
|
||||||
|
def setting(name):
|
||||||
|
match = re.search(r'\*'+name+r'\s+(\d+)',text)
|
||||||
|
if not match: raise FormatError(f'Missing ASE scene setting {name}')
|
||||||
|
return int(match[1])
|
||||||
|
return {'source':path.name, 'sha256':hashlib.sha256(data).hexdigest(),
|
||||||
|
'framesPerSecond':setting('SCENE_FRAMESPEED'),
|
||||||
|
'ticksPerFrame':setting('SCENE_TICKSPERFRAME'), 'objects':objects,
|
||||||
|
'coordinates':'(ASE x,z,y), texture (u,-v); matched OMT duplicate assets'}
|
||||||
|
|
||||||
|
|
||||||
|
def expand_mesh(mesh: dict) -> dict:
|
||||||
|
result = {'vertices':[], 'uvs':[], 'indices':[], 'normals':[]}
|
||||||
|
for face_index, face in enumerate(mesh['faces']):
|
||||||
|
a,b,c = [mesh['vertices'][i] for i in face]
|
||||||
|
ab,ac = [b[i]-a[i] for i in range(3)], [c[i]-a[i] for i in range(3)]
|
||||||
|
normal = [ab[1]*ac[2]-ab[2]*ac[1], ab[2]*ac[0]-ab[0]*ac[2], ab[0]*ac[1]-ab[1]*ac[0]]
|
||||||
|
length = math.sqrt(sum(v*v for v in normal))
|
||||||
|
normal = [v/length for v in normal] if length else [0.,0.,0.]
|
||||||
|
for corner,index in enumerate(face):
|
||||||
|
result['indices'].append(len(result['vertices']))
|
||||||
|
result['vertices'].append(mesh['vertices'][index])
|
||||||
|
result['uvs'].append(mesh['uvs'][mesh['textureFaces'][face_index][corner]] if mesh['textureFaces'] else [0.,0.])
|
||||||
|
result['normals'].append(normal)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def export_ase(path: Path, output: Path, base_path: Path | None = None):
|
||||||
|
if output.resolve().is_relative_to(path.parent.resolve()) or path.resolve().is_relative_to(output.resolve()):
|
||||||
|
raise ValueError('Output must be separate from original inputs')
|
||||||
|
animation = read_ase(path)
|
||||||
|
base_animation = read_ase(base_path) if base_path else None
|
||||||
|
if base_animation and len(base_animation['objects']) != len(animation['objects']):
|
||||||
|
raise FormatError('Morph base and animation object counts differ')
|
||||||
|
output.mkdir(parents=True, exist_ok=True)
|
||||||
|
frames = []
|
||||||
|
for object_index,item in enumerate(animation['objects']):
|
||||||
|
for sample in item['samples']:
|
||||||
|
if base_animation:
|
||||||
|
base = base_animation['objects'][object_index]['samples'][0]
|
||||||
|
if len(base['vertices']) != len(sample['vertices']):
|
||||||
|
raise FormatError('Morph sample vertex count differs from base shape')
|
||||||
|
# Original OMedia3DMorphAnim supplies only per-index vectors to a
|
||||||
|
# base shape; its polygon/UV topology remains unchanged.
|
||||||
|
sample = {**sample, 'faces':base['faces'], 'textureFaces':base['textureFaces'],
|
||||||
|
'uvs':base['uvs']}
|
||||||
|
if not sample['textureFaces']:
|
||||||
|
raise FormatError('Mesh has no UVs; supply --base with the original textured morph base')
|
||||||
|
filename = f"frame_{len(frames):04d}.json"
|
||||||
|
mesh = expand_mesh(sample)
|
||||||
|
mesh['meshes'] = [{'name': item['name'],'firstIndex':0,'indexCount':len(mesh['indices']),'material':-1}]
|
||||||
|
(output/filename).write_text(json.dumps(mesh,separators=(',',':'))+'\n', encoding='utf-8')
|
||||||
|
frames.append({'node':item['name'],'time':sample['time'], 'path':filename})
|
||||||
|
manifest = {key:value for key,value in animation.items() if key != 'objects'}
|
||||||
|
manifest['frames'] = frames
|
||||||
|
if base_animation:
|
||||||
|
manifest['morphBase'] = {'source':base_animation['source'], 'sha256':base_animation['sha256'],
|
||||||
|
'mode':'Original base topology and UV, sample positions by vertex index'}
|
||||||
|
manifest['normalMode'] = 'Flat normals reconstructed from geometry; original smoothing not yet replicated'
|
||||||
|
(output/'animation.json').write_text(json.dumps(manifest,indent=2)+'\n', encoding='utf-8')
|
||||||
|
return manifest
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
p = argparse.ArgumentParser(description=__doc__)
|
||||||
|
p.add_argument('ase',type=Path)
|
||||||
|
p.add_argument('--output',type=Path,required=True)
|
||||||
|
p.add_argument('--base',type=Path,help='Original textured shape whose topology a morph retains')
|
||||||
|
args = p.parse_args()
|
||||||
|
manifest = export_ase(args.ase,args.output,args.base)
|
||||||
|
print(f"{manifest['source']}: {len(manifest['frames'])} samples at {manifest['framesPerSecond']} fps")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__': main()
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Validate every original OMT index, mesh, material, canvas and RIFF boundary."""
|
||||||
|
import argparse
|
||||||
|
from collections import Counter
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import struct
|
||||||
|
|
||||||
|
from omt import Database, FormatError, read_canvas, read_material, read_shape
|
||||||
|
|
||||||
|
|
||||||
|
def audit(original: Path) -> dict:
|
||||||
|
counts, conversions, failures = Counter(), Counter(), []
|
||||||
|
archives = sorted(original.rglob('*.omt'))
|
||||||
|
for path in archives:
|
||||||
|
try:
|
||||||
|
db = Database(path)
|
||||||
|
except (OSError, FormatError) as ex:
|
||||||
|
failures.append({'source': str(path), 'error': str(ex)})
|
||||||
|
continue
|
||||||
|
for entry in db.entries:
|
||||||
|
counts[entry.type] += 1
|
||||||
|
function = {'3DSh': read_shape, '3DMa': read_material, 'Canv': read_canvas}.get(entry.type)
|
||||||
|
try:
|
||||||
|
payload = db.payload(entry)
|
||||||
|
if function:
|
||||||
|
function(payload)
|
||||||
|
conversions[entry.type] += 1
|
||||||
|
elif entry.type == 'Wave':
|
||||||
|
start = payload.find(b'RIFF')
|
||||||
|
if start < 0 or payload[start+8:start+12] != b'WAVE':
|
||||||
|
raise FormatError('No RIFF WAVE header')
|
||||||
|
size = struct.unpack_from('<I', payload, start+4)[0]+8
|
||||||
|
if size < 12 or start+size > len(payload): raise FormatError('RIFF size exceeds resource')
|
||||||
|
conversions['Wave'] += 1
|
||||||
|
except (FormatError, struct.error) as ex:
|
||||||
|
failures.append({'source': str(path), 'type': entry.type,
|
||||||
|
'id': entry.id, 'name': entry.name, 'error': str(ex)})
|
||||||
|
return {'archives': len(archives), 'counts': dict(counts),
|
||||||
|
'decoded': dict(conversions), 'failures': failures}
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument('original', type=Path)
|
||||||
|
parser.add_argument('--report', type=Path)
|
||||||
|
args = parser.parse_args()
|
||||||
|
result = audit(args.original)
|
||||||
|
if args.report:
|
||||||
|
args.report.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
args.report.write_text(json.dumps(result, indent=2)+'\n', encoding='utf-8')
|
||||||
|
print(json.dumps(result, indent=2))
|
||||||
|
raise SystemExit(bool(result['failures']))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__': main()
|
||||||
@@ -0,0 +1,390 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Read-only Open Media Toolkit 0MF2 database recovery. No third-party modules."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from dataclasses import asdict, dataclass
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
import struct
|
||||||
|
import wave
|
||||||
|
import zlib
|
||||||
|
|
||||||
|
|
||||||
|
class FormatError(ValueError):
|
||||||
|
"""Malformed or unsupported original resource."""
|
||||||
|
|
||||||
|
|
||||||
|
class Reader:
|
||||||
|
def __init__(self, data: bytes, position: int = 0):
|
||||||
|
self.data = data
|
||||||
|
self.position = position
|
||||||
|
|
||||||
|
def take(self, size: int) -> bytes:
|
||||||
|
if size < 0 or self.position < 0 or self.position + size > len(self.data):
|
||||||
|
raise FormatError(f"Read outside resource at 0x{self.position:x}, size {size}")
|
||||||
|
result = self.data[self.position:self.position + size]
|
||||||
|
self.position += size
|
||||||
|
return result
|
||||||
|
|
||||||
|
def unpack(self, fmt: str):
|
||||||
|
return struct.unpack(fmt, self.take(struct.calcsize(fmt)))
|
||||||
|
|
||||||
|
def u8(self): return self.unpack('>B')[0]
|
||||||
|
def u16(self): return self.unpack('>H')[0]
|
||||||
|
def u32(self): return self.unpack('>I')[0]
|
||||||
|
def f32(self): return self.unpack('>f')[0]
|
||||||
|
def boolean(self): return self.u16() != 0
|
||||||
|
def vec3(self): return self.unpack('>3f')
|
||||||
|
def string(self): return self.take(self.u8()).decode('latin1')
|
||||||
|
|
||||||
|
def count(self, minimum_bytes: int):
|
||||||
|
count = self.u32()
|
||||||
|
if count > (len(self.data) - self.position) // minimum_bytes:
|
||||||
|
raise FormatError(f"Impossible count {count} at 0x{self.position - 4:x}")
|
||||||
|
return count
|
||||||
|
|
||||||
|
def link(self):
|
||||||
|
if not self.boolean(): return None
|
||||||
|
return {'type': self.take(4).decode('latin1'), 'id': self.u32()}
|
||||||
|
|
||||||
|
def end(self):
|
||||||
|
if self.position != len(self.data):
|
||||||
|
raise FormatError(f"Unconsumed resource bytes: {len(self.data)-self.position}")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Entry:
|
||||||
|
type: str
|
||||||
|
id: int
|
||||||
|
offset: int
|
||||||
|
size: int
|
||||||
|
name: str
|
||||||
|
|
||||||
|
|
||||||
|
class Database:
|
||||||
|
def __init__(self, path: Path | str):
|
||||||
|
self.path = Path(path)
|
||||||
|
self.data = self.path.read_bytes()
|
||||||
|
reader = Reader(self.data)
|
||||||
|
if reader.take(4) != b'0MF2':
|
||||||
|
raise FormatError('Expected 0MF2 database signature')
|
||||||
|
self.table_offset = reader.u32()
|
||||||
|
if not 8 <= self.table_offset < len(self.data):
|
||||||
|
raise FormatError('Database table offset is outside file')
|
||||||
|
reader.position = self.table_offset
|
||||||
|
groups = reader.count(8)
|
||||||
|
self.entries = []
|
||||||
|
keys = set()
|
||||||
|
for _ in range(groups):
|
||||||
|
tag = reader.take(4).decode('latin1')
|
||||||
|
for _ in range(reader.count(13)):
|
||||||
|
identity, offset, size = reader.unpack('>III')
|
||||||
|
name = reader.string()
|
||||||
|
if offset < 8 or offset + size > len(self.data):
|
||||||
|
raise FormatError(f'Resource {tag}:{identity} outside file')
|
||||||
|
if (tag, identity) in keys:
|
||||||
|
raise FormatError(f'Duplicate resource {tag}:{identity}')
|
||||||
|
keys.add((tag, identity))
|
||||||
|
self.entries.append(Entry(tag, identity, offset, size, name))
|
||||||
|
self.table_end = reader.position
|
||||||
|
|
||||||
|
def payload(self, entry: Entry) -> bytes:
|
||||||
|
return self.data[entry.offset:entry.offset + entry.size]
|
||||||
|
|
||||||
|
|
||||||
|
def read_shape(data: bytes) -> dict:
|
||||||
|
"""Decode 3DSP versions 0..4 using OMT2.dll read_class evidence."""
|
||||||
|
r = Reader(data)
|
||||||
|
if r.take(4) != b'3DSP': raise FormatError('Expected 3DSP shape')
|
||||||
|
version = r.u32()
|
||||||
|
if version > 4: raise FormatError(f'Unsupported 3DSP version {version}')
|
||||||
|
bounds = r.unpack('>8f')
|
||||||
|
legacy = {'unused': r.f32(), 'flag': r.boolean()} if 0 < version < 3 else None
|
||||||
|
vertices = [r.vec3() for _ in range(r.count(12))]
|
||||||
|
polygons = []
|
||||||
|
for _ in range(r.count(1)):
|
||||||
|
if version <= 2:
|
||||||
|
flags = {'first': r.boolean(), 'second': r.boolean(), 'legacy': r.u16()}
|
||||||
|
identifier = r.u32()
|
||||||
|
extra = r.unpack('>2f') if version else None
|
||||||
|
legacy_flag = r.u16()
|
||||||
|
corners = r.u16()
|
||||||
|
if corners > (len(data)-r.position)//12: raise FormatError('Impossible polygon corner count')
|
||||||
|
points = []
|
||||||
|
for _ in range(corners):
|
||||||
|
index = r.u32()
|
||||||
|
uv = r.unpack('>2f')
|
||||||
|
normal_index = r.u32() if version > 1 else index
|
||||||
|
if index >= len(vertices): raise FormatError(f'Invalid vertex index {index}')
|
||||||
|
points.append({'vertex': index, 'uv': uv, 'normal': normal_index})
|
||||||
|
normal = r.vec3()
|
||||||
|
# Old polygon format has an explicit material-present bool around link.
|
||||||
|
material = r.link() if r.boolean() else None
|
||||||
|
polygons.append({'flags': flags, 'id': identifier, 'extra': extra,
|
||||||
|
'legacy_flag': legacy_flag, 'corners': points,
|
||||||
|
'normal': normal, 'material': material})
|
||||||
|
else:
|
||||||
|
flags = r.u16()
|
||||||
|
identifier = r.u32()
|
||||||
|
points = []
|
||||||
|
for _ in range(r.count(20)):
|
||||||
|
index, normal_index, colour_index = r.unpack('>3I')
|
||||||
|
uv = r.unpack('>2f')
|
||||||
|
if index >= len(vertices): raise FormatError(f'Invalid vertex index {index}')
|
||||||
|
points.append({'vertex': index, 'normal': normal_index,
|
||||||
|
'colour': colour_index, 'uv': uv})
|
||||||
|
polygons.append({'flags': flags, 'id': identifier, 'corners': points,
|
||||||
|
'normal': r.vec3(), 'material': r.link()})
|
||||||
|
colours = [r.unpack('>4f') for _ in range(r.count(16))]
|
||||||
|
normals = [r.vec3() for _ in range(r.count(12))]
|
||||||
|
additional = [r.unpack('>4f') for _ in range(r.count(16))] if version > 2 else []
|
||||||
|
r.end()
|
||||||
|
if not all(math.isfinite(v) for point in vertices for v in point):
|
||||||
|
raise FormatError('Nonfinite shape coordinate')
|
||||||
|
return {'version': version, 'bounds': bounds, 'legacy': legacy, 'vertices': vertices,
|
||||||
|
'polygons': polygons, 'colours': colours, 'normals': normals,
|
||||||
|
'additional': additional}
|
||||||
|
|
||||||
|
|
||||||
|
def safe_name(value: str) -> str:
|
||||||
|
return re.sub(r'[^a-zA-Z0-9._-]+', '_', value).strip('._')[:100] or 'unnamed'
|
||||||
|
|
||||||
|
|
||||||
|
def read_material(data: bytes) -> dict:
|
||||||
|
r = Reader(data)
|
||||||
|
version = r.u16()
|
||||||
|
if version != 1: raise FormatError(f'Unsupported material version {version}')
|
||||||
|
style = r.u16()
|
||||||
|
if style > 2: raise FormatError(f'Unsupported material style {style}')
|
||||||
|
shininess, legacy_flag = r.f32(), r.boolean()
|
||||||
|
colours = {}
|
||||||
|
if style == 0:
|
||||||
|
colours['diffuse'] = [v/65535 for v in r.unpack('>3H')]
|
||||||
|
else:
|
||||||
|
for name in ('emission', 'diffuse', 'specular', 'ambient'):
|
||||||
|
colours[name] = [v/65535 for v in r.unpack('>3H')]
|
||||||
|
legacy_boolean = r.boolean()
|
||||||
|
texture, texture_override_flag = None, False
|
||||||
|
if style == 2:
|
||||||
|
texture_override_flag = r.boolean()
|
||||||
|
if r.u16() == 1: texture = r.link()
|
||||||
|
trailing = r.u16()
|
||||||
|
r.end()
|
||||||
|
return {'version': version, 'style': style, 'shininess': shininess,
|
||||||
|
'legacyFlag': legacy_flag, 'colours': colours,
|
||||||
|
'legacyBoolean': legacy_boolean if style else False, 'textureOverrideFlag': texture_override_flag,
|
||||||
|
'texture': texture, 'trailing': trailing}
|
||||||
|
|
||||||
|
|
||||||
|
def unpack_row(encoded: bytes, expected_size: int, unit: int) -> bytes:
|
||||||
|
"""OMT PackBits: LE signed control of unit width, payload in source pixel order."""
|
||||||
|
r = Reader(encoded)
|
||||||
|
output = bytearray()
|
||||||
|
while len(output) < expected_size:
|
||||||
|
control_bytes = r.take(unit)
|
||||||
|
# 32-bit variant tests the low signed 16 bits, as original routine does.
|
||||||
|
control = int.from_bytes(control_bytes[:min(unit, 2)], 'little', signed=True)
|
||||||
|
if control == -128: continue
|
||||||
|
count = control + 1 if control >= 0 else 1 - control
|
||||||
|
if len(output) + count*unit > expected_size:
|
||||||
|
raise FormatError('RLE run exceeds decoded row')
|
||||||
|
output.extend(r.take(count*unit) if control >= 0 else r.take(unit)*count)
|
||||||
|
r.end()
|
||||||
|
return bytes(output)
|
||||||
|
|
||||||
|
|
||||||
|
def read_canvas(data: bytes) -> dict:
|
||||||
|
r = Reader(data)
|
||||||
|
magic = r.take(4)
|
||||||
|
if magic != b'OmCv': raise FormatError(f'Unsupported canvas signature {magic!r}')
|
||||||
|
flags = r.u32()
|
||||||
|
if r.take(4) != b'OmGW': raise FormatError('Missing OmGW pixel header')
|
||||||
|
width, height, depth = r.unpack('>3H')
|
||||||
|
packing, key_type, row_bytes = r.u8(), r.u8(), r.u16()
|
||||||
|
if not width or not height or width*height > 16777216: raise FormatError('Invalid canvas dimensions')
|
||||||
|
if packing not in (0, 1, 2): raise FormatError('Unknown canvas packing')
|
||||||
|
if depth not in (8, 16, 32): raise FormatError(f'Unsupported canvas depth {depth}')
|
||||||
|
has_key = r.boolean()
|
||||||
|
key = None
|
||||||
|
if key_type == 0:
|
||||||
|
rgb = r.unpack('>3H')
|
||||||
|
if has_key:
|
||||||
|
channels = [v*(31 if depth == 16 else 255)//65535 for v in rgb]
|
||||||
|
key = ((channels[0]<<10)|(channels[1]<<5)|channels[2]) if depth == 16 else ((channels[0]<<16)|(channels[1]<<8)|channels[2])
|
||||||
|
elif has_key: key = r.u32()
|
||||||
|
palette = None
|
||||||
|
if r.boolean():
|
||||||
|
signature = r.take(4)
|
||||||
|
if signature not in (b'OPa2', b'OmPa'): raise FormatError('Invalid canvas palette signature')
|
||||||
|
r.string() # Palette name; absent for these originals.
|
||||||
|
count = r.u32()
|
||||||
|
if count > 256: raise FormatError('Palette exceeds 256 colours')
|
||||||
|
palette = [tuple(v & 255 for v in r.unpack('>3H')) for _ in range(count)]
|
||||||
|
if signature == b'OPa2': r.take(4096) # Legacy palette matching accelerator.
|
||||||
|
if count <= 2: palette = [(255,255,255), (0,0,0)]
|
||||||
|
if depth == 8 and palette is None: raise FormatError('Indexed canvas has no palette')
|
||||||
|
size = r.u32()
|
||||||
|
compressed = Reader(r.take(size))
|
||||||
|
r.end()
|
||||||
|
if row_bytes < width*(depth//8): raise FormatError('Canvas row stride too small')
|
||||||
|
pixels = bytearray()
|
||||||
|
for _ in range(height):
|
||||||
|
row = unpack_row(compressed.take(compressed.u16()), row_bytes, 1 << packing)
|
||||||
|
for x in range(width):
|
||||||
|
v = int.from_bytes(row[x*(depth//8):(x+1)*(depth//8)], 'big')
|
||||||
|
if depth == 8:
|
||||||
|
if v >= len(palette): raise FormatError('Pixel index outside palette')
|
||||||
|
rgb = palette[v]
|
||||||
|
elif depth == 16:
|
||||||
|
rgb = (((v>>10)&31)<<3, ((v>>5)&31)<<3, (v&31)<<3)
|
||||||
|
else: rgb = ((v>>16)&255, (v>>8)&255, v&255)
|
||||||
|
pixels.extend((*rgb, 0 if key is not None and v == key else 255))
|
||||||
|
compressed.end()
|
||||||
|
return {'width': width, 'height': height, 'depth': depth, 'flags': flags,
|
||||||
|
'packing': packing, 'rowBytes': row_bytes, 'colourKey': key,
|
||||||
|
'rgba': bytes(pixels)}
|
||||||
|
|
||||||
|
|
||||||
|
def write_png(path: Path, width: int, height: int, rgba: bytes):
|
||||||
|
if len(rgba) != width*height*4: raise ValueError('Incorrect RGBA byte count')
|
||||||
|
def chunk(tag, payload):
|
||||||
|
return struct.pack('>I', len(payload)) + tag + payload + struct.pack('>I', zlib.crc32(tag+payload))
|
||||||
|
scanlines = b''.join(b'\0' + rgba[y*width*4:(y+1)*width*4] for y in range(height))
|
||||||
|
path.write_bytes(b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', struct.pack('>IIBBBBB', width, height, 8, 6, 0, 0, 0))
|
||||||
|
+ chunk(b'IDAT', zlib.compress(scanlines)) + chunk(b'IEND', b''))
|
||||||
|
|
||||||
|
|
||||||
|
def export_scene(database: Database, output: Path):
|
||||||
|
"""Expanded original corner data; triangle fans preserve source winding."""
|
||||||
|
validate_output(database, output)
|
||||||
|
output.mkdir(parents=True, exist_ok=True)
|
||||||
|
scene = {'source': database.path.name, 'sha256': hashlib.sha256(database.data).hexdigest(),
|
||||||
|
'coordinates': 'Original OMT world coordinates, unchanged',
|
||||||
|
'vertices': [], 'uvs': [], 'normals': [], 'indices': [], 'meshes': [],
|
||||||
|
'materials': [], 'textures': [], 'warnings': []}
|
||||||
|
for entry in database.entries:
|
||||||
|
payload = database.payload(entry)
|
||||||
|
if entry.type == '3DSh':
|
||||||
|
shape = read_shape(payload)
|
||||||
|
for poly in shape['polygons']:
|
||||||
|
first_index = len(scene['indices'])
|
||||||
|
first_vertex = len(scene['vertices'])
|
||||||
|
for corner in poly['corners']:
|
||||||
|
scene['vertices'].append(shape['vertices'][corner['vertex']])
|
||||||
|
scene['uvs'].append(corner['uv'])
|
||||||
|
ni = corner['normal']
|
||||||
|
scene['normals'].append(shape['normals'][ni] if ni < len(shape['normals']) else poly['normal'])
|
||||||
|
for i in range(1, len(poly['corners'])-1):
|
||||||
|
scene['indices'].extend((first_vertex, first_vertex+i, first_vertex+i+1))
|
||||||
|
if len(poly['corners']) > 3:
|
||||||
|
scene['warnings'].append(f'{entry.id}: polygon with {len(poly["corners"])} corners fan triangulated; concavity not validated')
|
||||||
|
material = poly['material']['id'] if poly['material'] else -1
|
||||||
|
if scene['meshes'] and scene['meshes'][-1]['shapeId'] == entry.id and scene['meshes'][-1]['material'] == material:
|
||||||
|
scene['meshes'][-1]['indexCount'] += len(scene['indices'])-first_index
|
||||||
|
else:
|
||||||
|
scene['meshes'].append({'name': entry.name, 'shapeId': entry.id,
|
||||||
|
'firstIndex': first_index, 'indexCount': len(scene['indices'])-first_index, 'material': material})
|
||||||
|
elif entry.type == '3DMa':
|
||||||
|
material = read_material(payload)
|
||||||
|
scene['materials'].append({'id': entry.id, 'name': entry.name, **material})
|
||||||
|
elif entry.type == 'Canv':
|
||||||
|
canvas = read_canvas(payload)
|
||||||
|
base = f'texture_{entry.id:08x}'
|
||||||
|
(output / (base + '.rgba')).write_bytes(canvas.pop('rgba'))
|
||||||
|
rgba = (output / (base + '.rgba')).read_bytes()
|
||||||
|
write_png(output / (base + '.png'), canvas['width'], canvas['height'], rgba)
|
||||||
|
scene['textures'].append({'id': entry.id, 'name': entry.name,
|
||||||
|
'path': base + '.rgba', 'png': base + '.png', **canvas})
|
||||||
|
(output / 'scene.json').write_text(json.dumps(scene, separators=(',', ':')) + '\n', encoding='utf-8')
|
||||||
|
return scene
|
||||||
|
|
||||||
|
|
||||||
|
def write_obj(shape: dict, path: Path):
|
||||||
|
"""Preserve original positions/UVs/polygon normals in a standard interchange file."""
|
||||||
|
lines = ['# Original OMT coordinates; no transform or guessed smoothing applied']
|
||||||
|
lines.extend('v ' + ' '.join(format(v, '.9g') for v in p) for p in shape['vertices'])
|
||||||
|
for poly in shape['polygons']:
|
||||||
|
lines.extend('vt ' + ' '.join(format(v, '.9g') for v in c['uv']) for c in poly['corners'])
|
||||||
|
lines.extend('vn ' + ' '.join(format(v, '.9g') for v in p['normal']) for p in shape['polygons'])
|
||||||
|
uv_index = 1
|
||||||
|
for normal_index, poly in enumerate(shape['polygons'], 1):
|
||||||
|
if poly['material']: lines.append(f"usemtl material_{poly['material']['id']}")
|
||||||
|
corners = []
|
||||||
|
for corner in poly['corners']:
|
||||||
|
corners.append(f"{corner['vertex']+1}/{uv_index}/{normal_index}")
|
||||||
|
uv_index += 1
|
||||||
|
if len(corners) >= 3: lines.append('f ' + ' '.join(corners))
|
||||||
|
path.write_text('\n'.join(lines) + '\n', encoding='utf-8')
|
||||||
|
|
||||||
|
|
||||||
|
def extract(database: Database, output: Path, raw: bool = False):
|
||||||
|
output = output.resolve()
|
||||||
|
validate_output(database, output)
|
||||||
|
output.mkdir(parents=True, exist_ok=True)
|
||||||
|
report = {'source': str(database.path), 'sha256': hashlib.sha256(database.data).hexdigest(),
|
||||||
|
'table_offset': database.table_offset, 'entries': []}
|
||||||
|
for entry in database.entries:
|
||||||
|
info = asdict(entry)
|
||||||
|
payload = database.payload(entry)
|
||||||
|
base = f'{safe_name(entry.type)}_{entry.id:08x}_{safe_name(entry.name)}'
|
||||||
|
if raw and entry.type != '0HDR':
|
||||||
|
(output / (base + '.bin')).write_bytes(payload)
|
||||||
|
if entry.type == '3DSh':
|
||||||
|
try:
|
||||||
|
shape = read_shape(payload)
|
||||||
|
write_obj(shape, output / (base + '.obj'))
|
||||||
|
(output / (base + '.json')).write_text(json.dumps(shape), encoding='utf-8')
|
||||||
|
info.update(vertices=len(shape['vertices']), polygons=len(shape['polygons']))
|
||||||
|
except FormatError as ex: info['unsupported'] = str(ex)
|
||||||
|
if entry.type == 'Wave':
|
||||||
|
start = payload.find(b'RIFF')
|
||||||
|
if start >= 0 and payload[start+8:start+12] == b'WAVE':
|
||||||
|
size = struct.unpack_from('<I', payload, start+4)[0] + 8
|
||||||
|
if start + size <= len(payload):
|
||||||
|
target = output / (base + '.wav')
|
||||||
|
target.write_bytes(payload[start:start+size])
|
||||||
|
try:
|
||||||
|
with wave.open(str(target)) as wav:
|
||||||
|
info['audio'] = {'channels': wav.getnchannels(), 'rate': wav.getframerate(),
|
||||||
|
'sample_width': wav.getsampwidth(), 'frames': wav.getnframes()}
|
||||||
|
except wave.Error:
|
||||||
|
info['audio'] = {'encoding': 'non-PCM RIFF preserved unchanged'}
|
||||||
|
else: info['unsupported'] = 'RIFF size outside resource'
|
||||||
|
report['entries'].append(info)
|
||||||
|
(output / 'manifest.json').write_text(json.dumps(report, indent=2) + '\n', encoding='utf-8')
|
||||||
|
return report
|
||||||
|
|
||||||
|
|
||||||
|
def validate_output(database: Database, output: Path):
|
||||||
|
source_directory = database.path.parent.resolve()
|
||||||
|
target = output.resolve()
|
||||||
|
if target.is_relative_to(source_directory) or database.path.resolve().is_relative_to(target):
|
||||||
|
raise ValueError('Output must be separate from original input directory')
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument('database', type=Path)
|
||||||
|
parser.add_argument('--output', type=Path)
|
||||||
|
parser.add_argument('--raw', action='store_true', help='Also extract exact raw resources')
|
||||||
|
parser.add_argument('--scene', action='store_true', help='Export runtime scene.json with decoded textures')
|
||||||
|
args = parser.parse_args()
|
||||||
|
db = Database(args.database)
|
||||||
|
if args.output:
|
||||||
|
if args.scene:
|
||||||
|
scene = export_scene(db, args.output)
|
||||||
|
print(f"{len(scene['vertices'])} vertices, {len(scene['indices'])//3} triangles, {len(scene['textures'])} textures; {args.output}")
|
||||||
|
return
|
||||||
|
report = extract(db, args.output, args.raw)
|
||||||
|
failures = [e for e in report['entries'] if 'unsupported' in e]
|
||||||
|
print(f'{len(db.entries)} resources; {len(failures)} unsupported conversions; {args.output}')
|
||||||
|
for fail in failures[:10]: print(f" {fail['type']}:{fail['id']} {fail['unsupported']}")
|
||||||
|
else:
|
||||||
|
for e in db.entries: print(f'{e.type} {e.id:10d} {e.offset:10d} {e.size:10d} {e.name}')
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__': main()
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
"""Small bounded decoder for the supplied non-interlaced 8-bit PNG textures."""
|
||||||
|
from pathlib import Path
|
||||||
|
import struct
|
||||||
|
import zlib
|
||||||
|
|
||||||
|
from omt import FormatError, Reader
|
||||||
|
|
||||||
|
|
||||||
|
def read_png(path: Path) -> dict:
|
||||||
|
r = Reader(path.read_bytes())
|
||||||
|
if r.take(8) != b'\x89PNG\r\n\x1a\n': raise FormatError('Invalid PNG signature')
|
||||||
|
header, palette, transparency = None, None, b''
|
||||||
|
compressed = bytearray()
|
||||||
|
ended = False
|
||||||
|
while r.position < len(r.data):
|
||||||
|
size = r.u32()
|
||||||
|
tag, data = r.take(4), r.take(size)
|
||||||
|
if r.u32() != zlib.crc32(tag+data): raise FormatError('PNG checksum mismatch')
|
||||||
|
if tag == b'IHDR': header = struct.unpack('>IIBBBBB',data)
|
||||||
|
elif tag == b'PLTE': palette = data
|
||||||
|
elif tag == b'tRNS': transparency = data
|
||||||
|
elif tag == b'IDAT': compressed.extend(data)
|
||||||
|
elif tag == b'IEND': ended = True; break
|
||||||
|
elif tag[0] & 32 == 0: raise FormatError(f'Unsupported critical PNG chunk {tag!r}')
|
||||||
|
if not ended or not header: raise FormatError('Incomplete PNG')
|
||||||
|
r.end()
|
||||||
|
width,height,depth,colour,compression,filter_method,interlace = header
|
||||||
|
if depth != 8 or compression or filter_method or interlace: raise FormatError('Only non-interlaced 8-bit PNG is supported')
|
||||||
|
channels = {0:1,2:3,3:1,4:2,6:4}.get(colour)
|
||||||
|
if not channels or not width or not height or width*height > 16777216: raise FormatError('Invalid PNG dimensions/colour')
|
||||||
|
stride = width*channels
|
||||||
|
expected = (stride+1)*height
|
||||||
|
decoder = zlib.decompressobj()
|
||||||
|
raw = decoder.decompress(bytes(compressed),expected+1)
|
||||||
|
if len(raw)!=expected or not decoder.eof or decoder.unused_data or decoder.unconsumed_tail:
|
||||||
|
raise FormatError('Invalid PNG decoded length')
|
||||||
|
rgba, previous = bytearray(), bytearray(stride)
|
||||||
|
for y in range(height):
|
||||||
|
mode = raw[y*(stride+1)]
|
||||||
|
scan = bytearray(raw[y*(stride+1)+1:(y+1)*(stride+1)])
|
||||||
|
if mode > 4: raise FormatError('Invalid PNG scanline filter')
|
||||||
|
for x in range(stride):
|
||||||
|
a = scan[x-channels] if x>=channels else 0
|
||||||
|
b = previous[x]
|
||||||
|
c = previous[x-channels] if x>=channels else 0
|
||||||
|
if mode==1: predict = a
|
||||||
|
elif mode==2: predict = b
|
||||||
|
elif mode==3: predict = (a+b)//2
|
||||||
|
elif mode==4:
|
||||||
|
p=a+b-c;pa,pb,pc=abs(p-a),abs(p-b),abs(p-c)
|
||||||
|
predict = a if pa<=pb and pa<=pc else b if pb<=pc else c
|
||||||
|
else: predict=0
|
||||||
|
scan[x] = (scan[x]+predict)&255
|
||||||
|
for x in range(width):
|
||||||
|
pixel = scan[x*channels:(x+1)*channels]
|
||||||
|
if colour==3:
|
||||||
|
index=pixel[0]
|
||||||
|
if palette is None or index*3+3>len(palette): raise FormatError('PNG pixel outside palette')
|
||||||
|
rgba.extend((*palette[index*3:index*3+3],transparency[index] if index<len(transparency) else 255))
|
||||||
|
elif colour==6: rgba.extend(pixel)
|
||||||
|
elif colour==2:
|
||||||
|
alpha=0 if transparency and tuple(pixel)==struct.unpack('>3H',transparency) else 255
|
||||||
|
rgba.extend((*pixel,alpha))
|
||||||
|
else:
|
||||||
|
alpha=pixel[1] if colour==4 else 0 if transparency and pixel[0]==struct.unpack('>H',transparency)[0] else 255
|
||||||
|
rgba.extend((pixel[0],pixel[0],pixel[0],alpha))
|
||||||
|
previous=scan
|
||||||
|
return {'width':width,'height':height,'rgba':bytes(rgba)}
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Prepare local scene/audio assets without modifying the original installation."""
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from omt import Database, export_scene, extract
|
||||||
|
from ase import export_ase
|
||||||
|
from png import read_png
|
||||||
|
|
||||||
|
|
||||||
|
def find_archive(original: Path, name: str) -> Path:
|
||||||
|
matches = [p for p in (original / 'omt').iterdir()
|
||||||
|
if p.name.casefold() == (name if name.casefold().endswith('.omt') else name + '.omt').casefold()]
|
||||||
|
if len(matches) != 1:
|
||||||
|
raise ValueError(f'Expected exactly one archive for {name}, found {len(matches)}')
|
||||||
|
return matches[0]
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument('--original', type=Path, default=Path('../Original'))
|
||||||
|
parser.add_argument('--output', type=Path, default=Path('artifacts/extracted'))
|
||||||
|
parser.add_argument('--scenes', nargs='+', default=['level1b', 'VR01'])
|
||||||
|
parser.add_argument('--audio', nargs='*', default=['sounds', 'musicjimmyshouse', 'musicneighborhood'])
|
||||||
|
parser.add_argument('--animations', nargs='*', default=['jimstop', 'jimrun', 'jimjump'])
|
||||||
|
args = parser.parse_args()
|
||||||
|
if args.output.resolve().is_relative_to(args.original.resolve()):
|
||||||
|
parser.error('Output must be outside the original installation')
|
||||||
|
for name in args.scenes:
|
||||||
|
source = find_archive(args.original, name)
|
||||||
|
scene = export_scene(Database(source), args.output / source.stem)
|
||||||
|
print(f"{source.name}: {len(scene['indices'])//3} triangles, {len(scene['textures'])} textures")
|
||||||
|
for name in args.audio:
|
||||||
|
source = find_archive(args.original, name)
|
||||||
|
report = extract(Database(source), args.output / source.stem)
|
||||||
|
count = sum(1 for e in report['entries'] if 'audio' in e)
|
||||||
|
print(f'{source.name}: {count} audio resources')
|
||||||
|
if args.animations:
|
||||||
|
texture_source = next(p for p in (args.original/'png').iterdir() if p.name.casefold()=='jimycarl.png')
|
||||||
|
texture = read_png(texture_source)
|
||||||
|
target = args.output/'animations'
|
||||||
|
target.mkdir(parents=True,exist_ok=True)
|
||||||
|
(target/'jimycarl.rgba').write_bytes(texture.pop('rgba'))
|
||||||
|
base = next(p for p in (args.original/'ASE').iterdir() if p.name.casefold()=='jimstop.ase')
|
||||||
|
for name in args.animations:
|
||||||
|
matches = [p for p in (args.original/'ASE').iterdir() if p.stem.casefold()==name.casefold() and p.suffix.casefold()=='.ase']
|
||||||
|
if len(matches)!=1: raise ValueError(f'Expected one ASE for {name}')
|
||||||
|
manifest = export_ase(matches[0],target/name,base)
|
||||||
|
manifest['texture'] = {'path':'../jimycarl.rgba', **texture}
|
||||||
|
(target/name/'animation.json').write_text(json.dumps(manifest,indent=2)+'\n',encoding='utf-8')
|
||||||
|
print(f"{matches[0].name}: {len(manifest['frames'])} animation samples")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__': main()
|
||||||
Reference in new issue
Block a user