pmf: obfuscated Ogg Theora movies

Source

docs/formats/pmf.md

SHA-256

219db7ea…849feddd

The game's full-motion movies, in movie/: the opening, the chapter movies, the trailer and one title movie per language. A .pmf is an ordinary Ogg file (one Theora video stream and one Ogg Skeleton stream, no audio) whose bytes are XORed with a repeating 60-byte key, RotXor's key 1, by file position. There is no header, no ZIP and no other wrapping. The game reads the file straight from disk, removes the XOR as it reads, and hands the bytes to libogg and libtheora. A movie's sound isn't in the .pmf: it is a separate sabf sound bank, played alongside and used as the movie's clock (Sound).

Layout

A .pmf is n bytes, with no header of its own.

• confirmed: after removing the XOR, every file is a run of Ogg pages from offset 0 to the end of the file, with nothing before, between or after them: 19,547 pages in Prologue.PMF, 1,587 to 7,323 in the others, none with a bad CRC or a gap in its sequence numbers. Source: pmfscan.cs, all 17 files.
• confirmed: the XOR is its own inverse, so applying the same step to an Ogg file makes a .pmf, and applying it to an unchanged .pmf's Ogg gives back the original byte for byte. Source: XOR; pmfscan.cs shows the game's step yields valid Ogg for all 17 files.

Key 1

The 15 key words are RotXor's key 1, the same 60 bytes at 0x140ba5300 that the .dat deobfuscation uses from start word 8. Here the start word is the word index in the file, so word w uses key word w mod 15, and the key repeats every 60 bytes of the file. Key word k is bytes 4k to 4k + 3 of the key, in order.

• likely: the read callback passes position >> 2 as the start word, and 0x1401a2bf0 XORs each whole 4-byte word of the buffer with key word (start + i) mod 15, leaving any bytes after the last whole word alone. Source: 0x14026c260 (ftell, fread, then 0x1402b0970(buffer, bytes read, position)), 0x1402b0970, 0x1401a2bf0.
• confirmed: every file starts FF DB 0A 4A, which is OggS XOR key word 0 (B0 BC 6D 19). Source: pmfscan.cs, all 17 files.

What the Ogg holds

Each file has two logical streams, both opened on the first pages and both ended with an end-of-stream page. These are data facts, confirmed by pmfscan.cs over all 17 files, unless marked.

• confirmed: no file has a Vorbis (or any other) audio stream. Source: pmfscan.cs, all 17 files.

The Theora identification header, the same in every file except where noted:

The files

Movies are file-table group 2000; the file ID picks the movie. Frames and length are from the Theora stream's last granule position. Sound is the sound bank row for the file ID. All values confirmed (ftmovie.cs, pmfscan.cs, system/system_data.dat#6), except the Plays column, which is likely (static analysis, Who plays a movie).

• confirmed: each file's FileTable.bin size equals its size on disk. Source: ftmovie.cs.

How the game reads it

Static analysis of the unpacked executable, so every fact here is at most likely; the functions are in the names list.

Opening

• likely: every movie is started through Movie_Open (0x1402f3cd0, a wrapper over 0x14026b0f0) with group 2000 and a file ID; nothing opens a .pmf by name. Source: the three callers of 0x1402f3cd0 (Who plays a movie) all pass 2000.
• likely: 0x14026b0f0 creates a VideoTheoraDecoder and calls 0x140284f80, which opens group 2000 and the file ID through the file table with flags 0x2000000 (FileTable_OpenById, 0x1402af860). That opens a file-system handle on the table's path under the data root and keeps the full path in it; nothing is read through the file system's queue, so the table's size isn't used. Source: 0x14026b0f0, 0x140284f80, 0x1402af860 → FileDesc_OpenPath (0x140295760, size flag clear; the full path is the string at the start of the handle).
• likely: the decoder then opens that path itself with _fsopen(path, "rb", _SH_DENYNO) and starts a decoding thread over it. If the open or the start fails, the movie decoder is replaced by a VideoNullDecoder and the movie is skipped, with no error. Source: VideoTheoraDecoder::vfunction12 (0x14026bdd0), 0x14026b0f0 (falls back to VideoNullDecoder).

Reading

• likely: the decoding thread reads the file in 4096-byte freads from offset 0 to the end of the file, each into libogg's sync buffer (ogg_sync_buffer, ogg_sync_wrote), through the read callback 0x14026c260, which removes the XOR from the bytes just read (Key 1). It stops when a read returns 0 bytes. Source: 0x1401e09c0 (buffer, read of 0x1000, wrote), 0x14026c260.
• likely: the only seek is back to offset 0, to play a movie again from the start (a loop flag in the decoder state); every read therefore starts at a multiple of 4096, and the key stays aligned. Source: 0x1401e0ae0 (calls the seek callback 0x14026c2e0 with offset 0, SEEK_SET, at the end of the file when the flag at +0x80 is set).
• likely: the file's size in FileTable.bin is never used: the decoder reads until fread returns nothing. Source: 0x14026c260, 0x1401e09c0; the handle's size (+0x48) isn't read by the movie code.
• likely: the read position goes through ftell, which returns a 32-bit long on Windows; a file of 2 GiB (2,147,483,648 bytes) or more would get a wrong key from that point. Source: 0x14026c260 (_ftell_nolock, then (uint)position >> 2).

What it checks

The decoding thread (0x1401e0ae0, started by 0x1401e29f0 with at most 75 decoded frames queued) reads the beginning-of-stream pages, then checks the Theora header. Nothing in the .pmf is checked by the game's own code before libogg sees it.

• likely: of the streams that start on the first pages, the first Theora stream is the video and the first Vorbis stream the audio; any other stream (such as the Skeleton) is dropped. Source: 0x1401e0ae0 (th_decode_headerin 0x1401f63b0 tried first, vorbis_synthesis_headerin 0x140631000 only while no Vorbis stream is found, otherwise the stream is cleared).
• likely: decoding stops, and nothing is shown, if the file has neither a Theora nor a Vorbis stream, if frame_width or frame_height is over 99,999, if the colour space is over 2 (only unspecified, Rec. 470M and Rec. 470BG), if the pixel format isn't 0 (4:2:0) or 3 (4:4:4), or if the Theora setup header is missing. Source: 0x1401e0ae0 (the th_info fields at +0x04, +0x08, +0x2C, +0x30; field order from libtheora's codec.h).
• likely: the frame rate is taken from fps_numerator / fps_denominator and timestamps each frame. Source: 0x1401e0ae0.
• likely: decoded Vorbis audio is never played: the decoder takes every queued audio packet and frees it at once. Source: VideoTheoraDecoder::vfunction8 (0x14026c300: 0x1401e3130 / 0x1401e31b0 in a loop).
• likely: the video is drawn from two textures created at the decoded frame's width and height; no size limit was found beyond the 99,999 check. Frame buffers come from two 'movie pool' allocators, falling back to the general allocator for the movie heap when they can't serve a request. Source: 0x14026c300 (texture creation), 0x14026b960 (pools), 0x14026c1e0 (fallback to 0x140198c90 with heap 0x1A).

Sound

• likely: when a movie is opened, 0x14026b0f0 reads row file ID of system/system_data.dat#6 (an xlce table of 18 rows × 8 bytes: two u32 sound IDs, the second used when the voice setting at +0x2B of 0x140ce4e30 is set) and, if the ID isn't 0, opens that sound bank by ID, reads it whole and plays it. The group is the voice language's (401 for Japanese, 402 for English, from 0x1402860d0), except for sound IDs 71 and 100, which come from the group at 0x140ce1aa8 + 0x20: the sound manager's group, 400 (sound/), stored there when game setup (0x1402c2110) creates the sound manager with it (0x140287200). Source: 0x14026b0f0 (table at 0x140ce4da0, set from system_data entry 6 by 0x1402bd7d0), 0x1402860d0, 0x140286e90, 0x1402887c0.
• confirmed: the table's rows 1–9 hold the same ID in both columns, and rows 0 and 10–17 hold 0; IDs 71 and 100 exist only in group 400 among the sound groups, which fits. Source: ftmovie.cs.
• likely: a movie with a sound bank is timed by the sound: each frame is chosen by the sound's playback position, and the movie ends when the sound stops playing or the video ends, whichever comes first. A movie with no sound (the title movies) is timed by the clock. Source: 0x14026b5d0 (0x1402860f0 position and 0x140286110 state of sound slot 0; finished flag at +0x73).

Who plays a movie

• likely: the caption player (textseq/textseqcaption.cpp) plays the chapter movies and the trailer: TextSeqCaption_Start (0x140430f30 → 0x140431490) loads textseq/textseq_<lang>.dat (group 2700, file ID = language index + 1) and picks a caption sequence (entry) from a list at 0x140c9df00 by movie number 1–9, plus 1 when the voice setting is set; the sequence's first row of type 1 holds the movie's file ID at +0x06, which 0x1404316d0 passes to Movie_Open. It is started from the title (0x1404242e0, movie numbers 1 and 2), the Warren Report's movie player (0x1404c39f0) and 0x140317850. Source: those functions.
• confirmed: caption sequences 1–17 in every textseq_<lang>.dat checked (ja-jp, en-us) name movie IDs 1, 1, 2, 2, …, 8, 8, 9. Source: textseq.cs.
• likely: the title screen plays file ID 10 + language index (0 Japanese, 1 English, 2 French, 3 German, 4 Spanish, 5 Simplified Chinese, 6 Traditional Chinese, 7 Korean, the order of the textseq_<lang>.dat files). Source: 0x1404adc70 (language from 0x14026ca60).
• likely: the gallery's movie entry plays the file ID in the u16 at +0x00 of a row of menu/menu_data.dat#87. Source: MenuGalleryMovie::vfunction5 (0x1405142c0).

Making a replacement movie

What a mod has to produce, from the facts above. Not tested in game.

• Encode an Ogg file with one Theora stream: 4:2:0 (or 4:4:4), colour space unspecified, Rec. 470M or Rec. 470BG. An audio stream is ignored, and so is a Skeleton stream. Any resolution and frame rate the header can carry should work; the originals are 1920 × 1080 at 29.97 frames per second.
• XOR it as in Layout: every whole 4-byte word w with key word w mod 15, leaving the last n mod 4 bytes as they are. Keep it under 2 GiB.
• Put it at the original's path. The game doesn't need its FileTable.bin size changed, but set it to the new size anyway, like every other rebuilt file (filetable.md).
• For movies 1–9, the picture follows the sound bank's playback and stops when it stops: a replacement of a different length needs its sound bank replaced to match (see the sabf doc). Captions come from textseq/.

Leads

• The gallery's movie list: menu/menu_data.dat#87 holds 12, 0, 1, …, 11 in its first field, which don't all read as movie file IDs; how MenuGalleryMovie is given its row is unchecked.
• What sets the movie number at 0x140d5c698 that 0x140317850 passes to the caption player (probably an event-script native), and the list of sequences at 0x140c9df00.
• What sets the decoder's loop flag (+0x80 of the decoding state; the fourth argument of 0x1401e29f0, not shown by the decompiler at its one call).
• The decoding wrapper's structure (4096-byte feeds, the 99,999 check, the colour-space check, audio and video queues) looks like icculus's TheoraPlay; checking it against that source would name its functions.
• Confirm in game: a re-encoded movie of another resolution, a movie with an audio stream, and a file over 2 GiB.