How docs are written

Source

docs/templates/README.md

SHA-256

a9a1599b…d2766e2b

Every doc and schema about the game's data or code (format docs, table and file schemas, executable docs, issue reports) follows these conventions. A new format doc or issue report starts from its template here, keeping its section order (sections that don't apply yet may be left out); a new schema follows schema.md (schema.md). Table and file pages are generated, never written:

The toolchain reads the layouts, value lists and entry names in the format docs and schemas, so their shapes below matter. A fix to a table or a file page is made in its schema, then tor pages re-run.

An issue report owns its issue's story: the symptom, reproduction, cause, fix and test results. What it finds about a field or a function also goes in that field's schema or the executable doc, and the report links there. Its For modders section is the one place a finding is simplified on purpose, for players and modders who don't read code.

Notation

• Game paths are relative to the game's Data/ folder: battle/entry/....
• Offsets are hex (0x1A4) and sizes are in bytes. Values are little-endian unless the doc says otherwise.
• A data doc or schema describes the data: its structure and what it means. What a tool covers, how it names or lays things out, and where it disagrees with the data go in that tool's doc; a data doc mentions a tool only in Source, or in a short note where the data contradicts it.
• A format doc or table schema describes everything needed to rebuild its data: order, alignment, padding, how offsets and counts are derived, and any bytes whose meaning is unknown but which must be kept as they are.

Evidence and confidence

Each documented format, field or value says how it was found (tool output, hex inspection, a DuckDB query, static analysis of the executable with the function's address and the tool, in-game test, or, as a cross-check only (Reference tools), an outside source with its URL or repo path and the date checked) and how sure we are: confirmed, likely or guess, kept visibly apart. A fact found only by static analysis is at most likely until a decode or an in-game test confirms it. Prose facts mark their confidence in bold: confirmed:, likely:, guess:.

Evidence block

A two-column table right after a doc's opening line. It covers the whole doc; individual facts still carry their own confidence, and their own source where it differs. A generated page's is generated: a file page's holds the file's SHA-256, and a table page's links the file pages of every file its schema matches; another doc relying on a file that has a page links to that page instead of repeating the hash (a game patch can change a file, and the hash shows which version the findings describe; findings are then re-verified and updated in place, rule 13 (../rules.md)). A schema has no evidence block: what it would hold is in its keys (name_from, code, a field's code, text and data) or generated.

Docs may add rows (a file page's Wrapping and Payload, say). Table pages add Entry path, Rows × size, Schema and Code.

A passing round trip proves that every byte is kept and that the layout fits every file it covers, not what a field means. It supports byte preservation and rebuild rules; it does not establish field boundaries, signedness or meanings. Those claims keep their own evidence and confidence.

Reference tools

The tools in reference_tools/ (Gibbed's tools, Nightmare and its modules) and the docs in legacy/ are references: consult them, check our findings against them, and, where the game and our analysis don't show what something is, use what they say to inform our own layouts and names. Whatever they contribute is cited in Source, and no name or layout is copied from them unchecked. Where a reference covers the same bytes, Source says whether it agrees or not: "agrees with Gibbed's arms_data sheet". A fact or name whose only source is a reference is guess, until our own evidence confirms or replaces it.

Names

Every name we give (a format, table, entry, field, value, function, file, command or term) follows these rules, so that a name is made once and never redone.

1. The game's own name first, where it has one: a string or text key in its files, a file or folder name, a magic, a class or source-file name in the executable. Where the name is written down says where it is: a field's Source, a doc's Name from row, a term's row in the glossary. Where the game has two words for one thing, the word in its English text wins, since that's what players and modders read (music, not the code's bgm); the code's word (a text key, class or source file) is used only where the English text has none (clut, vfx) or is ambiguous (tribe), and the glossary lists the other word under Other names.
2. Otherwise ours, saying what the data does: from what the code does with it (the function's address cited) or what the decoded data shows (the DuckDB query cited).
3. References inform, they don't decide. Where the game and our analysis don't show what something is, we make our own name from what a reference calls it, marked guess and citing the reference in Source, until our own evidence confirms or replaces it. A reference's name is never copied unchecked; a name we make in our own style may end up the same as theirs, and that's fine. Where a reference names the same thing, the glossary lists that name under Other names, so it can be searched for.
4. Only words we've checked. A word that claims a scope (local, global), a role, a unit or a direction is used only when the code, the data or a cited reference shows it. Something whose meaning isn't known from any of them is named by where it is: unk_<offset> for a field, unk_<part>_<offset> for a field after a variable-size part (unk_header_end for the one after a pakd's IDs), <file stem>_<index> for a table or entry and <file stem>_<index>_<index> for a nested one (never entry_<n>).
5. One term per concept, the same in every doc, table, field, function, file and command, and every term in the glossary with what it means and the evidence for its name.
6. The same kind of thing gets the same kind of name. Tables, fields and values are snake_case.
7. A generic word is always qualified. A word that means several things (flag, kind, slot, side, version, …) is never used alone: it says where the thing lives or what it belongs to (game flag, unit flag, unwrap kind, equipment slot), and a field that packs yes/no bits is <what>_flags. The glossary lists each qualified form and where the bare word is allowed.

Tables

• Entry path. A table is identified by its game path, #, and its pakd entry index, with / between nested indexes: battle/battle_data_release.dat#14, or #75/3 for entry 3 of the pack at entry 75. A table that many files share, at the same entry, is identified by a glob over the game path, matched case-insensitively: battle/entry/entry_unit_[0-9]*.dat#0. A table that every entry of a nested pakd shares is identified by a glob over the entry path, the same way: battle/battle_data_release.dat#75/* (* also matches /, so it takes nested entries below them too). Where a doc names an entry exactly and another matches it by a glob, the exact one lays it out. A table that is a whole file, not a pakd entry, is its game path alone: effect/EffectMaterialData.bin. The files carry no table names, so the entry path is the only stable ID.
• Name. A table's name, used as its schema's and page's filename and title, follows Names, with where it comes from in the schema's name_from. A table shared by many files has one name for all of them. Until its purpose is known from the code or data, it's <file stem>_<index> (e.g. battle_data_release_22).
• One schema per table. Each table that gets field documentation has its own schema (schema.md (schema.md)), and from it its page. One that holds several kinds of record by row range (#16 holds spells, finishers and skills) is one table, with its row ranges in row_ranges.
• Size. The row count and row size are read from each entry's header and shown on the table's page, never written in the schema.
• Layouts. A table's layout is its schema's fields (and header), with types from Types; every other binary layout (a format's header or record) is one Markdown table in its format doc with the columns Offset, Size, Type, Name, Meaning, Confidence and Source, in that order. Field offsets are relative to the start of a row; absolute offsets into the payload may be given alongside, though they shift when any earlier entry changes size. Gaps between known fields are listed as unknown fields, never skipped.
• Header. A table that isn't xlce gives its header in the schema's header, with offsets from the start of the entry and a row_count field. Rows start right after the header, and the row size is where the last field ends.
• Field names follow Names. A field whose meaning is unknown is unk_<offset>, e.g. unk_0x1C.
• Repeated fields. In a format doc, a field repeated at a fixed stride may be listed once, with Offset written 0x22 + 4n, n = 0–9 and [n] in its Name (equip_skill_id[n]); Size is one element's size. In a schema it's a repeat block (schema.md (schema.md#repeated-fields)).
• Confirmed fields. A field becomes confirmed only after a hex inspection, decode or in-game test, recorded with the field. Layouts are never copied from a reference (Reference tools).

Types

Value lists

A value list (an enum, or the names behind an ID) used by more than one table lives in one place, and each field that uses it names it; a list used by one table stays in its schema or doc. Its names follow Names: the game's own text where it has it, otherwise ours from the code that reads the value. The toolchain names each value with it in the pages and the decoded data.

In a schema, a field names its list with enum (its own schema's enums, or <schema>.<list>), and a field whose value picks a text key names it with text_link: the value picks a row of the linked table, whose str_ofs64 field holds a text key, and that key's English text (localize/LocalizeText_en-us.dat) is the name (schema.md (schema.md#fields)). A key with no English text, or a row past the end, names nothing.

In a format doc, the link in Meaning has one of these forms:

A link to another doc without Names: is an ordinary reference, not a value list.

File pages

Each game file has one page at docs/files/<path under Data/> with its extension replaced by .md: battle/battle_data_release.dat → docs/files/battle/battle_data_release.md. docs/files/README.md (hand-written) describes Data/ as a whole.

tor pages writes the whole page on every run, and nothing in it is edited by hand: its title, a marker line saying so, then:

• Notes, from its file schema (schemas/files/<path under Data/>.yaml, schema.md (schema.md#file-schemas)) when it has one: what the file holds, its Entry names, How the game loads it, and Leads;
• the file's evidence block (Contents);
• a byte map that accounts for every byte of the payload from offset 0 to the end with no gaps, each region linked to the format doc or table page that lays it out, a region the generator can't explain listed as unknown (a nested pakd may be one region);
• every row of each documented table, with each field's Meaning and Confidence from its schema;
• a Text keys section: the text keys found in its entries and the text table each belongs to, from the English text (localize/LocalizeText_en-us.dat) for now. The English text's page lists, in a Used by section, every extracted entry where each of its tables' keys were found, from the same token observations; no page is read as an input.

A file schema names a container's entries, where a name is known, in entries: one entry per row ("11", 75/3), or a range or list of them (3–10, 1, 3, … 49) that labels nothing. The generator labels a single entry with its name in the byte map and, for an entry no table schema covers, in its section heading. Each entry's kind, offset, size and row count come from the byte map.

Table pages

Each table schema has one page, docs/tables/<name>.md, that tor pages writes whole on every run from the schema and every matching entry in the complete extracted dataset, whichever file-page selectors the run was given: its description, an evidence block (entry path, row counts and size, the files it's based on with their hashes, the schema, name from, code, checked, method, round trip), Notes, Row ranges, Header, Fields (each field's layout, name, meaning and confidence from the schema, its values over every row, and its source), Value lists and Leads. A page whose schema is gone is deleted.

A file or table page is linked from docs/README.md (../README.md) only once its schema says something; every other doc is linked from there.