Skip to content

Native linking

Function matching proves individual recovered translation units against reference instruction streams. It does not prove that those objects close over symbols, share a compatible ABI, or can reproduce a PE image. The native linker track makes those remaining obligations explicit.

Reference-layout source components

uv run crimson match report --refresh builds the configured components in tools/native/reference_layout.json and saves their receipts with the public matching evidence. uv run crimson native reference-link rebuilds the same components from fresh saved evidence. Artifacts live under artifacts/native-reference-link/.

The first component is the already proven grim-slot-state-accessors source cluster: four functions at 0x100072c0, 0x100072d0, 0x100072e0 and 0x10007300, backed by the independently bounded 128-element int/float arrays at 0x1005bf00 and 0x1005c100. All four compiler sections stay together, in compiler order. Only COFF subsection names (including their section-definition symbols) and timestamps change; instructions, function/data symbols and relocation records remain intact. Ordinary C++ definitions supply the arrays, with compile-time size checks. The pinned VC6 linker and authentic MSVCRT archive resolve the component without runtime imports, address aliases or placeholder providers.

Zero-filled COFF reservations retain the native section geometry around the component. They contain no copied reference payload and receive no code or data credit. Compiler alignment bytes also receive none. Verification checks linker map addresses, exact final function/storage bytes without masking, native section permissions, and the exact source/native PE base-relocation slots. Each relocation must target concrete storage compiled from the recorded source. The receipt binds source object, compiler/linker, archive and final PE hashes. CI validates the source-bound saved evidence; where ignored build artifacts exist, their hashes and final byte proof are checked again.

This earns 64 linked code bytes and 1,024 linked data bytes in 1.9.93. It is a passive PE with no entry point, not a runnable engine. Data credit describes loaded virtual storage; zero-filled arrays may have different file backing. Complete original source-file boundaries, PE headers, unbuilt regions and whole-image identity remain unproven. Existing structural linker receipts continue to receive zero public linked credit. Additional versions need their own compiler, native placement and relocation proof before receiving credit.

Image audits

Both grim.dll and crimsonland.exe have native audit tracks. Their active port scopes have one canonical scratch for every owned function. Isolated one-function translation units remain the default so pooling, inlining, constructor ordering, and static symbol changes cannot disturb established evidence.

Recovered Grim-owned bodies live under decomp/1.9/grim/. The module directories form a plausible logical source layout, not a claim about historical filenames or physical objects. They cover every canonical Grim function plus the recovered platform/provider functions. Scratch directories keep their matcher configuration and experimental evidence while their SOURCE fields bind to the central body. decomp/1.9/grim/layout.json distinguishes inferred subsystem grouping from native-proven translation-unit ownership; the native-link tests check it against every source file and source/config binding.

tools/native/translation_units/crimsonland.exe.json lists the executable's ownership clusters (tools/match/STATUS.md tabulates the current counts). Each is a proven minimum co-resident group, not a complete original source file. Two shapes recur:

  • Metadata lifecycles. VC6 generates each quest, bonus, or perk metadata array's initializer, registrar, and finalizer as four COFF-local functions in the translation unit that owns the global array. Compiling them separately turns the registrar's local finalizer relocation into a false external game-function dependency. The cluster binds the four recovered functions to their actual $E4/$E1/$E3/$E2 members and includes the physical object once.
  • Callback local-static destructors. A database or screen callback's canonical object emits its local-static destructor thunks as COFF-local $E functions, and native registration pushes to crt_atexit identify each thunk. Binding them to the callback removes redundant standalone objects.

Every configured member is still compared independently with its reference function using the canonical scratch boundaries. The audit rejects a cluster that lowers the byte ratio, adds unresolved or mismatched references, or loses an already exact encoded body even when normalized instructions and references still agree. This is source-provenance modeling, not a linker alias: the object retains the compiler-generated local symbols and local relocations. Adjacent callbacks stay separate: shared widget types and native address order alone do not prove one original TU.

reference-tu-audit-2026-09-08 records registration addresses, symbol bindings, and remaining reference classifications. lifecycle-ownership-2026-09-08 records the callback-thunk follow-up; its machine-readable evidence is tools/native/translation_units/crimsonland.exe-lifecycle-evidence.json.

The Grim configuration models four proven source islands: the slot-state accessors, line renderer and local vector destructor, monochrome text renderer, and JAZ decoder. The JAZ island combines its contiguous constructor, zlib-status helper, allocation method, and payload method in native address order under one /GX /MD provider. Each member of all four islands remains an exact match against its isolated reference boundary, and the combined COFF objects retain their compiler-generated local symbols and internal calls without changing the function inventory.

Grim has one explicit link-time exception in tools/native/linker_aliases/grim.dll.json. Both native cleanup calls in grim_decode_jaz_texture pass the decode-scope address in ecx and branch to the selected one-byte grim_noop function. The audit verifies those direct call instructions in the pinned DLL, the exact _grim_noop object definition at the recorded target address, and the decoder's exact decorated destructor reference. It then emits a deterministic i386 COFF weak-external alias object for GrimJazDecodeScope::~GrimJazDecodeScope -> _grim_noop. No destructor body is invented and the canonical noop retains its recovered identity.

Generate the checked-in reports:

uv run crimson native audit --image grim.dll --out-dir analysis/native/grim.dll
uv run crimson native audit --image crimsonland.exe --out-dir analysis/native/crimsonland.exe

The command compiles the canonical scratch set with each scratch's recorded VC6 profile, forcing an isolated rebuild rather than accepting cached object provenance. It compiles the Grim ABI assertion unit, parses the resulting i386 COFF objects, and writes:

  • an address-ordered canonical object manifest and link list;
  • a .def file mapping the reference export name and ordinal to its decorated object symbol;
  • exact decorated symbol definitions and references;
  • strong duplicate and COMDAT/coalescible definition inventories;
  • linker directives and requested default libraries;
  • explicit weak-external linker aliases and their evidence;
  • reference PE exports and imports;
  • unresolved symbols classified as owned functions, excluded functions, game data, imports, known toolchain support, or unclassified external dependencies;
  • an evidence-limited data manifest.

All JSON is stable, sorted, free of timestamps and absolute paths, and hashes its material inputs. VC6's COFF TimeDateStamp is explicitly zeroed for object hashing; clean rebuilds otherwise retain the same content hash. The compiler-bundle fingerprint covers the material Bin and Include trees, not unrelated SDK or library directories. A shared audit digest detects mixed-generation JSON, and component hashes cover the object list and export definition. The report never adds fake providers.

The shared semantic name_map.json and data_map.json inputs use a canonical per-image JSON projection. The name map contains callable entry points only; arrays, scalar globals, sentinels, and function-pointer variables belong in the data map. An edit to Grim-only rows therefore invalidates the Grim audit without needlessly rebuilding Crimsonland, while edits to shared top-level data-map metadata still invalidate both images. Older audit records without projection metadata retain their full-file hash behavior.

objects.json distinguishes function_count from physical object_count. Each physical row has a functions array; ordinary rows contain one binding, while an explicit cluster contains every independently validated member. The link list contains each physical object exactly once.

The executable audit keeps any remaining closure debt explicit: same-display-name definitions with incompatible decorated linkage are not treated as resolutions, absent definitions remain visible, and repeated non-COMDAT .bss owners remain hard duplicates. The modeled ownership clusters close the executable's current game-function debt without weakening those rules.

Known compiler helpers are classified as toolchain references. CRT entry points are stricter: _sscanf, for example, is classified as toolchain only when its referencing object requests a recognized VC6 CRT default library. The closure row records that library evidence; without it, the symbol remains an unclassified external and blocks game-owned closure.

Closure gates

The first native-link victory is all game-owned symbols resolving, not whole-PE byte identity.

The report separates three gates:

  1. function_closure: no unresolved in-scope game functions, no duplicate strong definitions, and every reference export has an unambiguous mapping.
  2. game_owned_closure: function closure plus no unresolved mapped game data or unclassified external symbols.
  3. all_references_closed: no unresolved COFF reference of any category.

Use --require-game-closure when the second gate is expected to pass. It exits with status 1 while honest game-data debt remains; malformed inputs or build failures exit with status 2.

CI verifies the checked-in artifacts for both images with:

uv run crimson native verify --require-game-closure --allow-absent-toolchain

This recomputes the shared audit digest, checks the companion hashes and every available recorded input, and enforces both game-owned closure claims without requiring a proprietary compiler bundle on the CI runner. The exception is narrow: only absent ignored files below tools/match/bin/ or tools/match/compilers/ are permitted. If those files are present, their recorded hashes are still checked; all tracked sources, configs, catalogs, definitions, reference binaries, and generated companions remain mandatory.

Grim is the first image with a checked-in provider boundary. Once its game-owned closure gate passes, run:

uv run crimson native link --image grim.dll --out-dir analysis/native/grim.dll/link

tools/native/providers/grim.dll.json must cover the unresolved-symbol set exactly. It distinguishes exports that the reference PE really imports from MSVCRT.dll, USER32.dll, WINMM.dll, and d3d8.dll. The original VC6 SP6 MSVCRT.LIB supplies the MSVCRT imports plus the exact atonexit.obj runtime adapter and dllsupp.obj toolchain absolutes. The original DirectX 8.1 d3dx8.lib supplies the D3DX closure symbols, including byte-proven aliases from the recovered d3dx_copy_texture_filtered name to the archive's D3DXComputeNormalMap and from jpeg_std_error to the archive's namespaced IJG body. Deterministic VC6 builds from the pinned IJG 6a and zlib 1.1.3 sources supply the codec symbols. The remaining closure imports use generated import libraries, and the recovered platform archive supplies the host-replaced closure functions.

The recovered platform and D3DX archives also need transitive symbols while the linker selects and prunes their members. These are modeled as link-dependency providers, not counted as closure coverage. The exact grim_window_proc callback is retained through this boundary together with a weak alias from its matcher-only grim_noop_value spelling to the native grim_noop. Every recovered-platform member, including that callback, is raw-byte exact. /OPT:REF discards unused link-dependency imports; every retained output import is required to exist in the pinned reference PE table. Every provider group cites repository evidence. Per-provider symbol counts are in the summary of analysis/native/grim.dll/link/link.json.

All provider archives are ignored inputs pinned by size and SHA-256 to analysis/library_provenance.json. Extract the proprietary historical archives and rebuild the open-source codec archives at their configured paths before linking:

mkdir -p tools/native/providers/build/vc6-sp6
cabextract -F vc98/lib/msvcrt.lib \
  -d tools/native/providers/build/vc6-sp6 \
  /path/to/VS6sp61.cab

mkdir -p tools/native/providers/build/directx-8.1
unzip -j /path/to/DX81SDK_FULL.exe \
  DXF/DXSDK/lib/d3dx8.lib \
  -d tools/native/providers/build/directx-8.1

uv run python scripts/build_native_codec_providers.py \
  --jpeg-tar /path/to/jpegsrc.v6a.tar.gz \
  --zlib-tar /path/to/zlib-1.1.3.tar.gz

The codec recipe verifies its pinned source and tool hashes, applies IJG's documented one-byte Windows boolean ABI, normalizes archive and COFF timestamps, and requires exact matches for the recovered libjpeg functions plus zlib's uncompress entry before publishing either archive.

The recovered platform archives (grim-platform.lib and crimsonland-platform.lib) need no manual step. They are built from the exact platform scratches by crimson-re/src/crimson_re/native_providers.py, together with a data object for the globals only those functions reference. That data object depends on which globals the audit's own data object already defines, so crimson native link rebuilds an archive from its fresh audit whenever the local copy is missing or does not match its pin. If the rebuilt archive still misses its pin, its recovered sources changed: update the derived-artifact row in analysis/library_provenance.json and the archive entry in tools/native/providers/<image>.json.

The link command rebuilds the canonical audit and any stale recovered platform archive, rejects other archive drift, synthesizes the remaining closure imports and D3DX link dependencies with the recorded VC6 tools, emits deterministic weak COFF aliases where compiler decoration differs from the reference export name, and creates an explicitly labeled per-symbol COMDAT placeholder COFF object for the remaining providers. It then invokes the VC6 release linker with the recovered entry point, image base, export definition, and dead-code elimination. It verifies every configured closure export plus the complete output import-table subset against the reference image. It writes analysis/native/grim.dll/link/link.json; bulk PE, map, library, response, log, and provider outputs remain ignored. Generated archive, object, PE header, and PE export-directory timestamps are normalized, so identical inputs produce the same linked-image hash.

This remains a structural linker milestone, not a runtime test or whole-image byte-match claim. No placeholder COMDAT remains, so the canonical Grim receipt now reports runnable=true. It validates every output import against the reference PE and emits a deterministic image with a normalized zero timestamp; link.json records the output's PE header fields and hash. The separate closure report still classifies imports, toolchain symbols, and deliberately excluded functions at the port boundary; the provider-backed link is what proves those required edges are concretely linkable.

The executable uses the same provider boundary, with two important differences. It is linked as a Windows GUI PE at base 0x00400000, and its runtime is the statically linked VC6 SP6 LIBCMT.LIB rather than Grim's MSVCRT.DLL seam:

uv run crimson native link --image crimsonland.exe --out-dir analysis/native/crimsonland.exe/link

tools/native/providers/crimsonland.exe.json covers the non-game closure exactly. The pinned VC6 archive supplies the excluded CRT functions plus __fltused and sscanf; the pinned DirectX 8.1 archive supplies D3DXVec2Normalize; generated import libraries supply exports that are present in the reference PE; a deterministic recovered-source archive supplies the exact registry helpers and the complete legacy DirectX version island. No closure symbol remains a placeholder.

The recovered-source recipe compiles the exact all-scope platform functions: the DirectX wrapper, its DxDiag query, the file-version fallback, the version-resource leaves, and the two registry helpers. It rejects any instruction or reference regression before publishing the normalized archive. The DxDiag member also carries the two byte-proven GUID definitions from its original translation unit.

The executable now enters through VC6's authentic WinMainCRTStartup member from wincrt0.obj. A deterministic weak alias maps the CRT's _WinMain@16 reference to the recovered _crimsonland_main@16, so the link exercises the GUI startup graph instead of selecting the console startup and fabricating a _main shim.

Archive matching proves that the vector-normalization initializer and thunk are D3DX8's init_D3DXVec2Normalize and _D3DXVec2Normalize@8 in d3dxmath.obj. Recovered callsites now use the original archive identity directly, so this provider needs no compatibility alias.

The archive-backed graph's transitive symbols mostly map to reference-backed KERNEL32, ADVAPI32, OLE32, OLEAUT32, or VERSION imports; a few resolve to CRT members such as __snprintf and _tolower in the pinned VC6 archive. Archive imports absent from the reference import boundary (locale, console-handler and TLS helpers) stay explicitly configured as link-only placeholders, but per-symbol COMDATs let /OPT:REF prove that they are discarded and do not survive in the produced PE.

Every output import of the canonical structural link is present in the reference table, including DSOUND ordinal 11 and OLEAUT32 ordinals 8 and 9. Every link-only placeholder is discarded, so the CLI reports placeholders=0/N runnable=True. The output is an i386 Windows GUI PE at base 0x00400000 with a normalized zero timestamp. The checked record, analysis/native/crimsonland.exe/link/link.json, holds the symbol counts, discarded placeholders, and output PE header fields; the PE, map, response, log, generated provider libraries, aliases, and placeholder object remain ignored.

ABI boundary

Each image assertion unit is compiled by the same 32-bit VC6 toolchain before the audit succeeds. The Grim unit checks the recovered C++ surface's pointer and scalar widths, configuration-record and interface sizes, vtable extent and selected slot offsets, default packing, factory signature, and representative __thiscall/variadic member types. The Crimsonland unit checks the common scalar model, evidence-backed gameplay/UI record sizes and offsets, console object packing, and the recovered mod API's C++ member-pointer surface.

The recovered C++ interface is the callable authority. Analyzer-oriented C headers preserve analysis layouts but are not a replacement SDK for native __thiscall dispatch.

Data boundary

analysis/ghidra/maps/data_map.json supplies curated identities; tools/native/data_definitions/<image>.json supplies the proven sizes, alignments, and initializers used to emit linkable data objects. The audit's analysis/native/<image>/data.json summary reports coverage: every mapped EXE entry is fully specified, while Grim still has partially specified rows. Neither claims that every original data object is mapped. Unknown extents are not inferred from adjacent labels, and .data membership does not imply a zero initializer.

Tracked artifacts and their exact field contract are described in analysis/native/README.md.