rrt/docs/README.md

# RT3 Reverse-Engineering Handbook

This handbook is the project bootstrap for reverse-engineering and rewriting Railroad Tycoon 3.
It is written for future us first: enough structure to resume work quickly, without pretending the
project is already mature.

## Canonical Target

- Canonical executable: `rt3_wineprefix/drive_c/rt3/RT3.exe` (patch 1.06)
- Reference executable: `rt3_wineprefix/drive_c/rt3_105/RT3.exe` (patch 1.05)
- Canonical SHA-256: `01b0d2496cddefd80e7e8678930e00b13eb8607dd4960096f527564f02af36d4`
- Reference SHA-256: `9e96b0695cb722a700f99c8dce498d34da7235e562b1e275bcc1764f8c9b7eb1`

## Documents

- `setup-workstation.md`: toolchain baseline and local environment setup.
- `re-workflow.md`: how to analyze the binary, record findings, and export reusable artifacts.
- `function-map.md`: canonical schema and conventions for function-by-function mapping.
- `control-loop-atlas.md`: compatibility index for the split atlas, preserving legacy anchors.
- `control-loop-atlas/`: canonical section files for the atlas narrative.
- `runtime-rehost-plan.md`: bottom-up runtime replacement plan and milestone breakdown.

## Repo Conventions

- `docs/`: stable project guidance and durable design notes.
- `tools/py/`: committed Python helpers for analysis and validation.
- `artifacts/exports/`: committed derived outputs that can be regenerated.
- Local-only state stays untracked: `.venv/`, Ghidra projects, Rizin databases, crash dumps, and other
  bulky/generated working files.

## Current Baseline

The current technical milestone is a repeatable loop-mapping workflow for the 1.06 executable.
Before injection work or deep file-format work, we capture:

- executable hashes and PE metadata
- section layout, imports, and notable strings
- a starter subsystem inventory plus a control-loop atlas
- focused address and string context exports for branch-deepening passes
- a reusable CLI RE kit for branch dossiers where the atlas needs deeper grounding
- a stable curated function ledger in `artifacts/exports/rt3-1.06/function-map.csv`

Current coverage is broad enough to support future sessions without rediscovery, especially in:

- CRT startup and bootstrap handoff
- shell frame, layout, presentation, deferred-message, and frontend overlay flow
- Multiplayer.win UI, chat, session-event, and transport ownership
- map/scenario load and text-export paths
- shared support layers such as intrusive queues, vectors, hashed stores, and tracked heaps

README maintenance rule:

- Keep this section at subsystem level only.
- Do not mirror per-pass function additions here.
- Detailed mapping progress belongs in `artifacts/exports/rt3-1.06/function-map.csv` and the derived branch artifacts under `artifacts/exports/rt3-1.06/`.

Current local tool status:

- Ghidra is installed at `~/software/ghidra`
- `~/software/ghidra/ghidraRun` launches successfully in an interactive shell
- Rizin is installed and available on `PATH`
- `winedbg` works with `rt3_wineprefix`
- RT3 launches under `/opt/wine-stable/bin/wine` when started from `rt3_wineprefix/drive_c/rt3`

## Next Focus

The atlas milestone is broad enough that the next implementation focus has already shifted downward
into runtime rehosting. The current runtime baseline now includes deterministic stepping, periodic
trigger dispatch, normalized runtime effects, staged event-record mutation, fixture execution,
state-diff tooling, tracked save-slice documents for captured-runtime inputs, overlay import
documents that combine captured snapshots with save-derived state, and a packed-event persistence
bridge that now reaches per-record summaries and selective executable import.

The highest-value next passes are now:

- preserve the atlas and function map as the source of subsystem boundaries while continuing to
  avoid shell-first implementation bets
- keep using overlay imports as the context bridge when selectively executable packed rows still
  need live company state that save slices do not persist
- treat broader real grouped-descriptor recovery as the active packed-event frontier now that the
  first company-scoped batch already parses, summarizes, and executes through the ordinary runtime
  path when overlay context resolves its symbolic company scope: descriptor `2` `Company Cash`,
  descriptor `13` `Deactivate Company`, and descriptor `16` `Company Track Pieces Buildable`
- descriptors `1` `Player Cash` and `14` `Deactivate Player` now join that executable real batch
  through the same ordinary runtime path, backed by the minimal player runtime and overlay-import
  context
- widen real packed-event executable coverage descriptor by descriptor after identity, target mask,
  and normalized effect semantics are all grounded, not just after row framing is parsed
- the first grounded condition-side unlock now exists for negative-sentinel `raw_condition_id = -1`
  company scopes, and the first ordinary nonnegative condition batch now executes too: numeric
  thresholds for company finance, company track, aggregate territory track, and company-territory
  track
- exact named-territory binding now executes too, while named-territory no-match cases remain the
  explicit binding blocker frontier
- real descriptors `8` `Economic Status`, `9` `Confiscate All`, and `15` `Retire Train` now join
  the executable batch through the same ordinary runtime path, backed by the opaque economic-status
  lane and the minimal event-owned train roster
- descriptor `3` `Territory - Allow All` now executes as company-to-territory access rights through
  the same ordinary runtime path; shell purchase-flow parity remains out of scope, and mixed
  supported/unsupported real rows still stay parity-only
- whole-game ordinary-condition execution now exists too: special-condition thresholds,
  candidate-availability thresholds, and economic-status-code thresholds now gate imported runtime
  records, and the packed-event frontier now reports explicit unmapped world-condition and
  world-descriptor buckets
- that whole-game condition batch is now metadata-driven too: special-condition label ids,
  economic-status, and the generic `%1 Avail.` candidate-availability template plus candidate-name
  side strings all decode through checked-in world-condition metadata instead of fixture-only ids
- the first real whole-game grouped-descriptor batch is now metadata-driven too: checked-in
  descriptor metadata covers special-condition and candidate-availability setters, and descriptor
  `110` `Disable Stock Buying and Selling` now executes too through the checked-in keyed runtime
  flag `world.disable_stock_buying_and_selling`
- that world-toggle path now covers a broader recovered boolean scenario-rule band too:
  descriptors `111..138` now decode through checked-in metadata into either keyed `world_flags`
  or the bounded `world_restore.limited_track_building_amount` scalar for finance/trading,
  construction, and governance restrictions
- the late recovered world-toggle band now executes too where current evidence is equally strong:
  `Use Bio-Accelerator Cars`, `Disable Cargo Economy`, `Disable Train Crashes`,
  `Disable Train Crashes AND Breakdowns`, and `AI Ignore Territories At Startup`
- whole-game ordinary-condition coverage is broader now too: checked-in world-flag condition ids
  can lower into `world_flag_equals` gates for boolean equality/inequality forms, so real packed
  rows can gate whole-game effects on existing `world_flags`
- the tracked parity save-slice now keeps its remaining non-imported residue as structured
  `real_packed_v1` parity records, with the first captured leftover now identified as the
  locomotives-page `Unknown Loco Available` band and moved onto the explicit
  `blocked_unmapped_world_descriptor` frontier
- the next recovered locomotives-page descriptor batch is partially executable too:
  descriptors `454..456` (`All Steam/Diesel/Electric Locos Avail.`) now lower through checked-in
  metadata into keyed `world_flags`, while the wider locomotive availability/cost scalar bands now
  split cleanly between executable scalar availability/cost rows and the remaining world-side
  scalar families
- raw `.smp` inspection/export now reconstructs the persisted save-side named locomotive table and
  derives a minimal locomotive catalog from its row order, so save-slice documents can carry both
  `RuntimeState.named_locomotive_availability` and the catalog context needed for descriptor
  lowering
- recovered scalar locomotive availability and locomotive-cost descriptors now import through that
  save-native or embedded `RuntimeState.locomotive_catalog` context into the ordinary
  `named_locomotive_availability` and `named_locomotive_cost` runtime maps
- cargo-production `230..240` and territory-access-cost `453` now execute too through minimal
  world-side scalar landing surfaces: slot-indexed `cargo_production_overrides` and
  `world_restore.territory_access_cost`
- world-scalar ordinary-condition coverage now matches those runtime surfaces too: checked-in
  metadata lowers named locomotive availability, named locomotive cost, named cargo-production
  slot thresholds, aggregate cargo production, limited-track-building-amount, and
  territory-access-cost rows into explicit runtime condition gates
- the remaining world-scalar condition frontier is now narrow and explicit: unsupported families
  such as `All Factory Production` stay parity-only on `blocked_unmapped_world_condition`
- keep in mind that the current local `.gms` corpus still exports with no packed event collection,
  so real descriptor mapping needs to stay plumbing-first until better captures exist
- use `rrt-hook` primarily as optional capture or integration tooling, not as the first execution
  environment
- keep `docs/runtime-rehost-plan.md` current as the runtime baseline and next implementation slice
  change

Regenerate the initial exports with:

```bash
python3 tools/py/collect_pe_artifacts.py \
  rt3_wineprefix/drive_c/rt3/RT3.exe \
  artifacts/exports/rt3-1.06
```

Regenerate the startup-focused Ghidra exports with:

```bash
python3 tools/py/export_startup_map.py \
  rt3_wineprefix/drive_c/rt3/RT3.exe \
  artifacts/exports/rt3-1.06
```

That default export now walks two roots:

- `entry:0x005a313b`
- `bootstrap:0x00484440`

For a focused branch-deepening pass, regenerate the analysis context exports with:

```bash
python3 tools/py/export_analysis_context.py \
  rt3_wineprefix/drive_c/rt3/RT3.exe \
  artifacts/exports/rt3-1.06 \
  --addr 0x00444dd0 \
  --addr 0x00508730 \
  --addr 0x00508880 \
  --string gpdLabelDB \
  --string gpdCityDB \
  --string 2DLabel.imb \
  --string 2DCity.imb \
  --string "Geographic Labels"
```

For the pending-template dispatch-store branch, regenerate the new branch dossier with:

```bash
python3 tools/py/rt3_rekit.py \
  pending-template-store \
  rt3_wineprefix/drive_c/rt3/RT3.exe \
  artifacts/exports/rt3-1.06
```

That dossier is now a targeted follow-up tool, not the default first pass.