rrt/docs/runtime-rehost-plan.md

# Runtime Rehost Plan

## Goal

Replace the shell-dependent execution path with a bottom-up runtime that can be tested headlessly,
grown incrementally, and later support one or more replacement frontends.

This plan assumes the current shell and presentation path remain unreliable for the near term. We
therefore treat shell recovery as a later adapter problem rather than as the primary execution
milestone.

## Current Baseline

The repo is already past pure scaffolding.

Implemented today:

- `rrt-runtime` exists with a deterministic calendar model, step commands, runtime summaries, and
  normalized runtime state validation
- periodic trigger dispatch exists, including ordered periodic maintenance, dirty rerun `0x0a`, and
  a normalized runtime-effect surface with staged event-record mutation
- snapshots, state dumps, save-slice projection, and normalized state diffing already exist in the
  CLI and fixture layers
- checked-in runtime fixtures already cover deterministic stepping, periodic service, direct trigger
  service, snapshot-backed inputs, save-slice-backed inputs, overlay-import-backed inputs,
  normalized state-fragment assertions, and imported packed-event execution
- overlay imports now preserve selected-company and controller-role context, and the normalized
  company-target model can execute `selected_company`, `human_companies`, and `ai_companies`
  symbolic scopes through the ordinary runtime service path while keeping condition-relative
  company scopes explicitly blocked
- real `0x4e9a` grouped rows now carry checked-in descriptor metadata, semantic family/preview
  summaries, and three recovered executable company-scoped families: descriptor `2` = `Company Cash`,
  descriptor `13` = `Deactivate Company`, and descriptor `16` = `Company Track Pieces Buildable`
- the first grounded condition-side unlock now exists for real packed rows: negative-sentinel
  `raw_condition_id = -1` company scope lowers `condition_true_company` into normalized company
  targets during import
- the first ordinary nonnegative condition-id batch now executes too: numeric-threshold company
  finance, company track, aggregate territory track, and company-territory track rows can import
  through overlay-backed runtime context
- exact named-territory binding now lowers candidate-name ordinary rows onto tracked territory
  names, a minimal player runtime now carries selected-player and role context, and real descriptor
  `1` = `Player Cash` and descriptor `14` = `Deactivate Player` now import and execute through the
  ordinary runtime path
- a first-class chairman-profile runtime now exists too, with overlay-backed selected-chairman
  context and the first chairman-targeted grouped-effect subset: the same real descriptors
  `1` = `Player Cash` and `14` = `Deactivate Player` now also import and execute through the
  hidden grouped target-subject lane when it resolves to selected-chairman scope, while broader
  chairman target scopes remain explicit parity on `blocked_chairman_target_scope`
- chairman governance state is broader now too: companies carry explicit chairman links plus
  book-value-per-share, investor-confidence, management-attitude, and takeover/merger cooldown
  lanes, and the first grounded chairman/control-transfer ordinary-condition batch now imports and
  executes through the same path for selected-chairman cash / holdings / net-worth /
  purchasing-power thresholds plus company book-value-per-share / investor-confidence /
  management-attitude thresholds; real chairman grouped-effect scope ordinals `0..3` now execute
  too as `condition_true`, `selected`, `human`, and `ai`, while wider ordinals remain parity
  frontier
- checked-in save-slice documents can now carry explicit company rosters and chairman profile
  tables too, and runtime projection/import will seed or replace company/chairman context from
  those save-owned surfaces; that lets the currently supported company-targeted and
  chairman-targeted descriptor/condition batches execute from standalone save-slice fixtures
  without overlay snapshots when the checked-in documents include that context, while raw `.gms`
  inspection/export still leaves those company/chairman surfaces absent
- a checked-in `EventEffects` export now exists too at
  `artifacts/exports/rt3-1.06/event-effects-table.json`, and a checked-in semantic closure layer
  now exists at `artifacts/exports/rt3-1.06/event-effects-semantic-catalog.json`
- recovered descriptor rows now classify into explicit semantic frontier buckets such as
  `blocked_shell_owned_descriptor`, `blocked_evidence_blocked_descriptor`, and
  `blocked_variant_or_scope_blocked_descriptor` instead of generic unmapped-descriptor buckets
- the first recovered company-governance descriptor tranche now executes through the generic
  `SetCompanyGovernanceScalar` surface: descriptor `56` `Credit Rating` and descriptor `57`
  `Prime Rate` now import through ordinary company target lowering
- adjacent recovered finance/control-transfer descriptors such as `55` `Stock Prices` and `58`
  `Merger Premium` now land on explicit shell-owned descriptor parity instead of generic unmapped
  descriptor buckets
- the recovered whole-game scalar economy/performance strip `59..104` now has a bounded runtime
  landing surface too: representative descriptors import as `SetWorldScalarOverride` and land in
  `RuntimeState.world_scalar_overrides`
- the grounded aggregate cargo-economics descriptors now execute too: descriptor `105`
  `All Cargo Prices` and descriptors `177..179` `All Cargo Production` / `All Factory Production`
  / `All Farm/Mine Production` import through bounded cargo override surfaces, while the named
  cargo-price and named cargo-production strips now sit on explicit
  `blocked_evidence_blocked_descriptor` parity instead of generic unmapped-descriptor frontier
- a minimal event-owned train surface and an opaque economic-status lane now exist in runtime
  state, and real descriptors `8` = `Economic Status`, `9` = `Confiscate All`, and `15` =
  `Retire Train` now import and execute through the ordinary runtime path when overlay context
  supplies the required train ownership data
- descriptor `3` = `Territory - Allow All` now imports and executes too, reinterpreted as
  company-to-territory access rights instead of a territory-owned policy bit; shell purchase-flow
  and selected-profile parity still remain outside the runtime surface
- the first whole-game ordinary-condition batch now executes too: special-condition thresholds,
  candidate-availability thresholds, and economic-status-code thresholds now gate runtime records
  through the same service path, and whole-game parity frontiers now report explicit unmapped
  world-condition and world-descriptor buckets rather than falling back to the generic ordinary
  or descriptor counts
- checked-in whole-game condition metadata now drives that same world-side condition batch too:
  special-condition thresholds use the real special-condition label ids, economic-status thresholds
  use the checked-in world label id, and candidate-availability thresholds now decode through the
  recovered `%1 Avail.` template plus the packed candidate-name side string instead of fixture-only
  placeholder ids
- checked-in whole-game grouped-descriptor metadata now drives the first real world-side effect
  batch too: real special-condition and candidate-availability setter rows now decode and import
  through the ordinary runtime path, and the first real world-flag family now executes too:
  descriptor `110` `Disable Stock Buying and Selling` lowers through checked-in keyed runtime
  metadata into `world.disable_stock_buying_and_selling`
- that world-side effect batch now covers a broader recovered boolean scenario-rule family too:
  descriptors `111..138` now lower through checked-in metadata into either keyed `world_flags` or
  the bounded `world_restore.limited_track_building_amount` scalar for finance/trading,
  construction, and governance/company-formation restrictions
- the same keyed world-toggle path now also covers the late special-condition band where current
  static evidence stays equally strong: `Use Bio-Accelerator Cars`, `Disable Cargo Economy`,
  `Disable Train Crashes`, `Disable Train Crashes AND Breakdowns`, and
  `AI Ignore Territories At Startup` now import and execute as keyed `world_flags`
- whole-game ordinary-condition decode now covers checked-in world-flag condition ids too for
  boolean equality/inequality forms, so real packed rows can gate whole-game effects on existing
  `world_flags` through `world_flag_equals` without fixture-authored placeholder ids
- the tracked parity save-slice no longer preserves an opaque `unsupported_framing` record; its
  remaining captured residue is now structurally decoded `real_packed_v1` parity state that lands
  in existing explicit blocker buckets, with the first leftover now pinned to the unresolved
  upper locomotives-page availability tail instead of anonymous descriptor residue
- the next recovered locomotives-page descriptor band is now partially executable too:
  descriptors `230..240`, `241..351`, `352..451`, `453`, `457..474`, and `475..502` now carry
  recovered world-side scalar metadata, while descriptors `454..456`
  (`All Steam/Diesel/Electric Locos Avail.`) now execute as keyed `world_flags`
- a first-class named locomotive availability runtime surface now exists too:
  save-slice documents can carry the persisted `[world+0x66b6]` name table into
  `RuntimeState.named_locomotive_availability`, and imported runtime effects can mutate that map
  through the ordinary event-service path without requiring Trainbuy or live locomotive-pool parity
- the recovered locomotives-page availability bands can now import as full scalar overrides
  through `RuntimeState.locomotive_catalog` into `RuntimeState.named_locomotive_availability`;
  raw `.smp` inspection/export now reconstructs the save-side locomotive row family and derives the
  catalog directly into save-slice documents, so standalone save-slice imports can execute those
  rows whenever the save carries enough catalog entries
- the grounded lower locomotive-cost band `352..409` now imports too through the same save-native
  or embedded catalog into the event-owned `RuntimeState.named_locomotive_cost` map when its
  scalar payloads are nonnegative; the unresolved lower tail and upper cost tail now stay on
  explicit parity instead of synthetic execution
- the remaining recovered scalar world families now execute as well: cargo-production `230..240`
  rows lower into slot-indexed `cargo_production_overrides`, and territory-access-cost descriptor
  `453` lowers into `world_restore.territory_access_cost`
- cargo slot identity is now save-native as well: the recipe-book probe derives a bounded
  `cargo_catalog` into save-slice documents and runtime state, so cargo descriptors and
  `%1 Production` conditions can expose stable slot metadata without a live cargo simulation layer
- world-scalar ordinary-condition coverage now aligns with those runtime surfaces too: checked-in
  metadata lowers named locomotive availability, named locomotive cost, named cargo-production
  slot thresholds, aggregate cargo production, factory/farm-mine/other cargo production,
  limited-track-building-amount, and territory-access-cost rows into explicit runtime condition
  gates
- cargo slot classification is now checked in across the full 11-slot recipe-book surface, so the
  remaining world-side frontier is broader descriptor/condition breadth rather than missing cargo
  classification or save/import context

That means the next implementation work is still breadth, not bootstrap. The current descriptor
frontier is no longer anonymous id recovery; it is the remaining recovered-but-nonexecutable
families from the checked-in semantic catalog, especially cargo-price, add-building, and other
descriptor clusters that now have explicit shell-owned or evidence-blocked status but not yet a
bounded executable landing surface. Raw save reconstruction for company/chairman context is still a
later tranche once stronger evidence exists. Richer runtime ownership should still be added only
where a later descriptor or condition family needs more than the current event-owned roster.

## Why This Boundary

Current static analysis points to one important constraint: the existing gameplay cadence is still
nested under shell-owned frame and controller ownership rather than under one clean detached
gameplay loop.

- `simulation_frame_accumulate_and_step_world` is still called from the shell-owned cadence and
  still performs shell-window and presentation-adjacent servicing.
- `shell_service_frame_cycle` owns frame refresh, deferred work, cursor updates, and one-time
  window visibility work.
- the world-view input path and ordinary controller input path still flow through shell-owned
  objects and globals.

That makes shell-first stabilization a poor rewrite target. The better target is the lower runtime:
calendar stepping, periodic world maintenance, scenario event service, runtime persistence, and the
stateful collections beneath them.

## Architecture

The runtime rehost should be split into four layers.

### 1. `rrt-runtime`

Purpose:

- pure runtime state
- deterministic stepping
- scenario and company maintenance
- persistence and normalization boundaries

Constraints:

- no controller-window ownership
- no presentation refresh
- no shell globals in the public model
- no dependency on message pumps or platform input

### 2. `rrt-fixtures`

Purpose:

- fixture schemas
- captured-state loading
- golden summaries and diff helpers
- normalization helpers for comparing original-runtime outputs against rehosted outputs

### 3. `rrt-cli`

Purpose:

- headless runtime driver
- fixture execution commands
- state diff and round-trip tools

This should become the first practical execution surface for the new runtime.

### 4. Future adapter layers

Possible later crates:

- `rrt-shell-adapter`
- `rrt-ui`
- `rrt-hook` capture bridges

These should adapt to `rrt-runtime`, not own the core simulation model.

## Rewrite Principles

1. Prefer state-in, state-out functions over shell-owned coordinators.
2. Choose narrow vertical slices that can be verified with fixtures.
3. Treat persistence boundaries as assets, because they give us reproducible test inputs.
4. Normalize state aggressively before diffing so comparisons stay stable.
5. Do not rehost shell or input code until the lower runtime already has a trustworthy step API.

## Candidate Boundaries

Good early targets:

- calendar-point arithmetic and step quantization helpers
- `simulation_advance_to_target_calendar_point`
- `simulation_service_periodic_boundary_work`
- `scenario_event_collection_service_runtime_effect_records_for_trigger_kind`
- `world_load_saved_runtime_state_bundle`
- `world_runtime_serialize_smp_bundle`
- company and placed-structure refresh helpers that look like collection or state transforms

Poor early targets:

- `simulation_frame_accumulate_and_step_world`
- `world_entry_transition_and_runtime_bringup`
- `shell_controller_window_message_dispatch`
- world-view camera and cursor service helpers
- shell window constructors or message handlers

## Milestones

### Milestone 0: Scaffolding (complete)

Goal:

- create the workspace shape for bottom-up runtime work
- define fixture formats and CLI entrypoints
- make the first headless command runnable even before the runtime is featureful

Deliverables:

- new crate `crates/rrt-runtime`
- new crate `crates/rrt-fixtures`
- `rrt-cli` subcommands for runtime and fixture work
- initial fixture file format checked into the repo
- baseline docs for state normalization and comparison policy

Exit criteria:

- `cargo run -p rrt-cli -- runtime validate-fixture <path>` works
- one sample fixture parses and normalizes successfully
- the new crates build in the workspace

### Milestone 1: Deterministic Step Kernel (complete)

Goal:

- stand up a minimal runtime state and one deterministic stepping API
- prove that a world can advance without any shell or presentation owner

Deliverables:

- calendar-point representation in Rust
- reduced `RuntimeState` model sufficient for stepping
- one `advance_to_target_calendar_point` execution path
- deterministic smoke fixtures
- human-readable state diff output

Exit criteria:

- one minimal fixture can advance to a target point and produce stable repeated output
- the same fixture can run for N steps with identical results across repeated runs
- state summaries cover the calendar tuple and a small set of world counters

### Milestone 2: Periodic Service Kernel (partially complete)

Goal:

- add recurring maintenance and trigger dispatch on top of the first step kernel

Deliverables:

- periodic boundary service modes
- trigger-kind dispatch scaffolding
- the first runtime-effect service path behind a stable API
- fixture coverage for one or two trigger kinds

Exit criteria:

- one fixture can execute periodic maintenance without shell state
- trigger-kind-specific effects can be observed in a normalized diff

Current status:

- periodic trigger ordering is implemented
- normalized trigger-side effects already exist for world flags, company cash/debt, candidate
  availability, and special conditions
- one-shot handling, dirty reruns, and staged append/activate/deactivate/remove behavior are
  already covered by synthetic fixtures
- the remaining breadth is richer trigger-family behavior and target resolution, not first-pass
  event-graph mutation

### Milestone 3: Persistence Boundary (partially complete)

Goal:

- load and save enough runtime state to support realistic fixtures

Deliverables:

- serializer and loader support for a narrow `.smp` subset or an equivalent normalized fixture view
- round-trip tests
- versioned normalization rules

Exit criteria:

- one captured runtime fixture can be round-tripped with stable normalized output

Current status:

- runtime snapshots and state dumps are implemented
- `.smp` save inspection and partial save-slice projection already feed normalized runtime state
- the packed event-collection bridge now carries per-record summaries into loaded save slices,
  projected runtime snapshots, normalized diffs, and fixtures
- the first decoded packed-event subset can now import into executable runtime records when the
  decoded actions fit the current normalized runtime-effect model
- tracked save-slice documents now provide a repo-friendly captured-runtime path without checking in
  raw `.smp` binaries
- overlay-backed captured-runtime inputs now provide enough runtime company context for symbolic
  selected-company and controller-role scopes without inventing company state from save bytes alone
- aggregate territory context and company-territory track counters now flow through tracked overlay
  snapshots, named-territory binding now executes on exact matches, and a minimal player runtime is
  now present, so the remaining gap is broader ordinary condition-id coverage beyond numeric
  thresholds plus wider real grouped-descriptor and territory-policy semantic coverage, not
  first-pass captured-runtime plumbing

### Milestone 4: Domain Expansion

Goal:

- add the minimum missing subsystems needed by failing fixtures

Likely order:

- company maintenance
- scenario event service breadth
- placed-structure local-runtime refresh
- candidate and cargo-service tables
- locomotive availability refresh

Exit criteria:

- representative fixtures from multiple subsystems can step and diff without shell ownership

### Milestone 5: Adapter and Frontend Re-entry

Goal:

- connect the runtime core to external control surfaces

Possible outputs:

- shell-compatibility adapter
- replacement CLI workflows
- replacement UI or tool surfaces

Exit criteria:

- external commands operate by calling runtime APIs rather than by reaching into shell-owned state

## Fixture Strategy

We should maintain three fixture classes from the start.

### `minimal-world`

Small synthetic fixtures for deterministic kernel testing.

Use for:

- calendar stepping
- periodic maintenance
- invariants and smoke tests

### `captured-runtime`

Fixtures captured from the original process or from serialized runtime state.

Use for:

- parity checks
- subsystem-specific debugging
- rehosted function validation

### `roundtrip-save`

Persistence-oriented fixtures built around real save data or normalized equivalents.

Use for:

- serializer validation
- normalization rules
- regression tests

Each fixture should contain:

- metadata
- format version
- source provenance
- input state
- command list
- expected summary
- optional expected normalized full state
- optional expected normalized state fragment when only part of the final state matters

## Normalization Policy

Runtime diffs will be noisy unless we define a normalization layer early.

Normalize away:

- pointer addresses
- allocation order
- container iteration order when semantically unordered
- shell-only dirty flags or presentation counters
- timestamps that are not semantically relevant to the tested behavior

Keep:

- calendar tuple
- company and world ids
- cash, debt, and game-speed-related runtime fields when semantically relevant
- collection contents and semantic counts
- trigger-side effects
- packed event-collection structural summaries when present

## Risks

### Hidden shell coupling

Some lower functions still touch shell globals indirectly. We should isolate those reads quickly and
replace them with explicit runtime inputs where possible.

### Fixture incompleteness

Captured state that omits one manager table can make deterministic functions appear unstable. The
fixture format should make missing dependencies obvious.

### Over-scoped early rewrites

The first two milestones should remain deliberately small. Do not pull in company UI, world-view
camera work, or shell windows just because their names are nearby in the call graph.

## Implemented Baseline

The currently implemented normalized runtime surface is:

- `CalendarPoint`, `RuntimeState`, `StepCommand`, `StepResult`, and `RuntimeSummary`
- fixture loading from inline state, snapshots, and state dumps
- `runtime validate-fixture`, `runtime summarize-fixture`, `runtime export-fixture-state`,
  `runtime summarize-state`, `runtime import-state`, and `runtime diff-state`
- deterministic stepping, periodic trigger dispatch, one-shot event handling, dirty reruns, and a
  normalized runtime-effect vocabulary with staged event-record mutation
- save-side inspection and partial state projection for `.smp` inputs, including per-record packed
  event summaries and selective executable import
- tracked save-slice documents plus save-slice-backed fixture loading for captured-runtime coverage

Checked-in fixture families already include:

- deterministic minimal-world stepping
- periodic boundary service
- direct trigger-service mutation
- staged event-record lifecycle coverage
- snapshot-backed fixture execution

## Next Slice

The recommended next implementation slice is broader ordinary-condition and grouped-descriptor
breadth on top of the now-stable numeric-threshold, whole-game-state, overlay-context,
named-territory, player, world/train, and company-territory-access batches.

Target behavior:

- preserve the current proof set for real ordinary-condition execution:
  company finance, company track, aggregate territory track, and company-territory track numeric
  thresholds all pass through parse, semantic summary, overlay-backed import, and ordinary trigger
  execution
- extend ordinary condition coverage beyond the current numeric-threshold and whole-game-state
  families only when comparator semantics, runtime ownership, and binding rules are grounded
  enough to lower honestly into the normalized runtime path
- keep named-territory ordinary rows on exact case-sensitive binding until captured evidence
  justifies alias tables or fuzzier matching
- keep player-owned condition scope within the minimal event runtime model until later slices need
  richer player metrics or profile/chairman ownership
- continue widening real grouped-descriptor execution only when both descriptor identity and
  runtime effect semantics are grounded enough to map into the normalized runtime path honestly
- keep descriptor `3` on the now-executable company-territory-access interpretation; do not drift
  back into territory-owned policy wording without new contrary evidence

Public-model expectations for that slice:

- additional checked-in ordinary-condition metadata entries beyond the current numeric-threshold
  allowlist
- richer ordinary-condition metadata and later runtime ownership only where new condition domains
  still remain blocked after the current named-territory and player-runtime unlocks
- more selective real-row `decoded_conditions` and `decoded_actions` only where the
  condition/effect-to-runtime mapping is supported end to end

Fixture work for that slice:

- preserve the new ordinary-condition tracked overlays for executable company finance, company
  track, aggregate territory track, and company-territory track thresholds
- preserve the new whole-game tracked save-slice fixtures for executable special-condition and
  candidate-availability/economic-status batches, plus the parity-only world-condition and
  world-descriptor frontier sample
- preserve the named-territory no-match tracked overlay as the explicit binding blocker frontier
- preserve the territory-access tracked overlays and parity samples so descriptor `3` access-rights
  execution does not regress while other grouped families widen
- keep the older negative-sentinel, mixed real-row, and company-scoped descriptor fixtures green so
  ordinary-condition breadth does not regress descriptor-side execution
- keep synthetic harness, save-slice, and overlay paths green as the real descriptor surface widens

Current local constraint:

- the local checked-in and on-disk `.gms` corpus still does not provide a richer captured packed
  event save set, so wider ordinary-condition and descriptor recovery still needs to rely on the
  grounded static tables and tracked JSON artifacts until new captures exist

Do not mix this slice with:

- shell queue/modal behavior
- shell territory-access purchase or selected-profile parity
- speculative condition execution without grounded runtime ownership
- speculative executable import for real rows whose descriptor meaning is still weak