rrt/docs/runtime-rehost-plan.md

Runtime Rehost Plan

Goal

Replace the shell-dependent execution path with a bottom-up runtime that can be tested headlessly, grown incrementally, and later support one or more replacement frontends.

This plan assumes the current shell and presentation path remain unreliable for the near term. We therefore treat shell recovery as a later adapter problem rather than as the primary execution milestone.

Current Baseline

The repo is already past pure scaffolding.

Implemented today:

• rrt-runtime exists with a deterministic calendar model, step commands, runtime summaries, and normalized runtime state validation
• periodic trigger dispatch exists, including ordered periodic maintenance, dirty rerun 0x0a, and a normalized runtime-effect surface with staged event-record mutation
• snapshots, state dumps, save-slice projection, and normalized state diffing already exist in the CLI and fixture layers
• checked-in runtime fixtures already cover deterministic stepping, periodic service, direct trigger service, snapshot-backed inputs, save-slice-backed inputs, overlay-import-backed inputs, normalized state-fragment assertions, and imported packed-event execution
• overlay imports now preserve selected-company and controller-role context, and the normalized company-target model can execute selected_company, human_companies, and ai_companies symbolic scopes through the ordinary runtime service path while keeping condition-relative company scopes explicitly blocked
• real 0x4e9a grouped rows now carry checked-in descriptor metadata, semantic family/preview summaries, and three recovered executable company-scoped families: descriptor 2 = Company Cash, descriptor 13 = Deactivate Company, and descriptor 16 = Company Track Pieces Buildable
• the first grounded condition-side unlock now exists for real packed rows: negative-sentinel raw_condition_id = -1 company scope lowers condition_true_company into normalized company targets during import, while player and territory scope variants remain parity-visible and explicitly blocked
• the first ordinary nonnegative condition-id batch now executes too: numeric-threshold company finance, company track, aggregate territory track, and company-territory track rows can import through overlay-backed runtime context, while named-territory bindings stay parity-only and player-owned condition scope still has no runtime owner

That means the next implementation work is breadth, not bootstrap. The recommended next slice is broader ordinary condition-id coverage beyond numeric thresholds, plus runtime ownership for the still-blocked player-scoped and named-territory condition families, alongside wider real grouped-descriptor coverage beyond the current company-scoped batch.

Why This Boundary

Current static analysis points to one important constraint: the existing gameplay cadence is still nested under shell-owned frame and controller ownership rather than under one clean detached gameplay loop.

• simulation_frame_accumulate_and_step_world is still called from the shell-owned cadence and still performs shell-window and presentation-adjacent servicing.
• shell_service_frame_cycle owns frame refresh, deferred work, cursor updates, and one-time window visibility work.
• the world-view input path and ordinary controller input path still flow through shell-owned objects and globals.

That makes shell-first stabilization a poor rewrite target. The better target is the lower runtime: calendar stepping, periodic world maintenance, scenario event service, runtime persistence, and the stateful collections beneath them.

Architecture

The runtime rehost should be split into four layers.

1. rrt-runtime

Purpose:

• pure runtime state
• deterministic stepping
• scenario and company maintenance
• persistence and normalization boundaries

Constraints:

• no controller-window ownership
• no presentation refresh
• no shell globals in the public model
• no dependency on message pumps or platform input

2. rrt-fixtures

Purpose:

• fixture schemas
• captured-state loading
• golden summaries and diff helpers
• normalization helpers for comparing original-runtime outputs against rehosted outputs

3. rrt-cli

Purpose:

• headless runtime driver
• fixture execution commands
• state diff and round-trip tools

This should become the first practical execution surface for the new runtime.

4. Future adapter layers

Possible later crates:

• rrt-shell-adapter
• rrt-ui
• rrt-hook capture bridges

These should adapt to rrt-runtime, not own the core simulation model.

Rewrite Principles

1. Prefer state-in, state-out functions over shell-owned coordinators.
2. Choose narrow vertical slices that can be verified with fixtures.
3. Treat persistence boundaries as assets, because they give us reproducible test inputs.
4. Normalize state aggressively before diffing so comparisons stay stable.
5. Do not rehost shell or input code until the lower runtime already has a trustworthy step API.

Candidate Boundaries

Good early targets:

• calendar-point arithmetic and step quantization helpers
• simulation_advance_to_target_calendar_point
• simulation_service_periodic_boundary_work
• scenario_event_collection_service_runtime_effect_records_for_trigger_kind
• world_load_saved_runtime_state_bundle
• world_runtime_serialize_smp_bundle
• company and placed-structure refresh helpers that look like collection or state transforms

Poor early targets:

• simulation_frame_accumulate_and_step_world
• world_entry_transition_and_runtime_bringup
• shell_controller_window_message_dispatch
• world-view camera and cursor service helpers
• shell window constructors or message handlers

Milestones

Milestone 0: Scaffolding (complete)

Goal:

• create the workspace shape for bottom-up runtime work
• define fixture formats and CLI entrypoints
• make the first headless command runnable even before the runtime is featureful

Deliverables:

• new crate crates/rrt-runtime
• new crate crates/rrt-fixtures
• rrt-cli subcommands for runtime and fixture work
• initial fixture file format checked into the repo
• baseline docs for state normalization and comparison policy

Exit criteria:

• cargo run -p rrt-cli -- runtime validate-fixture <path> works
• one sample fixture parses and normalizes successfully
• the new crates build in the workspace

Milestone 1: Deterministic Step Kernel (complete)

Goal:

• stand up a minimal runtime state and one deterministic stepping API
• prove that a world can advance without any shell or presentation owner

Deliverables:

• calendar-point representation in Rust
• reduced RuntimeState model sufficient for stepping
• one advance_to_target_calendar_point execution path
• deterministic smoke fixtures
• human-readable state diff output

Exit criteria:

• one minimal fixture can advance to a target point and produce stable repeated output
• the same fixture can run for N steps with identical results across repeated runs
• state summaries cover the calendar tuple and a small set of world counters

Milestone 2: Periodic Service Kernel (partially complete)

Goal:

• add recurring maintenance and trigger dispatch on top of the first step kernel

Deliverables:

• periodic boundary service modes
• trigger-kind dispatch scaffolding
• the first runtime-effect service path behind a stable API
• fixture coverage for one or two trigger kinds

Exit criteria:

• one fixture can execute periodic maintenance without shell state
• trigger-kind-specific effects can be observed in a normalized diff

Current status:

• periodic trigger ordering is implemented
• normalized trigger-side effects already exist for world flags, company cash/debt, candidate availability, and special conditions
• one-shot handling, dirty reruns, and staged append/activate/deactivate/remove behavior are already covered by synthetic fixtures
• the remaining breadth is richer trigger-family behavior and target resolution, not first-pass event-graph mutation

Milestone 3: Persistence Boundary (partially complete)

Goal:

• load and save enough runtime state to support realistic fixtures

Deliverables:

• serializer and loader support for a narrow .smp subset or an equivalent normalized fixture view
• round-trip tests
• versioned normalization rules

Exit criteria:

• one captured runtime fixture can be round-tripped with stable normalized output

Current status:

• runtime snapshots and state dumps are implemented
• .smp save inspection and partial save-slice projection already feed normalized runtime state
• the packed event-collection bridge now carries per-record summaries into loaded save slices, projected runtime snapshots, normalized diffs, and fixtures
• the first decoded packed-event subset can now import into executable runtime records when the decoded actions fit the current normalized runtime-effect model
• tracked save-slice documents now provide a repo-friendly captured-runtime path without checking in raw .smp binaries
• overlay-backed captured-runtime inputs now provide enough runtime company context for symbolic selected-company and controller-role scopes without inventing company state from save bytes alone
• aggregate territory context and company-territory track counters now flow through tracked overlay snapshots, so the remaining gap is broader ordinary condition-id coverage beyond numeric thresholds, named-territory binding, player runtime ownership, and wider real grouped-descriptor semantic coverage, not first-pass captured-runtime plumbing

Milestone 4: Domain Expansion

Goal:

• add the minimum missing subsystems needed by failing fixtures

Likely order:

• company maintenance
• scenario event service breadth
• placed-structure local-runtime refresh
• candidate and cargo-service tables
• locomotive availability refresh

Exit criteria:

• representative fixtures from multiple subsystems can step and diff without shell ownership

Milestone 5: Adapter and Frontend Re-entry

Goal:

• connect the runtime core to external control surfaces

Possible outputs:

• shell-compatibility adapter
• replacement CLI workflows
• replacement UI or tool surfaces

Exit criteria:

• external commands operate by calling runtime APIs rather than by reaching into shell-owned state

Fixture Strategy

We should maintain three fixture classes from the start.

minimal-world

Small synthetic fixtures for deterministic kernel testing.

Use for:

• calendar stepping
• periodic maintenance
• invariants and smoke tests

captured-runtime

Fixtures captured from the original process or from serialized runtime state.

Use for:

• parity checks
• subsystem-specific debugging
• rehosted function validation

roundtrip-save

Persistence-oriented fixtures built around real save data or normalized equivalents.

Use for:

• serializer validation
• normalization rules
• regression tests

Each fixture should contain:

• metadata
• format version
• source provenance
• input state
• command list
• expected summary
• optional expected normalized full state
• optional expected normalized state fragment when only part of the final state matters

Normalization Policy

Runtime diffs will be noisy unless we define a normalization layer early.

Normalize away:

• pointer addresses
• allocation order
• container iteration order when semantically unordered
• shell-only dirty flags or presentation counters
• timestamps that are not semantically relevant to the tested behavior

Keep:

• calendar tuple
• company and world ids
• cash, debt, and game-speed-related runtime fields when semantically relevant
• collection contents and semantic counts
• trigger-side effects
• packed event-collection structural summaries when present

Risks

Hidden shell coupling

Some lower functions still touch shell globals indirectly. We should isolate those reads quickly and replace them with explicit runtime inputs where possible.

Fixture incompleteness

Captured state that omits one manager table can make deterministic functions appear unstable. The fixture format should make missing dependencies obvious.

Over-scoped early rewrites

The first two milestones should remain deliberately small. Do not pull in company UI, world-view camera work, or shell windows just because their names are nearby in the call graph.

Implemented Baseline

The currently implemented normalized runtime surface is:

• CalendarPoint, RuntimeState, StepCommand, StepResult, and RuntimeSummary
• fixture loading from inline state, snapshots, and state dumps
• runtime validate-fixture, runtime summarize-fixture, runtime export-fixture-state, runtime summarize-state, runtime import-state, and runtime diff-state
• deterministic stepping, periodic trigger dispatch, one-shot event handling, dirty reruns, and a normalized runtime-effect vocabulary with staged event-record mutation
• save-side inspection and partial state projection for .smp inputs, including per-record packed event summaries and selective executable import
• tracked save-slice documents plus save-slice-backed fixture loading for captured-runtime coverage

Checked-in fixture families already include:

• deterministic minimal-world stepping
• periodic boundary service
• direct trigger-service mutation
• staged event-record lifecycle coverage
• snapshot-backed fixture execution

Next Slice

The recommended next implementation slice is broader ordinary-condition breadth on top of the now-stable numeric-threshold, overlay-context, and current company-scoped real-descriptor batch.

Target behavior:

• preserve the current proof set for real ordinary-condition execution: company finance, company track, aggregate territory track, and company-territory track numeric thresholds all pass through parse, semantic summary, overlay-backed import, and ordinary trigger execution
• extend ordinary condition coverage beyond numeric thresholds only when comparator semantics, runtime ownership, and binding rules are grounded enough to lower honestly into the normalized runtime path
• keep named-territory ordinary rows explicit and parity-visible until candidate-name territory binding is grounded
• keep player-owned condition scope explicit and parity-visible until there is a first-class player runtime model
• continue widening real grouped-descriptor execution only when both descriptor identity and runtime effect semantics are grounded enough to map into the normalized runtime path honestly

Public-model expectations for that slice:

• additional checked-in ordinary-condition metadata entries beyond the current numeric-threshold allowlist
• richer runtime ownership for still-blocked condition domains such as named territory and player scope
• more selective real-row decoded_conditions and decoded_actions only where the condition/effect-to-runtime mapping is supported end to end

Fixture work for that slice:

• preserve the new ordinary-condition tracked overlays for executable company finance, company track, aggregate territory track, and company-territory track thresholds
• preserve the named-territory tracked overlay as the explicit binding blocker frontier
• keep the older negative-sentinel, mixed real-row, and company-scoped descriptor fixtures green so ordinary-condition breadth does not regress descriptor-side execution
• keep synthetic harness, save-slice, and overlay paths green as the real descriptor surface widens

Current local constraint:

• the local checked-in and on-disk .gms corpus still does not provide a richer captured packed event save set, so wider ordinary-condition and descriptor recovery still needs to rely on the grounded static tables and tracked JSON artifacts until new captures exist

Do not mix this slice with:

• shell queue/modal behavior
• territory-access or selected-profile parity
• speculative condition execution without grounded runtime ownership
• speculative executable import for real rows whose descriptor meaning is still weak