sausage/docs/SAU-384-rehearsal.md

# Disposable handover rehearsal — SAU-384

Authority: [/docs/design/plugin-handover/design.md](/docs/design/plugin-handover/design.md), Rollout; [/docs/symphony/artifacts/SAU-356/plan.md](/docs/symphony/artifacts/SAU-356/plan.md), Approval and operator gates.

**Status: blocked, not a live execution approval.** Rehearsal evidence and the reviewed target bundle are under [/docs/symphony/artifacts/SAU-384](/docs/symphony/artifacts/SAU-384). No shipped plugin or host behavior changes on this branch. The temporary migration implementation and native controller were executed only in a throwaway box and deleted after the successful migration verification. Their hashes/deletion record and source-as-log remain, not an executable migration tool.

## Required resolution before completing the kit

1. Pin the actual external host launcher/service owner, stop/start interface, host architecture and runtime closure. Those are not projected into this world. The supplied local native artifact is **aarch64-linux**, dynamically linked to the Nix paths printed in the build evidence; it is NOT a verified live-host package. Do not guess a systemd unit, use `core.restart` as stop-all, or install a mismatching binary.
2. Resolved rollback choice: **unsubscribing works; merely re-keying the registration does not**. An actual pre-best-effort session component, reading a registration written by the target component, ignores `best_effort` and sends history to its new ID. Rehearsal: unsubscribe = one barrier callback and no gateway-shaped replay; re-key = barrier plus historical replay. The test used the actual old manager and a native callback sink, not browser SSE. The runbook now makes unsubscribe mandatory, using the card's authorized option. Do not claim an ID rename is safe.
3. **Closed in run 1369:** armed/exit-race delay-gate tests passed on real core with actual target WASM; replacement orchestrator readiness/one-owner/all-watch arming passed; actual old-manager/gateway rollback without re-registration reproduced unsubscribe safety and ID-only re-key replay. See [resume evidence](SAU-384-resume-1369.md). The delay/observation hooks were only in a disposable host copy, not the selected host artifact or committed code.

Use the card's explicit **unsubscribe** rollback option; an ID rename is not a safe alternative. The remaining blocker is the actual external host launcher/native installation manifest. Read-only SSH credential request **055995a51608c57dd25b323bea731fcb** is approved, but connection was refused before authentication for an unknown Citadel host. Mediator must add the observed public host key, then resume inspection; workers may not change live config. See [run 1372](SAU-384-resume-1372.md). No new approval request or live stop/restart permission is needed/granted.

## What was built and exercised

- Host source `e09c4f9f0ef6b5833d6be066771eecd9e0a1f572`. Disposable Nix build, native workspace suite passed after correcting the cold SDK-probe dependency cache. Target native binary was built and stripped, not released/deployed. CLI `--version` passed. Native controller used the real `sausage::Host`, real VFS, watches, locks, service supervisor and WASM execution; model/skills/memory/ledger/push effects were synthetic or loopback only.
- Reviewed session `cb844151` / SAU-409 final; orchestrator `c80a5854` / SAU-408; web `d883fbf2` / SAU-403; agent/board retained reviewed components. Fresh card-worktree builds: mail **1600 / 53 tests**, cred **1601 / 272**, cron **1602 / 24**, rooms **1603 / 59**, all passed. No activation. Component/source provenance and every bundled file hash are in `evidence/manifest.json`.
- Frontend is exactly approved commit `3d76026749a5d21722033bbfe8743ac368946b0f`, retained production build from SAU-375 rework-1318. It remains `publish:hold`; no publication was attempted.
- World SDK runtime sources already matched the selected host SDK. The only needed source-copy delta is **`src/lib.rs`**, documenting effective grace. Preserve world-owned Cargo manifests, relocated `interface`, and imports. A first broad SDK-copy attempt incorrectly included repository-relative Cargo paths; comparison caught this before any live change, and the final migration used only the one-file patch. Never copy the repository's SDK Cargo manifests over the world SDK.
- Accepted SAU-362 byte snapshot: **1,491 sessions / 580,355,405 bytes**, including **70,689,796-byte default history**. Pristine snapshot remained unpreopened. Executable clones used synthetic model settings; their historical byte volume was retained. Source snapshot consistency is per-file, not a transactional live snapshot.

## Successful checks and limits

| Check | Evidence/result |
|---|---|
| Old ownership → stop-all → offline swap → target boot | Old `mail.start`, old session `send` and detached ledger processes inventoried. Native external controller awaited `Host.stop`, verified zero remaining processes and host exit, then backed up/transformed/swapped. Final shutdown **0.0958 s** in this fixture. One target manager, no mail deliverer, cred locked. |
| Partial swap + rerun | Injected interruption after operation 3; hash-guarded completion and repeated installed-phase apply preserved all **4,374 non-target inventoried files**. No live/new work reset. |
| Exact losses/preservation | Only `pending.jsonl`, `cancel`, and unclaimed pending non-Surma delivery rows selected for removal. Original mail messages, done/Surma copies and uncertain `unknown` deliveries retained without replay; scheduled SQL IDs, calendar/outcome fields and non-session JSON bytes retained. Synthetic grant DB bytes preserved. Loss inventory backs every removed file/row. |
| Offline failure boundaries | Before-start rollback restored exact bytes; conflicting outside edits refused; ambiguous legacy `interrupt:true` refused without mutation. After-start apply and whole-world rollback both refused. |
| Default/advisor HTTP replay | Three identical same-ID POSTs each, before/after consumption; exactly one answered run per session. Default returned null on repeats; advisor returned its same receipt. |
| Explicit open target run | A run-start/consumed record with a finished-looking assistant tail, but no run-end, recovered under the **same agent-run UUID**, with one start. Legacy finished-looking history alone produced no target event/run. |
| Real VFS/runner-boundary work | Actual SDK `armed` then plugin-write `changed`; five callback-boundary runs. Run 1369 additionally forced publication before runner `armed` and after its empty-mailbox exit decision while detached completion was held; no lost/duplicate inputs (disposable host delay gates, actual target WASM). |
| Real core detached errors | Full nested error `data` survived a detached `core.call`. |
| Real cron research deadline | **305.427 s** send→stopped in the final gate; real cron invoked warning and Stop. The 305-second test deadline is the existing 300-second warning interval plus design grace 5 seconds, not a new default. Held inference was cut, no follow-up run. Original configured research default remains 1,200 seconds. |
| Effective grace / crash supervision | Passing native host suite exercises real WASI uniform-grace and before/after-readiness crash-supervision fixtures. Deliberate `core.kill` is NOT a crash: it disables that boot service; an initial rehearsal expecting its restart failed, correctly, and was replaced by the existing crash tests. |
| Orchestrator upgrade/idle | Real file-watcher upgrade, FIFO singleton release, old exit before one replacement start event. Paused empty-board/live-size-world host CPU **0.02 s / 20.0002 s ≈ 0.10%**. This is whole-host idle CPU, not per-plugin CPU or live active-work load. Run 1369 additionally verified target acknowledgement, one owner and all 13 core watches armed after old exit. |

### Live-size latency after real records exist

After producing actual run-start/run-end/mailbox-consumed records, subsequent full boot measured **0.544 s** from manager service-start to the next boot service-start (includes discovery/ack overhead). Narrow registration catch-up samples ranged **0.0239 s to 12.0419 s** in the integrated world: the long sample coincided with other synthetic recovery/surfacing work, and the untagged sink cannot isolate attribution. Do not present this as a bounded manager-only benchmark.

The clean final live-size end-to-end gate measured manager acknowledgement **0.6078 s**, registration catch-up **0.0128 s**, and `mail.send` publication to default **0.5376 s**. The integrated post-record mail sample was **0.5235 s**. Both publication samples are below the existing orchestrator **3 s** and cred **10 s** caller budgets (SAU-400 report cites the originating SAU-366 review). These warm-cache synthetic-loopback real-core observations are not live latency guarantees. First scanning still delays first new input after the fast boot acknowledgement.

## Resume verification

Run 1369 forced recompilation and repeated the complete workspace suite on the restored original host source: pass, including **208 host integration tests**. All **144 source files** matched the selected source snapshot. Full log `/logs/box/2026-10-31/tc-asa3yr-exec-3838117964.stderr` (host tests lines 64–434; equivalence line 785). New tests and their limitations are detailed in `SAU-384-resume-1369.md`; new test controllers and instrumentation were deleted after successful checks. Original bundle and production source are unchanged.

## Evidence and failures

All following logs are under `/logs/box/2026-10-31/`:

- Full workspace pass: `tc-asa3yr-exec-2320283969.stderr`. First cold-cache/full-load failure: `tc-asa3yr-exec-158345583.stderr`, missing offline async-task and a queued-build timing test. Populated the separate SDK-probe cache; entire suite then passed unchanged.
- Final stop/swap/start: `tc-asa3yr-exec-1922130515.stderr`; offline fault/grant preservation: `tc-asa3yr-exec-616022371.stderr`.
- Final live-size HTTP/deadline gate: `tc-asa3yr-exec-1243993190.stderr`. Earlier malformed mail `to` scalar, 30-second startup observation bound and blank JSONL parser mistakes were harness errors; corrected to a sequence, existing 120-second test budget and blank-line skipping. Final gate passed.
- Open-run recovery and boundary runs: `tc-asa3yr-exec-903419814.stderr` (subsequent console-watch attempt failed). A console is not a running watch caller; the final SDK guest watch passed in `tc-asa3yr-exec-3261552495.stderr`. Same log retains the real upgrade trace and idle sample. An identical already-acknowledged registration correctly produced no second catch-up; the final probe used a fresh registration identity.
- Actual old-manager unsubscribe/re-key comparison: `tc-asa3yr-exec-3307393394.stderr`.
- Native artifact architecture/runtime paths: `tc-asa3yr-exec-899103518.stderr`; bundle build/CLI: `tc-asa3yr-exec-1075679130.stderr`.
- Executed temporary tool and controller **as log only**, followed by deletion proof: `tc-asa3yr-exec-2902726782.stderr` (31,475 bytes). `evidence/tool-deletion.json` lists removed executables. No transition code was put in a shipped plugin.

The initial broad-copy fixture and earlier failed tests are not the final kit. `evidence/migration-plan.json` identifies the final verified migration. Provider billing/cancellation, power loss, hung host filesystem behavior, physical browsers and independent per-plugin recovery follow-ups are not claimed complete. No live DB was opened; the accepted session snapshot was reused. No live config/SDK change, stop, activation, release, restart, push, deployment or `/www` write occurred.

See [operator runbook](SAU-384-runbook.md) for approval gates and the deliberately blocked live steps.