surma/sausage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record disposable handover kit, rehearsal evidence and unresolved rollout gates

Browse Source 
Card: SAU-384
This commit is contained in:
SAU-384 implementer
2026-10-31 13:12:50 +00:00
parent e09c4f9f0e
commit 37e216a3c8
2 changed files with 146 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/SAU-384-rehearsal.md
+59
View File
@@ -0,0 +1,59 @@
# Disposable handover rehearsal — SAU-384
Authority: [/docs/design/plugin-handover/design.md](/docs/design/plugin-handover/design.md), Rollout; [/docs/symphony/artifacts/SAU-356/plan.md](/docs/symphony/artifacts/SAU-356/plan.md), Approval and operator gates.
**Status: blocked, not a live execution approval.** Rehearsal evidence and the reviewed target bundle are under [/docs/symphony/artifacts/SAU-384](/docs/symphony/artifacts/SAU-384). No shipped plugin or host behavior changes on this branch. The temporary migration implementation and native controller were executed only in a throwaway box and deleted after the successful migration verification. Their hashes/deletion record and source-as-log remain, not an executable migration tool.
## Required resolution before completing the kit
1. Pin the actual external host launcher/service owner, stop/start interface, host architecture and runtime closure. Those are not projected into this world. The supplied local native artifact is **aarch64-linux**, dynamically linked to the Nix paths printed in the build evidence; it is NOT a verified live-host package. Do not guess a systemd unit, use `core.restart` as stop-all, or install a mismatching binary.
2. Correct the rollback alternative: **unsubscribing works; merely re-keying the registration does not**. An actual pre-best-effort session component, reading a registration written by the target component, ignores `best_effort` and sends history to its new ID. Rehearsal: unsubscribe = one barrier callback and no gateway-shaped replay; re-key = barrier plus historical replay. The test used the actual old manager and a native callback sink, not browser SSE. Either make unsubscribe mandatory, or specify what stronger meaning of “re-key” is intended. Do not claim an ID rename is safe.
3. Complete remaining integrated acceptance: deterministic manager armed/exit-race injection on real core (the real callback-boundary sequence and primitive watch tests are not that injection), replacement orchestrator readiness after the observed upgrade, and old-manager/gateway rollback without re-registration. The old orchestrator's exit preceded the core's replacement service-start event; the immediate process snapshot still had no new owner, so readiness is not claimed.
The safe unsubscribe option can be selected without inventing compatibility code. The runbook nevertheless withholds a live-ready claim until the card's rollback wording and host installation manifest are unambiguous.
## What was built and exercised
- Host source `e09c4f9f0ef6b5833d6be066771eecd9e0a1f572`. Disposable Nix build, native workspace suite passed after correcting the cold SDK-probe dependency cache. Target native binary was built and stripped, not released/deployed. CLI `--version` passed. Native controller used the real `sausage::Host`, real VFS, watches, locks, service supervisor and WASM execution; model/skills/memory/ledger/push effects were synthetic or loopback only.
- Reviewed session `cb844151` / SAU-409 final; orchestrator `c80a5854` / SAU-408; web `d883fbf2` / SAU-403; agent/board retained reviewed components. Fresh card-worktree builds: mail **1600 / 53 tests**, cred **1601 / 272**, cron **1602 / 24**, rooms **1603 / 59**, all passed. No activation. Component/source provenance and every bundled file hash are in `evidence/manifest.json`.
- Frontend is exactly approved commit `3d76026749a5d21722033bbfe8743ac368946b0f`, retained production build from SAU-375 rework-1318. It remains `publish:hold`; no publication was attempted.
- World SDK runtime sources already matched the selected host SDK. The only needed source-copy delta is **`src/lib.rs`**, documenting effective grace. Preserve world-owned Cargo manifests, relocated `interface`, and imports. A first broad SDK-copy attempt incorrectly included repository-relative Cargo paths; comparison caught this before any live change, and the final migration used only the one-file patch. Never copy the repository's SDK Cargo manifests over the world SDK.
- Accepted SAU-362 byte snapshot: **1,491 sessions / 580,355,405 bytes**, including **70,689,796-byte default history**. Pristine snapshot remained unpreopened. Executable clones used synthetic model settings; their historical byte volume was retained. Source snapshot consistency is per-file, not a transactional live snapshot.
## Successful checks and limits
| Check | Evidence/result |
|---|---|
| Old ownership → stop-all → offline swap → target boot | Old `mail.start`, old session `send` and detached ledger processes inventoried. Native external controller awaited `Host.stop`, verified zero remaining processes and host exit, then backed up/transformed/swapped. Final shutdown **0.0958 s** in this fixture. One target manager, no mail deliverer, cred locked. |
| Partial swap + rerun | Injected interruption after operation 3; hash-guarded completion and repeated installed-phase apply preserved all **4,374 non-target inventoried files**. No live/new work reset. |
| Exact losses/preservation | Only `pending.jsonl`, `cancel`, and unclaimed pending non-Surma delivery rows selected for removal. Original mail messages, done/Surma copies and uncertain `unknown` deliveries retained without replay; scheduled SQL IDs, calendar/outcome fields and non-session JSON bytes retained. Synthetic grant DB bytes preserved. Loss inventory backs every removed file/row. |
| Offline failure boundaries | Before-start rollback restored exact bytes; conflicting outside edits refused; ambiguous legacy `interrupt:true` refused without mutation. After-start apply and whole-world rollback both refused. |
| Default/advisor HTTP replay | Three identical same-ID POSTs each, before/after consumption; exactly one answered run per session. Default returned null on repeats; advisor returned its same receipt. |
| Explicit open target run | A run-start/consumed record with a finished-looking assistant tail, but no run-end, recovered under the **same agent-run UUID**, with one start. Legacy finished-looking history alone produced no target event/run. |
| Real VFS/runner-boundary work | Actual SDK `armed` then plugin-write `changed`; five follow-ups published at durable callback/runner-end boundaries produced five distinct runs, each consuming its one ID. Does not force the exact armed-exit seam. |
| Real core detached errors | Full nested error `data` survived a detached `core.call`. |
| Real cron research deadline | **305.427 s** send→stopped in the final gate; real cron invoked warning and Stop. The 305-second test deadline is the existing 300-second warning interval plus design grace 5 seconds, not a new default. Held inference was cut, no follow-up run. Original configured research default remains 1,200 seconds. |
| Effective grace / crash supervision | Passing native host suite exercises real WASI uniform-grace and before/after-readiness crash-supervision fixtures. Deliberate `core.kill` is NOT a crash: it disables that boot service; an initial rehearsal expecting its restart failed, correctly, and was replaced by the existing crash tests. |
| Orchestrator upgrade/idle | Real file-watcher upgrade, FIFO singleton release, old exit before one replacement start event. Paused empty-board/live-size-world host CPU **0.02 s / 20.0002 s ≈ 0.10%**. This is whole-host idle CPU, not per-plugin CPU or live active-work load. Replacement readiness remains unverified. |
### Live-size latency after real records exist
After producing actual run-start/run-end/mailbox-consumed records, subsequent full boot measured **0.544 s** from manager service-start to the next boot service-start (includes discovery/ack overhead). Narrow registration catch-up samples ranged **0.0239 s to 12.0419 s** in the integrated world: the long sample coincided with other synthetic recovery/surfacing work, and the untagged sink cannot isolate attribution. Do not present this as a bounded manager-only benchmark.
The clean final live-size end-to-end gate measured manager acknowledgement **0.6078 s**, registration catch-up **0.0128 s**, and `mail.send` publication to default **0.5376 s**. The integrated post-record mail sample was **0.5235 s**. Both publication samples are below the existing orchestrator **3 s** and cred **10 s** caller budgets (SAU-400 report cites the originating SAU-366 review). These warm-cache synthetic-loopback real-core observations are not live latency guarantees. First scanning still delays first new input after the fast boot acknowledgement.
## Evidence and failures
All following logs are under `/logs/box/2026-10-31/`:
- Full workspace pass: `tc-asa3yr-exec-2320283969.stderr`. First cold-cache/full-load failure: `tc-asa3yr-exec-158345583.stderr`, missing offline async-task and a queued-build timing test. Populated the separate SDK-probe cache; entire suite then passed unchanged.
- Final stop/swap/start: `tc-asa3yr-exec-1922130515.stderr`; offline fault/grant preservation: `tc-asa3yr-exec-616022371.stderr`.
- Final live-size HTTP/deadline gate: `tc-asa3yr-exec-1243993190.stderr`. Earlier malformed mail `to` scalar, 30-second startup observation bound and blank JSONL parser mistakes were harness errors; corrected to a sequence, existing 120-second test budget and blank-line skipping. Final gate passed.
- Open-run recovery and boundary runs: `tc-asa3yr-exec-903419814.stderr` (subsequent console-watch attempt failed). A console is not a running watch caller; the final SDK guest watch passed in `tc-asa3yr-exec-3261552495.stderr`. Same log retains the real upgrade trace and idle sample. An identical already-acknowledged registration correctly produced no second catch-up; the final probe used a fresh registration identity.
- Actual old-manager unsubscribe/re-key comparison: `tc-asa3yr-exec-3307393394.stderr`.
- Native artifact architecture/runtime paths: `tc-asa3yr-exec-899103518.stderr`; bundle build/CLI: `tc-asa3yr-exec-1075679130.stderr`.
- Executed temporary tool and controller **as log only**, followed by deletion proof: `tc-asa3yr-exec-2902726782.stderr` (31,475 bytes). `evidence/tool-deletion.json` lists removed executables. No transition code was put in a shipped plugin.
The initial broad-copy fixture and earlier failed tests are not the final kit. `evidence/migration-plan.json` identifies the final verified migration. Provider billing/cancellation, power loss, hung host filesystem behavior, physical browsers and independent per-plugin recovery follow-ups are not claimed complete. No live DB was opened; the accepted session snapshot was reused. No live config/SDK change, stop, activation, release, restart, push, deployment or `/www` write occurred.
See [operator runbook](SAU-384-runbook.md) for approval gates and the deliberately blocked live steps.