sausage/docs/SAU-384-resume-1373.md

SAU-384 resume — run 1373

Ready for review, not live execution. Authority remains the card/design Rollout and SAU-356 approval gates. No host/session/cred operation, push or web publication was authorized or performed. No shipped host/plugin behavior changed.

Actual owner and native package resolved

After mediator installed the observed host key, reused the approved SSH grant for read-only inspection as surma. The actual owner is Herdr workspace w7, tab w7:t5, pane w7:p6. Its shell runs /nix/store/pz2b94a2v99r3msljdz82h7n3i81hhh5-sausage-loop/bin/sausage-loop /home/surma/world/. This is not a systemd Sausage unit. The loop uses a mutable remote flake, exits on zero and refreshes/restarts after any nonzero exit (its existing sleep is 2 s). Therefore stopping plugins or requesting core.restart is not a deliberate native stop boundary.

Citadel is aarch64-linux. Observed old package, derivation, original source and wrapper/binary hashes are pinned in evidence/resume-1373-inspection.json. Read the old derivation's actual CLI source: SIGTERM awaits Host.stop; an already-requested restart can still return 75. Do not reuse observed PIDs as constants.

Built the exact Nix release package from the unchanged card host source:

• /nix/store/spyd9rix6r38m6vxfppyy9z83yp5dh85-sausage-0.1.0
• bin/.sausage-wrapped SHA-256 683623c4c77dbcfff590c9f0aa78f045d07935a466836a099ccc53d7336853cb
• Package NAR and every closure path/hash: evidence/native-host-manifest.json. ELF is AArch64 with the pinned Nix glibc interpreter. The closure equals the observed old package's closure except for the host root itself.
• Empty caller PATH runtime check passed, building/testing an actual WASI plugin with C and the packaged toolchain; it was rebuilt with visible output, not inferred from --version.

The box shares Citadel's Nix store: this build made an inert store path visible there, not selected by the live launcher. No live package/configuration/boot/SDK was changed.

Native end-to-end and interruption tests

The disposable live-size world reused the accepted SAU-362 snapshot (1,491 sessions / 580,355,405 bytes). Executed the actual old Nix package and inspected loop, actual old mail/session components and actual target gateway; then:

1. Recorded old plugin ownership through the real gateway/proc projection.
2. External controller terminated the outer loop first, awaited its pidfd exit, then SIGTERM'd the exact old host and awaited its exit. The host survived its parent until its own graceful shutdown; controller survived outside the group. One final group inventory was empty; no process polling.
3. Backed up frozen state, applied the approved queue/mail losses and cron conversion, injected interruption after operation 3, completed and repeated the hash-guarded offline plan.
4. Changed a static external selector from the old to the exact new package. Marked started before launch. Started the same actual loop with explicit SAUSAGE_FLAKE=path:<selector>.
5. Verified the running OS executable was the new Nix binary; exactly one target manager, no old mail deliverer, locked cred, real subscriber-registration completion. After-start apply/whole-world rollback refused. Marked verified, then stopped cleanly.

Separate non-parent controller test used the reusable one-off native_freeze.py positional interface, not Python's parent-only wait(): pidfds pin the old loop/host, boot ID/start ticks/executable/group guard identities, owner exit precedes host stop. It proved interruption after owner exit then resume, stale-PID refusal without stopping the host, successful freeze and harmless frozen rerun. Another real-core test requested core.restart after loop exit: host exited and no owner/process in the old group could restart it. These tests close the external-owner gap; earlier run 1369 remains the evidence for armed/exit races, all-watch orchestrator upgrade ordering, and actual gateway rollback.

Native gate inventory contained 4,434 files. Of the 4,372 non-operation files, 4,370 stayed byte-identical after target activity; the only changes were the expected manager registration file and core journal. Histories/settings/mail/jobs/grants outside intended operations remained equal. Prior offline tests additionally proved exact losses, unknown mail retention, SQL ID/calendar/outcome preservation, grant preservation and rollback before start.

Caller-store observation

Read-only SQLite connections (mode=ro, query_only, read transactions) inspected board, cron and rooms; JSON workflow/config stores were also inventoried without printing prompts or grants. See evidence/resume-1373-inspection.json:

• Six legacy cron session templates need the already-tested one-time conversion. Do not execute historical jobs or rewrite RPC words inside their prompts.
• Both board's mediator hook and invariant target are already target-valid {session,text} calls. Preserve their bytes.
• Eight historical running board session-attempt snapshots are diagnostic/uncertain, never replayed (hooks.rs explicitly documents this). Zero board subscriptions/batches were observed.
• One abandoned room has no pending turn/review/split. Orchestrator run/skill-review records have no embedded old session/agent call objects; zero host jobs and no live session-manager registrations were observed. Preserve typed workflow records.
• Three board runs had no outcome at observation, including this worker. This is old in-flight work, not proof of settlement or permission to replay it.

The operator must repeat this audit on the frozen backup at the approved shutdown. This changing live observation cannot be that future snapshot. Unknown executable references block preflight; no shipped compatibility path was added.

Selected review artifacts and live steps

Selected coherent bundle: /docs/symphony/artifacts/SAU-384/cutover-targets.tar.gz (24,591,858 bytes), SHA-256 1bbcae4ef615ca8394b0c53b9a408441a49f8a618f059335c3bc248d958977c4. The 22 payloads are hash-pinned in evidence/manifest.json; bundle also carries that manifest. It contains the native NAR/closure manifest, reviewed plugins, approved web 3d760267, and SDK src/lib.rs only. It excludes the earlier diagnostic debug host. Older target-artifacts.tar.gz remains immutable prior evidence, not the selected native install.

Runbook now gives actual owner/world/package paths, exact loop-first stop/checkpoint interface, proposed external root /home/surma/sau-384-cutover (absent at inspection), pinned selector and restart command, five-entry boot order (session before web; no mail.start), frozen caller conversion, web-release/rollback boundaries and mandatory executable deletion. Surma's go precedes mediator's live core/session/cred operation; every push separately needs go. Only mediator/orchestrator lands/executes; SAU-375 stays publish:hold until session and web are active, then orchestrator.release.

All five executed one-off/migration/test scripts were deleted after verified success; Python bytecode cache was empty. Hashes/deletion proof: evidence/native-1373/native-tool-deletion.json. Source is retained as log only, never as a shipped plugin or retained migration executable. The live mediator must reconstruct its reviewed temporary copy, execute only after go, and delete it plus executable caches after verification too. Static target selector configuration must remain for the pinned native loop; it is not a migration tool.

Exact checks/logs

All full logs below are /logs/box/2026-10-31/:

• Nix package build/CLI: tc-asa3yr-exec-1149169379.stderr; packaged empty-PATH WASI+C gate: tc-asa3yr-exec-395020656.stderr.
• Actual native old stop/offline interruption/swap/new boot: tc-asa3yr-exec-3989255528.stderr.
• Non-parent controller interruption/stale-PID/frozen-rerun: tc-asa3yr-exec-4245550011.stderr; real core.restart after owner death: tc-asa3yr-exec-2683606349.stderr.
• Native architecture/closure equivalence/hashes: tc-asa3yr-exec-1926991205.stderr.
• Repeated cargo test --workspace: pass, including 208 host integration tests; tc-asa3yr-exec-844023236.stderr. All 105 Rust sources matched package/workspace copies. Run 1369's forced-recompile/source equivalence remains the earlier independent gate.
• Executed source-as-log and deletion: tc-asa3yr-exec-2945036769.stderr; 22 payload hashes/controller absence: tc-asa3yr-exec-643070322.stderr.
• New inventories/plan/losses/target process records/closure/source equivalence: evidence/native-1373/.

Prior builds remain mail 1600, cred 1601, cron 1602, rooms 1603; no plugin source changed. Their remote-less repos have no origin (fetch correctly rejected); merging shared local main was already-current for all four. Host fetch/merge was already current before work; final commit/fetch/merge is reported separately.

Failed approaches were corrected, not blockers: one incorrectly transcribed grant was rejected before any SSH command; the exact attached grant succeeded. zsh wildcard expansion and missing PATH tools were replaced by explicit installed Nix paths. core.ps does not exist; the native gate switched to the actual gateway /fs/proc projection and passed. Nix store logs were unavailable, so rebuilt the runtime check with visible -L output. One evidence-copy Path/string expression failed; corrected and resumed without re-executing migration. Full failure logs remain under /logs.

Limits, not silently claimed complete

No new production-latency guarantee is claimed. Prior post-record real-core live-size samples remain: final manager acknowledgement 0.6078 s, registration catch-up 0.0128 s, mail.send-to-default publication 0.5376 s (integrated 0.5235 s), below callers' existing 3 s / 10 s budgets. The integrated catch-up outlier 12.0419 s remains disclosed. Orchestrator idle whole-host CPU was ~0.10% in the paused fixture; prior real cron warning/stop took 305.427 s, and detached error data passed. These measurements and their synthetic/warm-cache limits are in implementation.md.

Live shutdown/frozen backup/push/activation/release/publication, physical browser display, power loss and provider cancellation/billing remain unverified/out of scope. Old unrecorded runs are not automatically inferred complete/resumed. Independent per-plugin recovery follow-ups remain independent; the design accepts first-update cuts until they land.