sausage/PROTOCOL.md

The Sausage protocol

This is the contract between the host and the actors. An actor in any language that follows it works in Sausage. The host publishes this file at /actor/broker/public/PROTOCOL.md, so every actor can read it.

Actors

An actor is a WASI 0.2 command (wasm32-wasip2): a program with main. The host runs one instance per actor. An instance can run forever.

The instance gets:

• stdin: one message per line, from the broker
• stdout: one message per line, to the broker
• stderr: free text, appended to /actor/<id>/stderr.log
• environment: SAUSAGE_ACTOR (its ID), SAUSAGE_VERSION (its version), and SAUSAGE_CANDIDATE=1 during a test run
• the network: outgoing HTTP and HTTPS through wasi:http (the host does TLS), and TCP, UDP, and DNS through wasi:sockets, including listening on a port of the host
• the filesystem described below

Messages

A line on stdin:

{"id": "m-17", "from": "console", "reply_to": "agent/a5", "body": {"type": "text", "text": "hi"}}

A line on stdout:

{"to": "console", "id": "a6", "reply_to": "m-17", "body": {"type": "text", "text": "hello"}}

• body is required. By convention it is an object with a "type". Text for people is {"type": "text", "text": "..."}.
• to is the recipient. Without to, the broker publishes the message to every subscriber of the sender.
• id is an optional string. If the actor sets it, the message ID is "<sender>/<id>". Otherwise, the broker assigns m-<n>. The broker always sets from.
• reply_to marks a reply: its value is the ID of the message it answers. There is no request/response. A reply is a new message.
• Never answer a reply, or two actors can answer each other forever. broker and cargo ignore messages that have a reply_to.
• Messages are fire and forget: delivery is at most once. Each actor has a mailbox of 1,000 messages, which keeps messages while the actor restarts or is upgraded. If the broker cannot deliver a message, the sender gets {"type": "undeliverable", "reason": "...", "to": "..."} from broker.
• An actor must keep reading stdin. If the broker must stop or upgrade an actor while a message waits for it to read stdin, it kills the actor.

Before the broker stops an instance, it sends {"id": "broker/shutdown", "from": "broker", "body": {"type": "shutdown"}} and then closes stdin. The broker still routes what the actor writes until it exits. After 5 seconds, it kills the instance.

Filesystem

Each instance sees these mounts:

Path	Access	Contents
/data	read-write by its actor; readable by every actor	the actor's state
/public	read-write	the files that the actor publishes
/actor	read-only	the actor/ folder of the world
/config	read-only	one non-secret global JSON file, owned by shared
/global	read-write for shared, read-only for all other actors	backing directory containing that same config file

An actor can write only its own /data, but can read every actor's /data through /actor/<id>/data/. /data is not private: any unprotected secret file there is readable by every actor.

/actor/<id>/ holds everything about an actor: actor.wasm, actor.json (installed version, running, subscriptions, tapped, watches), source/, public/, data/, stderr.log, and versions/<n>/. ls /actor lists all actors.

Files are plain POSIX files: a write is visible at once. To publish a file so that readers never see half of it, write a temporary file and rename it (the Rust SDK has write_atomic).

/config is global configuration, never credentials (in particular, never the LLM proxy token). The host initializes it to {} without reading or copying legacy actor settings. Configuration schema and typed edits belong to shared, not the host. Ordinary actors cannot write either /config or its alias /global/config: WASI mounts are read-only, with no message sender checks.

WASI preopens directories rather than individual files. The host mounts the isolated world global/ directory at / read-only and at /global with the owner's write access. shared publishes atomically by writing a sibling such as /global/config.tmp, closing it, then renaming it to /global/config. New opens of /config see the replacement, including in already-running actors; an already-open file retains its old inode. The root mount does not expose the rest of the host world. Candidate tests get a scratch copy of global/, so even shared tests cannot modify the live file.

On disk, the one JSON file is <world>/global/config. <world>/config is a relative symlink to it, for native/world filesystem readers (including an export of the world directory). Host-side atomic writers must replace <world>/global/config, not replace the symlink. Starting an older world adds the empty file and alias without changing actor state; an existing regular <world>/config is moved into the backing directory on first use.

Send large or binary data as a path under /actor/<id>/public/, not inside a message.

Actor-served files: /actor/<id>/fs/

Files under /actor/<id>/fs/ are not on disk: the actor <id> serves them. Every actor can read and write them. The host turns each file operation into a message from fs to <id> and waits up to 10 seconds for the reply. path is relative to fs/ and starts with / (the folder itself is /).

To make this virtual directory discoverable, an actor that serves it declares Actor-served files and fs_read in its leading //! documentation in source/src/main.rs. The host uses that declaration when listing /actor/<id>/; it does not probe every actor (which could stall on actors that do not handle filesystem messages). This is discovery metadata only: opening and reading the path still requires a valid fs_read reply.

Request	Reply
{"type": "fs_read", "path"}	{"type": "fs_file", "text"} (or "base64" instead of "text"), or {"type": "fs_dir", "entries": [{"name", "type": "file" | "dir"}]}
{"type": "fs_write", "path", "text"} (or "base64" for non-UTF-8 data)	{"type": "fs_done"}
{"type": "fs_mkdir", "path"}, {"type": "fs_remove", "path"}	{"type": "fs_done"}
{"type": "fs_rename", "path", "to"}	{"type": "fs_done"}

Any request can be answered with {"type": "fs_error", "errno": "no-entry"}. errno is a wasi:filesystem error code: no-entry, exist, not-directory, is-directory, not-empty, access, not-permitted, invalid, unsupported, busy, or io (also for anything else).

• Opening a file sends fs_read once. Reads come from that copy.
• After a change, closing (or syncing) the file sends the whole file in one fs_write.
• No reply within 10 seconds, or an actor that does not exist, makes the operation fail (io, or no-entry). An actor that opens its own fs/ waits for itself and fails after the timeout.
• fs is reserved: no actor can have that ID. It also sends fs_changed to watchers (see Watches).

The broker

Requests to broker. The answer is a message whose reply_to is the ID of the request: the request's type in past tense (spawned, upgraded, rolled_back, stopped, started, removed, subscribed, unsubscribed, tapping, untapped, watching, unwatched, exiting), or {"type": "error", "reason": "..."}.

type	Fields	Effect
spawn	actor, wasm, source?, built_by?	test, then start a new actor
upgrade	actor, wasm, source?, built_by?	test, then swap the version: a running actor is stopped (normal shutdown) and starts the new version; a stopped actor gets the new version and stays stopped, with its mailbox untouched, until start. The answer has running
rollback	actor, version	go back to an older version; like upgrade, a stopped actor stays stopped. The answer has running
stop, start	actor	stop or start an actor
remove	actor	stop the actor (normal shutdown), then delete its whole folder /actor/<id>/ (all versions, data/, public/, logs); answer {"type": "removed", "actor"}. Messages to it are then undeliverable, and the ID can be spawned again. An unknown actor or a native one (broker, console, cargo, fs) is an error
subscribe, unsubscribe	actor	receive (or not) what actor publishes
tap, untap		receive (or not) a copy of every message the broker routes
watch, unwatch	path	receive (or not) fs_changed for every change of path or anything under it; answer {"type": "watching" | "unwatched", "path"} with the path as an /actor/... path
exit		answer exiting, stop every actor, and end the sausage process with exit code 75 (a wrapper like nix run .#loop starts it again, updated)

Headless. sausage run --headless [dir] (or sausage --headless, or SAUSAGE_HEADLESS=1) turns the console input off: the host does not read stdin, so end of input (/dev/null) does not stop the world. nix run .#loop -- --headless <world> passes it through the loop. It runs until SIGTERM or SIGINT (stops every actor, exit code 0) or a broker exit request (exit code 75, as above). Without the flag, EOF on stdin or /quit ends the host.

Paths are /actor/..., or the sender's /public/... or /data/....

Taps. After tap, the sender gets a copy of every message that the broker routes (direct messages, replies, and publishes, between any actors, including its own), for debugging, logging, or a live view of the world:

{"type": "tapped", "message": {"id": "m-17", "from": "console", "to": "agent", "reply_to": null, "body": {...}}, "ts": 1792132579067}

• to is null for a publish: one copy per publish, not per subscriber.
• ts is the time of routing, in milliseconds since the Unix epoch.
• Copies come from broker. They are not routed themselves, so they are never tapped, and each message is tapped once per tapper.
• A copy that does not fit into the tapper's mailbox is dropped; routing never waits for a tapper, and nobody gets undeliverable for a copy.
• Never answer a copy: the answer would be tapped again, forever.
• Like subscriptions, a tap is saved in actor.json ("tapped": true) and survives restarts and upgrades. untap ends it.

Watches. The host watches the whole world folder with the OS file watcher: every change inside it counts, whoever makes it (actors, a 9P mount, a shell or git on the host). Actor-served files (/actor/<id>/fs/) are not on disk and do not count. After watch {path}, the sender gets for each change of path or a path under it (/actor/a covers /actor/a/x, not /actor/ab):

{"type": "fs_changed", "path": "/actor/notes/public/list.md", "op": "write"}

• op is write, create, remove, or rename (then to is the new path; a watcher of either path gets it).
• Changes of one path within about 100 ms become one event (a write after a create stays a create).
• The host's temporary files (names starting with .sausage-tmp-) are skipped; renaming one onto a file is a write of that file.
• Events come from fs. Like tap copies, they are not routed, a full mailbox drops them, and they never block the writer. Never answer them.
• If the OS drops events, every watcher gets one {"type": "fs_changed", "path": "/actor", "op": "rescan"}: anything may have changed.
• A watch is saved in actor.json ("watches": [...]) and survives restarts and upgrades. unwatch with the same path ends it.

Versions. A new version (from spawn, upgrade, or a changed actor.wasm on disk) is tested first, while the old version keeps running. Only when its tests pass does the broker copy it to versions/<n>/ (actor.wasm, source/, meta.json) and switch to it. So versions/ holds exactly the versions that were installed, and instances always run from there. If the tests fail, nothing changes, and the sender gets an error with the reason. All versions share the actor's data/ and public/: a new version must read the data of the old one.

Tests. The broker runs the candidate on a copy of the actor's data/ and public/, without network. The tests are JSON files in <source>/tests/*.json, one scenario per file:

{
  "description": "add, then list",
  "steps": [
    {"send": {"type": "add", "text": "milk"}, "expect": {"type": "added"}},
    {"send": {"type": "list"}, "expect": [{"type": "notes"}]}
  ]
}

Each step sends send (from from, default test), then reads the next line that the candidate writes for each expect entry, in order, within 10 seconds. An entry matches when it is a subset of the message body. With to, the message must also go to that recipient. Each file runs in a fresh instance. A version without test files passes.

Swap. The broker holds the mailbox, stops the old instance, makes the top-level actor.wasm and source/ a copy of the new version, and starts it. A stopped actor is not started: it keeps its mailbox and runs the new version at the next start. A rollback does the same with the older version.

Crashes. The broker restarts a crashed actor after 0.5 s, then doubles the delay up to 30 s. A new version that crashes 3 times within 30 seconds after an upgrade goes back to the previous version. The actor that asked for the upgrade gets {"type": "rolled_back", "from", "to", "reason", "last_exit"}.

The folder is the truth. At start, the broker boots every actor/<id>/actor.wasm. Afterwards, the file watcher reports a new or changed actor/<id>/actor.wasm (or a new folder actor/<id>/) once it is quiet for about 100 ms: a new file becomes a new actor, a changed one an upgrade, tested first with the tests in actor/<id>/source/. A file equal to the installed version changes nothing. If its tests fail, the file stays where it is, the reason goes to stderr.log, and the next change of the file tries again. When the OS drops events, the broker checks every actor folder again, as at start.

The cargo actor

{"type": "build", "source": "/actor/agent/public/drafts/notes"}

The build request may include "skip_tests": true to skip Rust unit tests. By default, cargo first runs cargo test for the host target (not WASI), so pure-logic unit tests in a wasm32-wasip2 actor crate can run without a WASI test runner. If tests fail or cannot compile, the build fails before the WASI build. The reply carries tests: {"passed": <count>, "failed": <count>, "output": "<cargo test stdout and stderr>"}; counts sum Cargo's test-result summaries, and a test command/compilation failure without a summary counts as one failure. With skip_tests: true, counts are zero and output says tests were skipped.

The reply is {"type": "built", "wasm": "/actor/cargo/public/builds/<n>/<name>.wasm", "built_by": "cargo", "tests": {...}, ...} or {"type": "build_failed", "errors": "...", "tests": {...}}. Pass wasm, source, and built_by on to spawn or upgrade.

A Rust actor depends on the SDK with sausage-actor = "0.1". cargo points that dependency at the SDK of the host, which it publishes at /actor/cargo/public/sdk/.