sausage/PROTOCOL.md

The Sausage protocol

This is the contract between the host and the actors. An actor in any language that follows it works in Sausage. The host publishes this file at /actor/broker/public/PROTOCOL.md, so every actor can read it.

Actors

An actor is a WASI 0.2 command (wasm32-wasip2): a program with main. The host runs one instance per actor. An instance can run forever.

The instance gets:

• stdin: one message per line, from the broker
• stdout: one message per line, to the broker
• stderr: free text, appended to /actor/<id>/stderr.log
• environment: SAUSAGE_ACTOR (its ID), SAUSAGE_VERSION (its version), and SAUSAGE_CANDIDATE=1 during a test run
• the network: outgoing HTTP and HTTPS through wasi:http (the host does TLS), and TCP, UDP, and DNS through wasi:sockets, including listening on a port of the host
• the filesystem described below

Messages

A line on stdin:

{"id": "m-17", "from": "console", "reply_to": "agent/a5", "body": {"type": "text", "text": "hi"}}

A line on stdout:

{"to": "console", "id": "a6", "reply_to": "m-17", "body": {"type": "text", "text": "hello"}}

• body is required. By convention it is an object with a "type". Text for people is {"type": "text", "text": "..."}.
• to is the recipient. Without to, the broker publishes the message to every subscriber of the sender.
• id is an optional string. If the actor sets it, the message ID is "<sender>/<id>". Otherwise, the broker assigns m-<n>. The broker always sets from.
• reply_to marks a reply: its value is the ID of the message it answers. There is no request/response. A reply is a new message.
• Never answer a reply, or two actors can answer each other forever. broker and cargo ignore messages that have a reply_to.
• Messages are fire and forget: delivery is at most once. Each actor has a mailbox of 1,000 messages, which keeps messages while the actor restarts or is upgraded. If the broker cannot deliver a message, the sender gets {"type": "undeliverable", "reason": "...", "to": "..."} from broker.
• An actor must keep reading stdin. If the broker must stop or upgrade an actor while a message waits for it to read stdin, it kills the actor.

Before the broker stops an instance, it sends {"id": "broker/shutdown", "from": "broker", "body": {"type": "shutdown"}} and then closes stdin. The broker still routes what the actor writes until it exits. After 5 seconds, it kills the instance.

Filesystem

Each instance sees three folders:

Path	Access	Contents
/data	read-write	the actor's state
/public	read-write	the files that the actor publishes
/actor	read-only	the actor/ folder of the world

/actor/<id>/ holds everything about an actor: actor.wasm, actor.json (installed version, running, subscriptions), source/, public/, data/, stderr.log, and versions/<n>/. ls /actor lists all actors.

A new or truncated file becomes visible when the writer closes it. Before that, readers see the old file or no file. If the instance crashes, the write is lost. A write to an existing file without truncate (append, edit) goes in place.

Send large or binary data as a path under /actor/<id>/public/, not inside a message.

The broker

Requests to broker. The answer is a message whose reply_to is the ID of the request: the request's type in past tense (spawned, upgraded, rolled_back, stopped, started, subscribed, unsubscribed, scanned), or {"type": "error", "reason": "..."}.

type	Fields	Effect
spawn	actor, wasm, source?, built_by?	test, then start a new actor
upgrade	actor, wasm, source?, built_by?	test, then swap the version
rollback	actor, version	go back to an older version
stop, start	actor	stop or start an actor
subscribe, unsubscribe	actor	receive (or not) what actor publishes
scan		boot new or changed actor.wasm files

Paths are /actor/..., or the sender's /public/... or /data/....

Versions. A new version (from spawn, upgrade, or a changed actor.wasm on disk) is tested first, while the old version keeps running. Only when its tests pass does the broker copy it to versions/<n>/ (actor.wasm, source/, meta.json) and switch to it. So versions/ holds exactly the versions that were installed, and instances always run from there. If the tests fail, nothing changes, and the sender gets an error with the reason. All versions share the actor's data/ and public/: a new version must read the data of the old one.

Tests. The broker runs the candidate on a copy of the actor's data/ and public/, without network. The tests are JSON files in <source>/tests/*.json, one scenario per file:

{
  "description": "add, then list",
  "steps": [
    {"send": {"type": "add", "text": "milk"}, "expect": {"type": "added"}},
    {"send": {"type": "list"}, "expect": [{"type": "notes"}]}
  ]
}

Each step sends send (from from, default test), then reads the next line that the candidate writes for each expect entry, in order, within 10 seconds. An entry matches when it is a subset of the message body. With to, the message must also go to that recipient. Each file runs in a fresh instance. A version without test files passes.

Swap. The broker holds the mailbox, stops the old instance, makes the top-level actor.wasm and source/ a copy of the new version, and starts it. A rollback does the same with the older version.

Crashes. The broker restarts a crashed actor after 0.5 s, then doubles the delay up to 30 s. A new version that crashes 3 times within 30 seconds after an upgrade goes back to the previous version. The actor that asked for the upgrade gets {"type": "rolled_back", "from", "to", "reason", "last_exit"}.

The folder is the truth. At start, and on scan, the broker boots every actor/<id>/actor.wasm. If the file differs from the installed version, it is a new version, tested with the tests in actor/<id>/source/. If its tests fail, the file stays where it is, the reason goes to stderr.log, and the next scan tries again.

The cargo actor

{"type": "build", "source": "/actor/agent/public/drafts/notes"}

The reply is {"type": "built", "wasm": "/actor/cargo/public/builds/<n>/<name>.wasm", "built_by": "cargo", ...} or {"type": "build_failed", "errors": "..."}. Pass wasm, source, and built_by on to spawn or upgrade.

A Rust actor depends on the SDK with sausage-actor = "0.1". cargo points that dependency at the SDK of the host, which it publishes at /actor/cargo/public/sdk/.