From 61d5d5f81785b3f23620023d5d9a2b67496df9fe Mon Sep 17 00:00:00 2001 From: Surma Date: Fri, 23 Oct 2026 02:46:47 +0100 Subject: [PATCH] sausage-plugin: the crate docs are the manual for agents; remove PROPOSAL.md Its content lives in PROTOCOL.md (the contract), DECISIONS.md (the reasons), and the SDK docs (how to write a plugin). --- PROPOSAL.md | 922 ----------------------------------------- sdk/imports/notes.json | 25 ++ sdk/imports/www.json | 25 ++ sdk/src/lib.rs | 410 ++++++++++++++++-- 4 files changed, 424 insertions(+), 958 deletions(-) delete mode 100644 PROPOSAL.md create mode 100644 sdk/imports/notes.json create mode 100644 sdk/imports/www.json diff --git a/PROPOSAL.md b/PROPOSAL.md deleted file mode 100644 index 4844762..0000000 --- a/PROPOSAL.md +++ /dev/null @@ -1,922 +0,0 @@ -# Proposal: plugins, processes, and JSON-RPC - -Status: a proposal on the branch `plugins`. When we accept it, its content -goes into `PROTOCOL.md` (the contract) and `DECISIONS.md` (the reasons). -Then this file goes away. - -## 1. The idea - -Sausage is a world of small WASI programs that an agent grows by talking to -it. The design uses three ideas that models already know: - -1. **JSON-RPC calls.** A call has a result, like an LLM tool call. LSP and - MCP use the same protocol. -2. **Processes.** Each call to a plugin starts a new WASI process. WASI has no - threads, so processes are the unit of parallelism. Calls are asynchronous, - so one process can make many calls at the same time. -3. **Files to inspect, calls to act.** The core shows its state as virtual - files in `/proc`, like Plan 9 and Linux. Each change of state is a call. - -## 2. Terms - -| Term | Meaning | -| --- | --- | -| world | The folder that a `sausage` host runs. It is `/` for every process. | -| core | The host. It runs processes, routes calls, serves the virtual files, and keeps the journal. | -| plugin | A folder `/plugins//` with a `plugin.wasm`. A plugin does not run by itself. | -| process | One run of a plugin version. Its ID is `-`, for example `notes-0197f3a2-7c1e-7d2b-9a41-5f0c2e8b1d34`. | -| call | A JSON-RPC request. `notes.add` calls the method `add` of the plugin `notes`. | -| interface | A set of methods. A plugin has one main interface. It can also declare endpoint interfaces. | -| endpoint | An interface that a running process serves on a TCP port. An endpoint file tells others where it is. | -| service | A call in `/config/boot.json`. The core starts it at boot and again after an upgrade of its plugin. | -| import | An interface of another plugin that a plugin uses. The SDK generates it from that plugin's description. | -| provider, consumer | The plugin that has an interface, and a plugin that imports it. | -| journal | The central log of the world. | - -## 3. The world folder - -```text -/ - plugins// - plugin.wasm the current version - plugin.prev.wasm the version before, for `core.revert` - source/ the source. `cargo.build` builds it from here. - imports/.json descriptions of the imported plugins - state// persistent state of a plugin, by convention - state/core/journal.sqlite the journal - config/ global configuration - boot.json the services - agent.json API, base URL, model, and token of the agent -``` - -- Every process can read and write all of it. There are no permissions. -- The world folder must not contain `proc`, `run`, or `self`. These names are - mount points (section 5). The host does not start a world that has them. - -## 4. Calls and processes - -### 4.1 The protocol - -- JSON-RPC 2.0 on stdin and stdout, one message per line, as in MCP's stdio - transport. -- A line is a message only if it is a JSON object with `"jsonrpc": "2.0"`. -- The core discards every other line on stdout. For the first discarded line - of a process, it writes one warning to the journal, without the content of - the line. Logs go to stderr. -- Each line on stderr goes to the journal. -- `params` is always an object. Batches are not supported. -- The SDK writes each message with one blocking write through std's stdout, - and it writes a newline before each message. So a `print!` without a - newline cannot damage a message. - -### 4.2 Method names - -- A method name has the form `.`. The core splits it at the - first `.`. -- Plugin names use `a-z`, `0-9`, `-`, and `_`. The name `core` is reserved. -- `notes.add` starts a new process of `notes` and sends it the request `add`. - `notes.rpc.discover` sends it `rpc.discover`. -- `core.` is a method of the core (section 10). -- Method names that start with `rpc.` are reserved: `rpc.discover`, - `rpc.retired`, and `rpc.endpoint`. - -### 4.3 What a call does - -1. The core gives the call a UUIDv7 and starts the process `-` - with the current version of the plugin. -2. The core writes the request to the stdin of the process. The request ID is - the process ID. -3. The process writes the response to stdout. The core sends the response to - the caller, with the caller's request ID. -4. If the process ends before it answers, the caller gets the error -32002. - Its `data` has the exit status and the last journal lines of the process. - Before the core decides this, it reads the stdout of the process to the - end. - -- A process can write calls to stdout at any time, to the core and to - plugins. The responses come on stdin. A process can have many calls in - flight. -- The core keys each pending request by the process and the exact JSON ID. - The core drops a response with an unknown or late ID. The SDK writes a note - about such a response to stderr. -- A notification to `plugin.method` (without an ID) starts an attached - process. That process gets the notification without an ID, and nobody gets - a result. - -### 4.4 Detached calls - -`core.call {"method": "www.start", "params": {...}, "detach": true}` makes the -call and returns its result to the caller. The new process has no parent, so -it does not end with its caller. - -Sent as a notification (without an ID), the same `core.call` starts the -process and forgets it. - -### 4.5 Notifications to the caller - -A process can tell its caller about progress. It sends the notification -`core.notify {"name": "text", "params": {...}}`. The core gives it to the -caller as `core.notify {"call": , "name": "text", -"params": {...}}`. After the call has ended, the core drops such -notifications. - -So a process never needs to know who called it. Endpoint calls send their -notifications over the endpoint connection (section 6). - -### 4.6 Lifetime - -A process lives while one of these runs: - -- a handler of a call, -- a background task, -- an endpoint that listens. - -The SDK's own reader of stdin does not count. When nothing of this runs, the -SDK returns from `main`, and the process ends. - -A process has three states: - -1. **running** -2. **stopping**: the core has closed stdin. EOF on stdin means "stop now". - - New calls of the process fail at once, and its pending calls fail. - - A stop hook can still write files and send notifications. It cannot - make calls, because no response can arrive. - - After 5 seconds, the core kills the process. -3. **ended**: the core releases the locks of the process (section 7) and - removes its folder in `/proc`. - -Rules: - -- `core.stop {pid}` starts the stopping state. `core.kill {pid}` kills the - process at once. -- When a process ends, the core stops each of its attached children. All - children stop in parallel, so a whole tree takes 5 seconds at most. -- A detached process has no parent. Only `core.stop`, `core.kill`, an - upgrade (section 4.7), and the end of the host stop it. -- At host shutdown, the core stops every process in parallel. - -Note: on Unix, children outlive their parent. Here, an attached child ends -with its parent. Only `detach` makes a process independent. - -### 4.7 Upgrades - -A process is an instance of one plugin version. When a new version becomes -active (section 11.3): - -1. New calls get the new version at once. -2. Each process of an older version gets a grace period to finish its work. - The default is 60 seconds. A plugin can declare its own value in - `rpc.discover` (`x-sausage-grace`). Then the core stops the process. -3. When the old process of a service has ended, the core starts the service - again with the new version. -4. When the last process of an older version has ended, the core starts a - process of the new version and calls `rpc.retired {"previous": - ""}`. A plugin can use this hook to migrate its - state, or to write a marker that its other new processes wait for. The - result goes to the journal. - -During the grace period, an old process can call a method that a new -provider no longer has. The time is limited, so this risk is acceptable. - -### 4.8 Environment and network - -- `SAUSAGE_PID`: the process ID. -- `SAUSAGE_PLUGIN`: the plugin name. -- `SAUSAGE_PARENT`: the parent: `console`, `core`, or a process ID. - -Every process gets the network: outgoing HTTP and HTTPS through `wasi:http`, -and TCP, UDP, and DNS through `wasi:sockets`. - -### 4.9 Scheduling - -- Each process is a task on the core's multi-thread tokio runtime, with work - stealing. -- A scheduler thread advances Wasmtime's epoch every 10 milliseconds. At - each tick, a process that computes yields to the runtime. So a busy process - cannot block the core or the other processes. -- `core.kill` aborts the task. It takes effect at the next tick at the latest. -- Most processes wait for IO most of the time. A better scheduler can come - later. - -### 4.10 Errors - -The standard JSON-RPC codes apply. The core adds these codes: - -| Code | Meaning | -| --- | --- | -| -32001 | There is no such plugin. | -| -32002 | The process ended without an answer. `data` has the exit status and the last journal lines. | -| -32003 | The activation was rejected. `data` has the reasons. | - -## 5. The filesystem - -Each process gets one preopen, `/`. The VFS resolves each path through a -mount table: - -| Path | Layer | Access | -| --- | --- | --- | -| `/` | host: the world folder | read-write | -| `/proc` | generated by the core | read-only | -| `/self` | memory: the folder `/proc//` | read-write | -| `/run` | memory, shared, empty at each host start | read-write | - -A listing of `/` shows `proc`, `run`, and `self`. A path from any folder -descriptor goes through the same mount table. - -```text -/proc/ - core/ - PROTOCOL.md the contract - graph.json plugins, imports, and the result of each check - graph.dot the same graph for Graphviz - locks.json held locks and waiting processes - plugins// - discover.json the rpc.discover document of the current plugin.wasm - client.rs the Rust code that import!("") generates - / - info.json plugin, version, method, params, parent, detached, start time, state - log the journal lines of this process - the files that the process writes to /self -``` - -Rules: - -- A process writes its own folder only through `/self`. Writes through - `/proc` always fail. -- In a process folder, the names `info.json`, `log`, and `fs` belong to the - core. A write to `/self/info.json` fails. -- A read of a generated file returns a snapshot that ends. For example, - `/proc//log` returns the lines up to the time of the open. -- The folder of a process goes away when the process ends. Its files are in - memory. Large or persistent data belongs in `/state`. -- Do not send a `/self` path to another process. There it means a different - folder, like `localhost` on another machine. - -Principle: **inspect through files, act through calls.** A file read is -cheaper than a call, and every language and tool can read a file. - -### 5.1 The VFS crate - -The VFS is a crate of its own, `sausage-vfs`, so that it can become public -later. It knows nothing about Sausage. - -- One trait with the operations of a filesystem: stat, list, open, read and - write at an offset, set size, create a folder, remove, and rename. -- The layers: - - **host:** a real folder, sandboxed with `cap-std`. Each file operation - runs in a blocking task. That task holds a guard of its process until - the system call has returned, even if the process was killed. - - **memory:** files and folders in memory. - - **generated:** files that a function makes at each open. - - **mounts:** sends each path to the right layer. Each process has its own - mount table, which shares most layers with the other processes. -- An adapter implements `wasi:filesystem` for Wasmtime on top of the trait. - It replaces the filesystem of `wasmtime-wasi` completely. -- Not now: overlays (a union of two layers). Deletes and renames across - layers need whiteouts and copy-up. The trait leaves room for overlays. -- Tests: - 1. Unit tests for each layer. - 2. Differential tests: `proptest` generates sequences of operations. They - run on the host layer and on `std::fs` in a temporary folder, and the - results must be equal. - 3. A model test for the memory layer. - 4. WASI tests: Rust guests use `std::fs` through the adapter. `rusqlite` - and `tempfile` must work. - -## 6. Endpoints - -- `listen(path, interface)` binds a free port on 127.0.0.1. When the listener - is ready, it writes an endpoint file at `path`, in one atomic step. The - file has one line: the address, the owner, and the interface. Example: - `tcp:127.0.0.1:41234 www-0197… control`. -- Usually the path is in `/self`. A service that others must find uses a - fixed path in `/run`, for example `/run/secrets`. -- `connect(path)` reads the file and connects. Its first request is the - built-in `rpc.endpoint`, which returns the owner, the plugin, and the - interface. If they do not match the file, `connect` fails. So a stale file - or a reused port cannot connect a client to the wrong process. -- If the file does not exist yet, `connect` waits a short time, for example - while the services start. -- JSON-RPC runs on the connection, one message per line. In an endpoint - handler, `ctx.notify` sends the notification over the connection. -- `listen` accepts only interfaces that the plugin declares (section 11.1). - -## 7. Locks - -- `core.lock {"path": "/state/notes/notes.json"}` returns when the caller - holds the lock. -- Each acquisition gets its own place in a FIFO queue. The same process - cannot take one lock twice: its second acquisition waits behind its first. -- With `"wait": false`, the call returns at once with `{"held": true}` or - `{"held": false}`. -- `core.unlock {path}` releases the lock. The SDK sends it as a notification - when a guard drops. If the future of an acquisition was dropped, the SDK - releases the lock when the grant arrives. -- When a process ends, the core releases all of its locks, also after a - crash or a kill. It waits until the last file operation of the process has - returned (section 5.1). So a write of a killed process cannot land after - the next process got the lock. -- Locks are advisory. The core does not block writes to the file: the path - is only a name. -- SQLite has no file locks in WASI. Hold the lock while the database is open. - -## 8. The journal - -- An SQLite file at `/state/core/journal.sqlite`. Only the core writes it. -- Each entry has a UUIDv7 (it sorts by time), the process ID, the plugin, the - stream (`stderr` or `core`), and the line. -- The core writes its own events too: start (with method and params), exit - status, stop, kill, upgrades, and the warning about discarded stdout. -- `core.log {plugin?, pid?, since?, until?, grep?, limit?}` queries the - journal. `/proc//log` shows the lines of one running process. -- Later: structured logging, like `tracing`. - -## 9. Boot and services - -At start, the core starts each service in `/config/boot.json`, in order: - -```json -[ - {"method": "secrets.serve", "params": {}}, - {"method": "www.start", "params": {"port": 8080}} -] -``` - -- A service is a detached call. The core waits for its result before it - starts the next one. The result shows that the service is ready. -- If a service fails, the core writes the error to the journal and continues. -- After an upgrade of the plugin of a service, the core starts the service - again with the new version (section 4.7). -- The core does not restart other detached processes. -- A singleton is a service with a fixed endpoint in `/run`. Only the core - starts it, so two instances cannot start at the same time. - -## 10. The core API - -| Method | Params | Result | -| --- | --- | --- | -| `core.call` | `method`, `params`, `detach`? | the result of the method | -| `core.notify` | `name`, `params` | none (a notification) | -| `core.stop` | `pid` | `{}` when the process has ended | -| `core.kill` | `pid` | `{}` | -| `core.lock` | `path`, `wait`? | `{"held": bool}` | -| `core.unlock` | `path` | none (a notification) | -| `core.activate` | `plugin` and `wasm`, or `plugins: [{plugin, wasm}]`. `force`? | the new `rpc.discover` documents | -| `core.revert` | `plugin`, `force`? | like `core.activate` | -| `core.log` | filters (section 8) | journal entries | -| `core.exit` | none | `{}`. Then the host stops with exit code 75. | - -Everything else is a file. - -## 11. Interfaces - -### 11.1 `rpc.discover` - -Each plugin answers `rpc.discover` with an OpenRPC document. The SDK -generates it from the code, with all doc comments. The schemas are JSON -Schema Draft 7, as OpenRPC requires. The document has these extension -fields: - -- `x-sausage-params` for each method: a `$ref` to the schema of the params - struct. So the name and the docs of the struct are not lost. -- `x-sausage-endpoints`: the endpoint interfaces by name, each with its - methods. -- `x-sausage-imports`: the methods that the plugin uses from each provider, - with their schemas. -- `x-sausage-grace`: the grace period in seconds, if the plugin declares one. - -The schema of params describes what the provider can deserialize. The schema -of a result describes what the provider serializes. (Schemars generates both -kinds. They differ, for example for fields with `#[serde(default)]`.) - -```json -{ - "openrpc": "1.2.6", - "info": {"title": "www", "version": "0.1.0"}, - "methods": [{ - "name": "start", - "summary": "Start the web server and return the ID of its process.", - "paramStructure": "by-name", - "params": [{"name": "port", "required": true, - "description": "The TCP port to listen on, on all interfaces.", - "schema": {"type": "integer", "minimum": 0, "maximum": 65535}}], - "result": {"name": "pid", "schema": {"type": "string"}}, - "x-sausage-params": {"$ref": "#/components/schemas/StartParams"} - }], - "components": {"schemas": {"StartParams": {"…": "…"}}}, - "x-sausage-endpoints": {"control": {"methods": [{"name": "stats", "…": "…"}]}}, - "x-sausage-imports": {"notes": {"methods": [{"name": "add", "…": "…"}]}} -} -``` - -### 11.2 Compatibility - -A provider can add methods, optional params, and result fields. For each -imported method: - -1. The provider has the method. -2. Each params value that the consumer can send is valid for the provider. -3. Each result that the provider can return is valid for the consumer. - -- The same rules apply to endpoint interfaces. -- There are no version numbers. This check replaces them. -- The check is our own. It covers a fixed subset of JSON Schema: objects with - `properties` and `required`, the primitive types, arrays, optional and - nullable values, string enums, and `$ref`. `#[schema]` gives a compile - error for a type outside this subset. So the check can decide every case. -- The check covers calls, not state. Old and new processes share the files in - `/state`, and `core.revert` restores only code. A plugin must keep its - state format compatible, or migrate it in `rpc.retired`. - -### 11.3 Activation - -`core.activate {"plugin": "notes", "wasm": "/state/cargo/builds/7/notes.wasm"}`: - -1. The core copies the file. All further steps use this copy, even if the - file changes. -2. It starts the copy and calls `rpc.discover`. -3. It waits until it has the documents of all plugins (section 11.4). A - missing document never counts as "no consumers". -4. It checks the imports of the new version against the current providers. -5. It checks each current consumer of the plugin against the new version. -6. If all checks pass, `plugin.wasm` becomes `plugin.prev.wasm`, and the copy - becomes `plugin.wasm`. Then the upgrade starts (section 4.7). - -If a check fails, nothing changes. The error lists each problem, with a -counterexample where possible: - -```text -notes was not activated: 1 consumer breaks. -- agent imports notes.add(text: string) -> Added. - The new notes.add has no param "text". It requires "body": string. - Counterexample: {"text": "milk"} is not valid for the new params. -Update agent first, or activate both together. -``` - -- The core runs one activation at a time. -- `plugins: [...]` activates several plugins together. The checks then use - the new set. The core writes all files under temporary names first, and - then renames them. -- `force: true` skips steps 4 and 5. -- `core.revert {plugin}` activates `plugin.prev.wasm` in the same way, and - the current version becomes `plugin.prev.wasm`. -- A person can also copy a file to `plugin.wasm` by hand. That skips the - checks (section 11.4). - -### 11.4 The graph - -- The core keeps the `rpc.discover` documents of all plugins in memory, by - file hash. At start, it gets them in the background. -- A file watcher on `/plugins/*/plugin.wasm` updates them, for example after - a copy by hand. Such a copy skips the checks. `graph.json` then shows each - broken import, and the journal records it. -- An import of a plugin that does not exist is "missing". -- Untyped calls (`call("x.y", ...)`) are not in the graph. - -## 12. Building and tests - -`cargo.build {"plugin": "notes"}` is a native method of the host. A call -still gets a process ID, a folder in `/proc`, and journal entries, like other -calls. The method does these steps: - -1. It copies `/plugins/notes/source/` once. The tests and the build both use - this copy. -2. It runs the tests under WASI. -3. It builds the release `.wasm` for `wasm32-wasip2`. -4. It returns `{"wasm": "/state/cargo/builds//notes.wasm", "tests": - {"passed", "failed", "output"}, "warnings"}`. If a step fails, it returns - an error with the compiler or test output. - -- The dependency `sausage-plugin = "0.1"` points to the SDK copy at - `/plugins/cargo/sdk/`. -- Build scripts can compile C: `CC_wasm32_wasip2` and `AR_wasm32_wasip2` - point to a WASI clang. -- The warnings include unused imported methods. Remove them from - `imports/.json`, so that fewer changes of the provider break the - plugin. -- A kill of the call stops the whole process group of cargo, with rustc, the - build scripts, and the tests. - -Tests: - -- `sausage` is the test runner (`CARGO_TARGET_WASM32_WASIP2_RUNNER`). It runs - each test in its own instance, in parallel. A panic ends only its own test. - Without this, the first panic would end the whole test run, because WASI - has `panic=abort`. -- The normal Rust test setup works: `#[test]`, `tests/*.rs`, name filters, - and `--ignored`. Async tests use `#[sausage_plugin::test]`, like - `#[tokio::test]`. `#[should_panic]` cannot work, because WASI cannot catch - a panic. -- Each test gets its own mount table: an empty temporary world as `/`, the - package folder (so that fixture files work), the variables that cargo sets, - and its own `/proc`, `/self`, and `/run`. -- The runner answers `core.lock`, `core.unlock`, `core.notify`, and - `core.log`. A call to a plugin fails with a clear error: a test has no - other plugins. - -## 13. The SDK - -Three crates: - -- `sausage-plugin`: the SDK. A plugin depends only on this crate. It - re-exports the macros, `serde`, and `schemars`. -- `sausage-plugin-macros`: the macros. -- `sausage-interface`: the OpenRPC documents, the compatibility check - (section 11.2), and the generator of client code. The macros and the host - both use it, so `import!` and `/proc/core/plugins//client.rs` give - the same code. - -| Item | Purpose | -| --- | --- | -| `#[rpc]` | Makes an `impl` an interface. On the main interface: `#[rpc(endpoints(name = Type), grace_seconds = N)]`. | -| `#[on_retired]` | Marks the method that answers `rpc.retired`. | -| `#[on_stop]` | Marks the stop hook: an optional method that runs at EOF on stdin (section 4.6). | -| `#[schema]` | For each type that a method takes or returns: adds the serde and schemars derives, and checks the subset of section 11.2. | -| `run(T)` | In `main`: serves the main interface on stdin and stdout. | -| `call(method, params)` | An untyped call. Returns a JSON value. | -| `.detach()`, `.on_notify(f)` | Options of a call. | -| `Ctx` | An optional second argument of a method: `ctx.notify(name, params)`, `ctx.exit_after_response()`. | -| `lock(path)` | A lock guard. `.try_now()` does not wait. `.keep()` holds the lock until the process ends. | -| `listen(path, value)`, `connect(path)` | Endpoints. | -| `background(future)` | Runs a task in this process. The process lives while a task runs. | -| `import!("name")` | Generates `mod name` from `imports/name.json`. | -| `state_dir()`, `write_atomic(path, bytes)`, `pid()` | Helpers. | -| `#[sausage_plugin::test]` | An async test. A plain `#[test]` also works. | - -Rules that the compiler enforces: - -- A method takes `&self`, an optional `ctx: &Ctx`, and at most one params - argument. -- The params argument is a `#[schema]` struct with named fields. Its fields - are the params of the method. -- Each method, each `#[schema]` type, and each field has a doc comment. - Other agents read these docs to use the interface. Write them like good - Rust documentation. - -Other rules: - -- Method docs become `summary` (the first paragraph) and `description` in - OpenRPC. Type and field docs become `description` in the JSON Schema. The - generated imports contain them again as doc comments. -- Each `import!` registers its document in a static list with the - `inventory` crate. `run` reads the list for `rpc.discover`. `import!` uses - `include_str!`, so cargo rebuilds the plugin when the file changes. -- The value that `run` receives must be cheap to create, because the core - also starts a plugin only for `rpc.discover`. Open files and databases in - the methods. -- Log with `eprintln!`. stdout belongs to the protocol. - -### 13.1 A plugin - -```toml -[package] -name = "notes" -version = "0.1.0" -edition = "2024" - -[dependencies] -sausage-plugin = "0.1" -serde_json = "1" -``` - -```rust -//! Notes: a list of short texts, stored in `/state/notes/notes.json`. -//! -//! Each call runs in its own process, so two calls can run at the same time. -//! `add` takes a lock on the file, so that no note gets lost. - -use std::path::PathBuf; - -use sausage_plugin::{Result, lock, rpc, schema, state_dir, write_atomic}; - -/// The notes plugin. It keeps no state in memory: each call reads the file. -struct Notes; - -/// The params of `add`. -#[schema] -struct AddParams { - /// The text of the new note. It can contain newlines. - text: String, -} - -/// The result of `add`. -#[schema] -struct Added { - /// The number of notes after the call, with the new note. - count: usize, -} - -#[rpc] -impl Notes { - /// Add a note at the end of the list. - /// - /// Returns the number of notes after the call. - async fn add(&self, AddParams { text }: AddParams) -> Result { - let _lock = lock(file()).await?; // released on drop, or when the process ends - let mut notes = load()?; - notes.push(text); - write_atomic(file(), serde_json::to_vec(¬es)?)?; - Ok(Added { count: notes.len() }) - } - - /// List all notes, the oldest first. - fn list(&self) -> Result> { - load() // no lock: write_atomic replaces the file in one step - } -} - -/// The file with the notes: a JSON list of strings. -fn file() -> PathBuf { - state_dir().join("notes.json") -} - -/// Read all notes. A missing file means no notes. -fn load() -> Result> { - match std::fs::read(file()) { - Ok(bytes) => Ok(serde_json::from_slice(&bytes)?), - Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(vec![]), - Err(e) => Err(e.into()), - } -} - -fn main() { - sausage_plugin::run(Notes); -} - -#[cfg(test)] -mod tests { - use super::*; - - #[sausage_plugin::test] - async fn add_appends_and_counts() { - assert_eq!(Notes.add(AddParams { text: "milk".into() }).await.unwrap().count, 1); - assert_eq!(Notes.add(AddParams { text: "eggs".into() }).await.unwrap().count, 2); - assert_eq!(Notes.list().unwrap(), ["milk", "eggs"]); - } - - #[test] - fn list_is_empty_without_a_file() { - assert!(Notes.list().unwrap().is_empty()); - } -} -``` - -### 13.2 A server with an endpoint - -```rust -//! A web server for the files in `/state/www/`. -//! -//! `start` starts the server in its own process and returns at once. Run it -//! as a service, or call it detached, or the server ends with its caller. -//! The running server serves the endpoint `control` at -//! `/proc//control`. - -use std::{cell::Cell, rc::Rc}; - -use sausage_plugin::{Ctx, Result, background, listen, pid, rpc, schema}; -use wstd::net::TcpListener; - -/// The www plugin. -struct Www; - -/// The params of `start`. -#[schema] -struct StartParams { - /// The TCP port to listen on, on all interfaces. - port: u16, -} - -#[rpc(endpoints(control = Control), grace_seconds = 5)] -impl Www { - /// Start the web server and return the ID of its process. - /// - /// The server runs until it gets `stop` on its endpoint, until - /// `core.stop`, or until its caller ends. Fails if the port is in use. - async fn start(&self, StartParams { port }: StartParams) -> Result { - let tcp = TcpListener::bind(&format!("0.0.0.0:{port}")).await?; - let control = Rc::new(Control { hits: Cell::new(0) }); - listen("/self/control", control.clone()).await?; - background(serve(tcp, control)); // the process lives while this task runs - Ok(pid()) - } -} - -/// The control endpoint of a running server. -struct Control { - /// The number of HTTP requests that the server answered. - hits: Cell, -} - -#[rpc] -impl Control { - /// The number of HTTP requests that the server answered since its start. - fn stats(&self) -> Result { - Ok(self.hits.get()) - } - - /// Stop the server. The process ends after this response. - fn stop(&self, ctx: &Ctx) -> Result<()> { - ctx.exit_after_response(); - Ok(()) - } -} - -/// Accept HTTP connections, and answer each request from `/state/www/`. -async fn serve(tcp: TcpListener, control: Rc) { - // … -} - -fn main() { - sausage_plugin::run(Www); -} -``` - -The same type can serve stdin and an endpoint. The example uses two types -for a reason: `www.stats` through the core would start a fresh process, and -that process would answer 0. - -### 13.3 A service with a fixed endpoint - -```rust -//! Secrets: keeps secrets in memory, for other plugins. -//! -//! Run it as a service: `{"method": "secrets.serve", "params": {}}` in -//! `/config/boot.json`. Clients connect to the endpoint `vault` at -//! `/run/secrets`. - -use std::rc::Rc; - -use sausage_plugin::{Result, listen, rpc}; - -/// The secrets plugin. -struct Secrets; - -#[rpc(endpoints(vault = Vault))] -impl Secrets { - /// Open the vault and serve it at `/run/secrets`. - /// - /// Returns when the endpoint is ready. The process then lives while the - /// endpoint listens. - async fn serve(&self) -> Result<()> { - listen("/run/secrets", Rc::new(Vault::open()?)).await - } -} -``` - -A client imports `secrets` and calls -`secrets::vault::connect("/run/secrets").await?`. - -### 13.4 Imports - -To use another plugin, an agent does these steps: - -1. Copy `/proc/core/plugins/notes/discover.json` to - `source/imports/notes.json`. -2. Read `/proc/core/plugins/notes/client.rs`: the exact code that the import - generates. -3. Write `import!("notes")` and the code that uses it. -4. Call `cargo.build`, and remove the imported methods that the warnings call - unused. -5. Call `core.activate`. - -```rust -sausage_plugin::import!("notes"); // imports/notes.json -sausage_plugin::import!("www"); // imports/www.json, with its endpoint "control" - -/// Add a note, then start a web server and return its number of answered requests. -async fn example() -> Result { - let added = notes::add(notes::AddParams { text: "milk".into() }).await?; - eprintln!("{} notes", added.count); // stderr goes to the journal - let pid = www::start(www::StartParams { port: 8080 }).detach().await?; - let control = www::control::connect(&format!("/proc/{pid}/control")).await?; - control.stats().await -} -``` - -- The params of each method are one struct. The fields have the same names as - the JSON params. -- Types take their names from the schemas. When the provider uses - `#[schema]`, those are the names of its Rust types: `AddParams` stays - `AddParams`. -- The generated items are private to the crate, so rustc warns about each - unused method. - -## 14. The agent - -- `agent.chat {"text": "..."}` answers a message from a person. The result is - the final text. The interim texts come as `core.notify` with the name - `text`. -- The agent takes a lock on `/state/agent/history.jsonl`. So a second message - waits until the first turn has ended. -- The agent declares a long grace period (for example 10 minutes), so that a - turn can finish after an upgrade. A new version applies at the next - `chat`. -- The tools: `call` (method, params, detach), `read_file`, `write_file`, - `list_dir`, and `copy_dir`. -- The configuration is in `/config/agent.json`. -- The prompt stays short. It says: - - where the contract is (`/proc/core/PROTOCOL.md`), - - where the SDK is (`/plugins/cargo/sdk/`), - - how to build and activate a plugin, - - where the agent's own source is (`/plugins/agent/source/`), - - that logs go to stderr, - - that the interface check does not cover state. - -## 15. The console - -- `rustyline` gives line editing and history. Notifications appear while you - type. -- A short prompt. Results are indented. Agent notifications are dimmed. - -| Input | Effect | -| --- | --- | -| text | `agent.chat` | -| `. {json}` | a call. The console prints the result. | -| `ls [path]`, `cat ` | the filesystem as a process sees it, with `/proc` and `/run` | -| `methods ` | the methods from `rpc.discover`, with params and docs | -| `ps` | the running processes | -| `log [plugin or pid] [-n N]` | the journal | -| `activate `, `revert ` | `core.activate`, `core.revert` | -| `stop `, `kill ` | `core.stop`, `core.kill` | -| `help`, `quit` | | - -`--headless` stays: no console input, and the host runs until SIGTERM or -`core.exit`. - -## 16. Caches - -- Wasmtime's own compile cache stores the compiled code of each `.wasm` file. - It supports components. -- The folder is `$XDG_CACHE_HOME/sausage/`, or `~/.cache/sausage/` without - that variable. `--cache-dir` changes the folder. - -## 17. Decisions - -- **JSON-RPC, not component interfaces.** Every language can read and write - lines of JSON, and models know JSON-RPC from LSP and MCP. -- **A process per call.** We get parallelism without threads in WASI. A new - version applies at the next call, and old processes retire after a grace - period. -- **Attached by default.** A process does not outlive its caller by - accident. -- **Services in one file.** Only the core starts and restarts them, so - upgrades and singletons need no logic in plugins. -- **Files to inspect, calls to act.** This keeps the core API small, and the - agent's file tools are enough to explore the world. -- **Generated imports and interface checks, without version numbers.** A model - that translates a description by hand makes errors. A generator does not. - The check catches the most common error in a world that changes itself: a - change to one plugin breaks another one. -- **A small schema subset.** With Draft 7 and a fixed subset, our own check - can decide every case. -- **Docs are part of the interface.** Other agents work from them, so the - compiler requires them. -- **No permissions.** The checks protect against mistakes, not against - plugins. Locks are advisory. -- **Our own VFS, in its own crate.** It works on Linux and macOS, it makes - `/proc` live, and plugin-served files need it later. Mounts now, overlays - later. -- **TCP endpoints for now.** They need no host code. VFS pipes can replace - them later behind the same SDK API. -- **Simple scheduling for now.** An epoch tick every 10 milliseconds keeps - the core responsive and makes kills reliable. - -## 18. Later - -- Plugin-served files at `/proc//fs/`, for example for an FTP plugin. A - request gets a timeout of 10 seconds, because a process that reads its own - served files would wait for itself forever. -- Overlays in the VFS. -- VFS pipes for endpoints, instead of TCP ports. -- Subscriptions to file changes. -- Structured logging. -- A secrets service. -- Cancellation of calls. -- A better scheduler. - -## 19. Plan - -The work happens on the branch `plugins`. Each step includes its tests. - -Already confirmed: - -- `inventory` works on `wasm32-wasip2`. -- Wasmtime's compile cache supports components. -- C code builds for `wasm32-wasip2`. - -Checks of risks first: - -1. An end-to-end pair: a provider and a consumer with a generated import, - through the real transport. This includes `typify` with Draft 7 from - schemars, and our compatibility check on real documents. -2. Fixture paths in tests under the runner. - -Then the steps: - -1. `PROTOCOL.md` and `DECISIONS.md` from this proposal. -2. `sausage-vfs`: the trait, the layers, the Wasmtime adapter, and its tests. -3. The core: a process per call, JSON-RPC routing, the process tree, upgrades, - services, locks, the journal, scheduling, and the compile cache. -4. The SDK and its macros. -5. `cargo.build`, the test runner, activation, and the graph. -6. The agent and the console. diff --git a/sdk/imports/notes.json b/sdk/imports/notes.json new file mode 100644 index 0000000..33267ba --- /dev/null +++ b/sdk/imports/notes.json @@ -0,0 +1,25 @@ +{ + "openrpc": "1.2.6", + "info": {"title": "notes", "version": "0.1.0"}, + "components": {"schemas": { + "AddParams": { + "description": "The parameters of add.", + "type": "object", + "properties": {"text": {"type": "string", "description": "The text to append."}}, + "required": ["text"] + }, + "Added": { + "description": "The result of add.", + "type": "object", + "properties": {"count": {"type": "integer", "minimum": 0, "description": "The total number of notes."}}, + "required": ["count"] + } + }}, + "methods": [{ + "name": "add", + "summary": "Append a note and return the new count.", + "params": [{"name": "text", "required": true, "schema": {"type": "string"}}], + "x-sausage-params": {"$ref": "#/components/schemas/AddParams"}, + "result": {"name": "result", "schema": {"$ref": "#/components/schemas/Added"}} + }] +} diff --git a/sdk/imports/www.json b/sdk/imports/www.json new file mode 100644 index 0000000..9643143 --- /dev/null +++ b/sdk/imports/www.json @@ -0,0 +1,25 @@ +{ + "openrpc": "1.2.6", + "info": {"title": "www", "version": "0.1.0"}, + "components": {"schemas": { + "StartParams": { + "description": "The server configuration.", + "type": "object", + "properties": {"port": {"type": "integer", "format": "uint16", "minimum": 0, "maximum": 65535, "description": "The HTTP port."}}, + "required": ["port"] + } + }}, + "methods": [{ + "name": "start", + "summary": "Start the HTTP server and return its process ID.", + "params": [{"name": "port", "required": true, "schema": {"type": "integer", "format": "uint16", "minimum": 0, "maximum": 65535}}], + "x-sausage-params": {"$ref": "#/components/schemas/StartParams"}, + "result": {"name": "result", "schema": {"type": "string"}} + }], + "x-sausage-endpoints": {"control": {"methods": [{ + "name": "stats", + "summary": "Return the number of HTTP requests.", + "params": [], + "result": {"name": "result", "schema": {"type": "integer", "format": "uint64", "minimum": 0}} + }]}} +} diff --git a/sdk/src/lib.rs b/sdk/src/lib.rs index 1f076da..ba2a5b7 100644 --- a/sdk/src/lib.rs +++ b/sdk/src/lib.rs @@ -1,52 +1,390 @@ //! Write single-threaded WASI plugins for Sausage. //! -//! Each core call starts a process. Keep its initial value cheap to construct. -//! Declare documented parameter and result types with [`macro@schema`]. -//! Declare methods with [`macro@rpc`], then serve them with [`run`]. +//! # Start here +//! +//! A plugin depends on `sausage-plugin = "0.1"` in its `Cargo.toml`. +//! Use Rust edition 2024. The cargo plugin supplies the local SDK dependency. +//! The SDK re-exports serde, schemars, serde_json, and its macros. +//! Build the plugin through `cargo.build`. Activate the returned WASM file through `core.activate`. +//! Read `/proc/core/PROTOCOL.md` for the host contract. +//! +//! Each core call starts a process. Keep the value passed to [`run`] cheap to construct. +//! The core also starts a process just for `rpc.discover`. +//! Open files, databases, and listeners in methods, not in `main`. +//! Log with `eprintln!`. Stdout belongs to JSON-RPC. +//! +//! # Methods and schemas +//! +//! Each `fn` in an [`macro@rpc`] impl becomes an RPC method. +//! Put helpers in a separate impl block without `#[rpc]`. +//! Both synchronous and async methods must follow these rules: +//! +//! - Take `&self` first. +//! - Optionally take `ctx: &Ctx` next. +//! - Take at most one parameter argument after that. +//! - Use a [`macro@schema`] struct with named fields for that argument. +//! - Return [`Result`]. +//! - Give each method, schema type, and field a doc comment. +//! +//! The parameter fields become the named JSON parameters. +//! A method without parameters accepts an empty object. +//! Its discovery entry has no `x-sausage-params` field. +//! Raw identifiers such as `r#type` use the plain wire name `type`. +//! Method docs become the OpenRPC summary and description. +//! Type and field docs become schema descriptions. Generated clients retain these docs. +//! Docs are part of the interface. Other agents read them to write callers. +//! +//! `#[schema]` adds serde and schemars derives. Do not add those derives yourself. +//! These types form the supported subset: +//! +//! - `String`, `bool`, and `()`. +//! - `u8`, `u16`, `u32`, `u64`, and `usize`. +//! - `i8`, `i16`, `i32`, `i64`, and `isize`. +//! - `f32` and `f64`. +//! - `Vec` and `Option` with supported element types. +//! - Other `#[schema]` structs and enums. +//! +//! Structs must have named fields. Enums must have unit variants. +//! Document the variants to explain their meanings to callers. +//! The permitted serde attributes are `rename`, `rename_all`, `default`, and `skip_serializing_if`. +//! For example, use `#[serde(default)]` for a field with a default value. +//! Use `#[serde(default, skip_serializing_if = "Option::is_none")]` for an omitted optional field. +//! `#[schema]` rejects other serde attributes and all `#[schemars(...)]` attributes. +//! Tuple structs, unit structs, tuple fields, maps, arbitrary JSON values, and data-carrying variants are outside this subset. +//! Such fields cannot appear in a `#[schema]` type. +//! +//! # A persistent plugin +//! +//! Calls can run concurrently in separate processes. Protect shared changes with [`lock`]. +//! Bind the guard to a name: `let _guard = lock(p).await?;`. +//! `let _ = lock(p).await?;` releases the lock at once. +//! The guard releases the lock when its scope ends. Process exit also releases locks. +//! `try_now()` does not wait. Check the returned guard with `held()`. +//! `keep()` retains an acquired lock until process exit. +//! +//! This complete plugin stores notes in [`state_dir`]. [`write_atomic`] replaces the file after a complete temporary write. +//! Create the state directory before the first write. //! //! ```no_run -//! use sausage_plugin::{rpc, schema, Result}; -//! struct Calculator; -//! /// The numbers to add. +//! use sausage_plugin::{lock, rpc, schema, state_dir, write_atomic, Result}; +//! use sausage_plugin::serde_json; +//! use std::path::PathBuf; +//! +//! /// Persistent notes shared by all calls. +//! struct Notes; +//! /// The parameters of add. //! #[schema] //! struct AddParams { -//! /// The first number. -//! a: i32, -//! /// The second number. -//! b: i32, +//! /// The text to append. Newlines are permitted. +//! text: String, +//! } +//! /// The result of add. +//! #[schema] +//! struct Added { +//! /// The total number of notes. +//! count: usize, //! } //! #[rpc] -//! impl Calculator { -//! /// Add two numbers. -//! fn add(&self, p: AddParams) -> Result { Ok(p.a + p.b) } +//! impl Notes { +//! /// Append a note and return the new count. +//! async fn add(&self, p: AddParams) -> Result { +//! std::fs::create_dir_all(state_dir())?; +//! let _guard = lock(Self::file()).await?; +//! let mut notes = Self::load()?; +//! notes.push(p.text); +//! write_atomic(Self::file(), serde_json::to_vec(¬es)?)?; +//! Ok(Added { count: notes.len() }) +//! } +//! /// List the notes, oldest first. +//! fn list(&self) -> Result> { Self::load() } //! } -//! fn main() { sausage_plugin::run(Calculator); } +//! // These helpers do not become RPC methods. +//! impl Notes { +//! fn file() -> PathBuf { state_dir().join("notes.json") } +//! fn load() -> Result> { +//! match std::fs::read(Self::file()) { +//! Ok(bytes) => Ok(serde_json::from_slice(&bytes)?), +//! Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(vec![]), +//! Err(e) => Err(e.into()), +//! } +//! } +//! } +//! fn main() { sausage_plugin::run(Notes); } //! ``` //! -//! Use [`call`] for outgoing calls. Await its builder for the JSON result. -//! Use `detach()` to detach the child and `on_notify(f)` to receive progress. -//! Use [`Ctx::notify`] to report progress to your caller. -//! Use [`Ctx::exit_after_response`] to end the process after the response. +//! # Calls and process lifetime //! -//! [`background`] keeps the process alive while a task runs. -//! At stdin EOF, pending calls fail and the optional [`macro@on_stop`] hook runs. -//! The stop hook can write files and send notifications, but cannot make calls. -//! [`macro@on_retired`] handles `rpc.retired` after an upgrade. +//! [`call`] returns a lazy builder. Await it to send the request and receive its JSON result. +//! `detach()` lets the child outlive this process. `on_notify(f)` receives its progress notifications. +//! Use [`Ctx::notify`] to send progress to the caller of the current request. +//! Do not call [`block_on`] in a handler. The runtime cannot nest `block_on` calls. +//! Write an async handler and use `.await` instead. //! -//! Use [`lock`] before shared state changes. A guard releases its lock on drop. -//! `try_now()` returns a guard whose `held()` method reports success. -//! `keep()` retains a lock until process exit. -//! [`state_dir`], [`write_atomic`], and [`pid`] provide filesystem and identity helpers. -//! Use [`macro@test`] for async tests. Native tests fail clearly without a core transport. -//! [`macro@import`] reads `imports/.json` and generates a private typed client. -//! Its functions return [`TypedCall`] builders with progress and detach options. -//! Declare endpoint types with `#[rpc(endpoints(control = Control))]`. -//! [`listen`] publishes a declared interface at an endpoint file. -//! [`connect`] verifies its owner before it returns an [`Endpoint`]. -//! Endpoint calls report progress through [`EndpointCall::on_notify`]. -//! Local transport errors use code -32000: no transport, stdin EOF, a closed endpoint, -//! or an endpoint identity or interface mismatch. Code -32002 belongs to core process failures. -//! Log with `eprintln!`. Stdout belongs to JSON-RPC. +//! A process ends after its request finishes if it has no remaining work. +//! [`background`] keeps it alive while a task runs. An endpoint listener also keeps it alive. +//! Neither mechanism detaches the process from its caller. Start long-lived processes detached or as services. +//! At stdin EOF, pending outgoing calls fail with -32000. The SDK stops endpoint listeners and runs the optional stop hook. +//! The stop hook can write files and send notifications. It cannot make new calls. +//! After the serve loop, the SDK flushes stdout and exits, even if background tasks remain. +//! [`Ctx::exit_after_response`] ends the process after its response. This also works in an endpoint handler. +//! +//! ```no_run +//! use sausage_plugin::{background, call, rpc, Ctx, Result}; +//! use sausage_plugin::serde_json::json; +//! /// A worker that delegates a job. +//! struct Worker; +//! #[rpc] +//! impl Worker { +//! /// Start an attached background task and return immediately. +//! fn start(&self, ctx: &Ctx) -> Result<()> { +//! ctx.notify("text", "worker started")?; +//! background(async { +//! let result = call("notes.list", json!({})) +//! .on_notify(|name, params| eprintln!("{name}: {params}")) +//! .await; +//! if let Err(error) = result { eprintln!("{error}"); } +//! }); +//! Ok(()) +//! } +//! /// Flush local files before this process stops. +//! #[sausage_plugin::on_stop] +//! fn stop(&self) -> Result<()> { +//! eprintln!("worker stopped"); +//! Ok(()) +//! } +//! } +//! fn main() { sausage_plugin::run(Worker); } +//! ``` +//! +//! # A server with a process endpoint +//! +//! Declare endpoint types on the main interface. Publish them with [`listen`]. +//! `listen` returns after it publishes the endpoint file, not after the listener ends. +//! `/self/control` belongs to this process. Clients find it at `/proc//control`. +//! This server counts requests in memory. A new core call cannot inspect the old process's counter. +//! +//! ```no_run +//! use sausage_plugin::{listen, pid, rpc, Ctx, Result}; +//! use std::{cell::Cell, rc::Rc}; +//! /// A server with process-local counters. +//! struct Server; +//! /// The interface of one running server. +//! struct Control { hits: Cell } +//! #[rpc(endpoints(control = Control), grace_seconds = 5)] +//! impl Server { +//! /// Start a counter server and return its process ID. +//! async fn start(&self) -> Result { +//! listen("/self/control", Rc::new(Control { hits: Cell::new(0) })).await?; +//! Ok(pid()) +//! } +//! } +//! #[rpc] +//! impl Control { +//! /// Record a request and return the new count. +//! fn record(&self) -> Result { +//! self.hits.set(self.hits.get() + 1); +//! Ok(self.hits.get()) +//! } +//! /// Return the number of recorded requests. +//! fn stats(&self) -> Result { Ok(self.hits.get()) } +//! /// Stop this process after the client receives the response. +//! fn stop(&self, ctx: &Ctx) -> Result<()> { +//! ctx.exit_after_response(); +//! Ok(()) +//! } +//! } +//! fn main() { sausage_plugin::run(Server); } +//! ``` +//! +//! # A service with a fixed endpoint +//! +//! A service can publish a fixed path in `/run`. The core clears `/run` when the host starts. +//! Configure this service in `/config/boot.json` with `{"method":"secrets.serve","params":{}}`. +//! Clients can then connect to `/run/secrets`. +//! +//! ```no_run +//! use sausage_plugin::{listen, rpc, schema, Result}; +//! use std::rc::Rc; +//! /// The service entry point. +//! struct Secrets; +//! /// The in-memory vault of this service process. +//! struct Vault { token: String } +//! /// The name of a secret. +//! #[schema] +//! struct GetParams { +//! /// The secret to read. +//! name: String, +//! } +//! #[rpc(endpoints(vault = Vault))] +//! impl Secrets { +//! /// Load the vault and publish its fixed endpoint. +//! async fn serve(&self) -> Result<()> { +//! let token = std::fs::read_to_string("/config/secret-token")?; +//! listen("/run/secrets", Rc::new(Vault { token })).await +//! } +//! } +//! #[rpc] +//! impl Vault { +//! /// Read the token, or return None for an unknown name. +//! fn get(&self, p: GetParams) -> Result> { +//! Ok((p.name == "token").then(|| self.token.clone())) +//! } +//! } +//! fn main() { sausage_plugin::run(Secrets); } +//! ``` +//! +//! [`connect`] reads the endpoint file and verifies its owner through `rpc.endpoint`. +//! The handshake checks the plugin, owner process ID, and interface name. +//! A generated endpoint client also requires its expected interface name. +//! A mismatch returns -32000. Endpoint calls bypass a fresh core process. +//! `connect` opens and closes a verification connection. Each call opens another connection and repeats the handshake. +//! The first call costs three round trips: verification, another handshake, and the request. +//! Later calls cost a handshake and a request each. The SDK does not pool connections. +//! +//! ```no_run +//! use sausage_plugin::{connect, Result}; +//! use sausage_plugin::serde_json::json; +//! async fn read_token() -> Result> { +//! let vault = connect("/run/secrets").await?; +//! vault.require_interface("vault")?; +//! vault.call("get", json!({"name":"token"})) +//! .on_notify(|name, params| eprintln!("{name}: {params}")) +//! .await +//! } +//! fn main() {} +//! ``` +//! +//! # Import another plugin +//! +//! 1. Copy `/proc/core/plugins//discover.json` to `source/imports/.json` in your plugin folder. +//! 2. Read `/proc/core/plugins//client.rs` for the exact generated client API. +//! 3. Write `sausage_plugin::import!("")` and the code that calls it. +//! 4. Build with `cargo.build`. Remove methods that rustc calls unused from the imported JSON document. +//! 5. Build again after that change. Call `core.activate` with the resulting WASM path. +//! +//! The macro reads `imports/.json` relative to `CARGO_MANIFEST_DIR`, not the current folder. +//! It generates a private module with typed [`TypedCall`] builders. +//! The module retains schema type names and documentation. Await its functions to get their typed results. +//! Builders support `detach()` and `on_notify(f)`. Generated endpoint modules provide typed `connect` functions. +//! The macro tracks the JSON file with `include_str!`, so Cargo rebuilds after changes. +//! It registers the imported contract for `rpc.discover` through inventory. +//! Remove unused methods from the JSON, not from generated Rust. This keeps `x-sausage-imports` small. +//! Retain the schemas that the remaining methods reference. +//! +//! This example imports the notes plugin above and a server with a `control` endpoint. +//! The `www` provider exposes `start(StartParams { port })` and `control.stats()`. +//! +//! ```no_run +//! use sausage_plugin::{rpc, Result}; +//! sausage_plugin::import!("notes"); +//! sausage_plugin::import!("www"); +//! /// A consumer of notes and the HTTP server. +//! struct Consumer; +//! #[rpc] +//! impl Consumer { +//! /// Add a note, start a detached server, and return its request count. +//! async fn example(&self) -> Result { +//! let added = notes::add(notes::AddParams { text: "milk".into() }).await?; +//! eprintln!("{} notes", added.count); +//! let pid = www::start(www::StartParams { port: 8080 }).detach().await?; +//! let control = www::control::connect(&format!("/proc/{pid}/control")).await?; +//! control.stats().await +//! } +//! } +//! fn main() { sausage_plugin::run(Consumer); } +//! ``` +//! +//! # Tests in a world +//! +//! `cargo.build` runs tests under the WASI test runner. +//! Both plain `#[test]` and async [`macro@test`] tests work. Each test gets its own instance and empty temporary world at `/`. +//! The current folder is `/`, not the package folder. Read fixtures through `env!("CARGO_MANIFEST_DIR")`. +//! `SAUSAGE_PLUGIN` is `test`, so `state_dir()` returns `/state/test`. +//! Create directories before writes. `std::env::temp_dir()` does not work under this runner. +//! Use an explicit path in the temporary world instead. +//! The runner answers core lock, unlock, notify, and log requests. Calls to other plugins fail because no other plugins exist. +//! Name filters, `tests/*.rs`, and `--ignored` work. `#[should_panic]` cannot work because WASI cannot catch panics. +//! +//! ```no_run +//! use sausage_plugin::{lock, state_dir, write_atomic}; +//! #[test] +//! fn reads_package_fixture() { +//! let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("Cargo.toml"); +//! assert!(std::fs::read_to_string(path).unwrap().contains("[package]")); +//! } +//! #[sausage_plugin::test] +//! async fn writes_local_state() { +//! let file = state_dir().join("test.txt"); +//! std::fs::create_dir_all(state_dir()).unwrap(); +//! let _guard = lock(&file).await.unwrap(); +//! write_atomic(&file, b"saved").unwrap(); +//! assert_eq!(std::fs::read(file).unwrap(), b"saved"); +//! } +//! fn main() {} +//! ``` +//! +//! # Errors +//! +//! [`Error`] carries `code`, `message`, and optional structured `data`. +//! `Error::new` constructs an error without data. Assign `data` when callers need structured details. +//! Do not reduce an error to its message when another agent needs its code or data. +//! +//! | Code | Meaning | +//! | --- | --- | +//! | -32700 | Invalid JSON. | +//! | -32600 | Invalid JSON-RPC request. | +//! | -32601 | Unknown method. | +//! | -32602 | Invalid method parameters. | +//! | -32603 | Internal error, including converted I/O and serde errors. | +//! | -32000 | Local SDK error: no transport, stdin EOF, endpoint closure, or identity or interface mismatch. | +//! | -32001 | Core: no such plugin. | +//! | -32002 | Core: the process ended without an answer. Data includes its exit status and journal tail. | +//! | -32003 | Core: activation failed. Data describes the reasons. | +//! +//! Parameter decoding failures use -32602 instead of the general serde conversion code. +//! +//! ``` +//! use sausage_plugin::Error; +//! use sausage_plugin::serde_json::json; +//! let mut error = Error::new(-32602, "unknown secret"); +//! error.data = Some(json!({"name":"missing"})); +//! assert_eq!(error.code, -32602); +//! assert_eq!(error.data.unwrap()["name"], "missing"); +//! ``` +//! +//! # State and upgrades +//! +//! The interface check covers calls, not state. Old and new processes share files in `/state`. +//! `core.revert` restores code, not state. Preserve compatibility or migrate the state. +//! `#[rpc(grace_seconds = N)]` declares the grace period for old processes. The default is 60 seconds. +//! New calls use the new version immediately. Old processes can still use old imports during their grace period. +//! [`macro@on_retired`] handles `rpc.retired` after the last process of an old version ends. +//! This hook is best effort. If the host stops first, the hook does not run. +//! New processes must handle unmigrated state. Check a migration marker and migrate if it is absent. +//! See section 3.7 of `/proc/core/PROTOCOL.md`. +//! Declare at most one stop hook and one retirement hook per interface. +//! +//! ```no_run +//! use sausage_plugin::{rpc, schema, state_dir, write_atomic, Result}; +//! /// A plugin that records retirement notices. +//! struct Migrator; +//! /// The version that no longer has running processes. +//! #[schema] +//! struct RetiredParams { +//! /// The SHA-256 of the previous WASM file. +//! previous: String, +//! } +//! #[rpc] +//! impl Migrator { +//! /// Record a retired version. Normal methods must also handle missing markers. +//! #[sausage_plugin::on_retired] +//! fn retired(&self, p: RetiredParams) -> Result<()> { +//! std::fs::create_dir_all(state_dir())?; +//! write_atomic(state_dir().join("retired-version"), p.previous.as_bytes()) +//! } +//! } +//! fn main() { sausage_plugin::run(Migrator); } +//! ``` #![warn(missing_docs)] mod discovery; mod endpoints;