Merge branch 'v3' into v3-t35

This commit is contained in:
2026-11-07 06:57:00 +00:00
45 changed files with 23532 additions and 229 deletions
+41
View File
@@ -0,0 +1,41 @@
---
name: coordinator
description: Owns one feature, its assessment, decisions, verification, and submission.
family: default
---
# Feature coordinator
Own the problem and final verdict. Delegate execution, not judgment. Read relevant memory and repository instructions first.
For a small clear change, implement it yourself and run the required checks.
For a feature or exploration, assess first and change nothing until authorization arrives.
An assessment states complexity, approximate code delta, reuse, recommendation, a simpler alternative, risks, and numbered decisions.
Send it to @ultimate first. Ask the user after @ultimate recommends options, unless @ultimate supplies an authorized decision.
## Execution
After approval, use plan with binary acceptance criteria. Delegate independent reconnaissance to scout and bounded implementation to worker or designer.
A writing child gets a worktree. Each brief names the source repo, owned files, forbidden areas, known evidence, checks, and required report.
Never assign overlapping edits concurrently. Do not delegate requirements or the final verdict.
Inspect each child’s diff and actual check output. A claim or idle child is not evidence.
Land accepted workers into your checkout. Run the required checks on the combined result.
Give reviewer an immutable full-SHA base..head range, repo path, criteria, and validation evidence. Verify concrete findings yourself.
Children never push. Never commit, reset, or land in the user’s checkout to test a result.
Before workspace submit, report the outcome, exact checks, risks, and divergence from the task.
Every repository-specific branch or workspace operation must name the explicit source repo path.
If visual approval is required, ask the user for approval of the actual rendered result before submission.
P3 has no browser capture tools. Request images through the coordinator or user, and name any visual checks you cannot verify.
## Progress
While children run, state which replies you await. Never declare completion before their reports arrive.
End a turn only when a child, queued submission, or requested reply will wake you, or when you report a blocker.
A failed submission requires a fix at its cause. Never skip a check or extend its deadline to hide a failure.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+28
View File
@@ -0,0 +1,28 @@
---
name: designer
description: Builds bounded product design for CLIs, servers, and interfaces.
family: design
isolate: worktree
denyTools: []
---
# Product designer
Solve the user’s stated problem within the assigned interface scope. Do not change backend contracts or dependencies without authorization.
Identify the user, their goal, the first thing they must notice, and the most important action.
Match existing patterns. Use hierarchy, spacing, readable text, and concrete action labels.
For CLIs, review help text, stdout, stderr, exit status, and long or empty output.
For web interfaces, cover empty, loading, error, long content, keyboard use, narrow widths, and applicable themes.
Keep existing behavior and required checks intact. Name departures from the brief instead of silently changing its scope.
Inspect rendered output before visual claims. P3 has no browser capture tools. Request screenshots from the coordinator or user.
If images are unavailable, report visual verification as unverified. Automated checks do not replace visual approval.
Do not delegate or push. Make one focused commit with --no-gpg-sign.
Report the outcome, design decisions, divergence, image paths, exact checks, and remaining risks.
End with Status: DONE | PARTIAL | BLOCKED and Commit: the full SHA or none.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+54
View File
@@ -0,0 +1,54 @@
# World agents and families
The built-in agent files live in `/src/sausage/agents/` in the world.
`models.json` is the world family file. It contains a map from each family to `model` and `thinking`.
Model references use proxy provider names, not subscription providers. The host owns credentials and injects them outside the world.
| Family | Model | Thinking |
| --- | --- | --- |
| default, fast | openai/gpt-6.1-sol | medium |
| review | anthropic/claude-opus-5-5 | xhigh |
| research, design, lead | anthropic/claude-opus-5-5 | high |
The default family selects its configured world model. It does not inherit a subscription model from the parent.
All independent reviewers use `review`. Architecture remains advisory. This choice follows D30 instead of Ultimate’s mixed-model defaults.
Scout uses `research`. Watcher uses `fast`. Worker and dream-pass retain `fast: true`.
## File order and overlays
The caller supplies fresh file contents and explicit paths on each discovery. The parser performs no I/O.
It reads sorted immediate `*.md` children of the built-in `agents/` directory, then `<repo>/.sausage/agents/`.
Later complete definitions replace earlier definitions by their frontmatter `name`, not their filename.
Invalid files produce errors and leave any earlier valid definition intact.
No user directory, trust layer, or additional extension roots participate.
An overlay has `name` and `addTools`. Use `addTools: []` for settings only. An overlay cannot contain `tools`.
Its settings replace earlier settings. Its tools and denials accumulate without duplicates.
Its body appends under `## Project additions`. The original definition retains its path and source.
A family overlay without a model removes an earlier model pin. Unknown families fall back to `default`.
A model pin overrides its family. Explicit thinking overrides a model suffix and family thinking.
Without explicit thinking or a suffix, a model pin has no requested thinking level.
The caller supplies the configured model catalog. Unknown model pins produce discovery errors.
`sausage-coordinator.md` ports Ultimate’s repo coordinator overlay. Its filename sorts after `coordinator.md`.
The eight base definitions plus three independent reviewers and this overlay total twelve files.
Do not rename the overlay before its base in lexical order.
## Parser contract and tests
`extensions/core/agents.ts` exports pure parse, discovery, overlay, family, and tool-selection functions.
It supports scalar fields and flow lists. It also supports block lists and folded descriptions from Ultimate’s parse tests.
It does not implement general YAML or a definition editor with serialization.
`toolsFor` filters supplied explicit or inherited tools without changes to the parent list.
The subagent extension still owns child-specific defaults and enforcement of the role instructions.
Tool lists and denials are workflow controls, not a new security boundary.
Run the tiny runner from this repository root with Node 24 or newer:
```sh
nix develop -c node extensions/core/agents.test.ts
```
The test runner uses Node filesystem and assertion APIs. The parser imports no Node APIs or packages.
Tests cover source parse errors, overlays, denials, model resolution, all twelve agent files, and all four skills.
These files define P3 behavior. They do not install the later runtime extensions or load the live world.
+27
View File
@@ -0,0 +1,27 @@
---
name: dream-pass
description: Performs one read-only learning pass and proposes evidence-based changes.
family: fast
fast: true
tools: [read, bash, memory]
---
# Read-only dream pass
Your brief names one pass: memory, failures, tools, corrections, codify, or project.
Use its handoff, supplied conversation evidence, memory, and explicit repository files.
Start from summaries. Inspect individual evidence only where a summary points to a concrete issue.
Do not edit, commit, change state, message anyone, or delegate. Use bash only for inspection.
P3 supplies no history-query tool. If the handoff lacks conversation evidence, name that limitation rather than invent a query.
Report the overall finding, then prioritized findings with conversation ids, commits, or path:line citations and observed counts.
Propose one change per slug with a title, size, risk, and evidence.
Memory changes use add, update, or delete operations with reasons, but you do not apply them.
Codify proposals name the memory they replace. Project proposals name the explicit repo and existing code they reuse.
Never propose a storage chapter switch. Name all checks you cannot perform.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+26
View File
@@ -0,0 +1,26 @@
{
"default": {
"model": "openai/gpt-6.1-sol",
"thinking": "medium"
},
"fast": {
"model": "openai/gpt-6.1-sol",
"thinking": "medium"
},
"review": {
"model": "anthropic/claude-opus-5-5",
"thinking": "xhigh"
},
"research": {
"model": "anthropic/claude-opus-5-5",
"thinking": "high"
},
"design": {
"model": "anthropic/claude-opus-5-5",
"thinking": "high"
},
"lead": {
"model": "anthropic/claude-opus-5-5",
"thinking": "high"
}
}
+36
View File
@@ -0,0 +1,36 @@
---
name: reviewer-architecture
description: Advisory review against the Sausage world and durable extension architecture.
family: review
tools: [read, bash]
---
Independently inspect the explicit repo and immutable full-SHA base..head range with git log and git diff.
Never substitute a range. If it is missing or ambiguous, stop and report blocked verification.
Do not read other reviewers’ reports. Report only concrete issues introduced by this range.
Stay read-only. Do not edit, commit, push, change task status, run scripts, launch children, or delegate.
The lead starts reviewers in parallel and merges their findings. The coordinator owns the final verdict.
Ask whether the range uses Sausage architecture instead of a parallel mechanism.
Read AGENTS.md and relevant extension contracts before judgment.
Check standard WASI files, HTTP, and sockets. The process interface is the only extra host import.
Host code accesses the world through the VFS. Native work belongs only to the isolated Rust builder.
Agents write CLIs and servers. Repos use explicit paths. Durable tasks and events should reuse existing extension contracts.
Name the existing piece and simpler alternative with path:line evidence. Leave correctness to its reviewer.
This review is advisory. Always report APPROVED, even when verification is blocked, and clearly state that limitation.
Architecture findings are P3 and never block.
P0 means breakage, data loss, or exposed secrets. P1 means likely functional failure.
P2 blocks only when acceptance criteria require it. P3 means nonblocking polish.
Start with APPROVED or NEEDS CHANGES. The advisory architecture reviewer always uses APPROVED.
Report Reviewed range, Findings, Verification, and Residual risks.
Each finding states priority, title, path:line, impact, evidence, and a suggested fix.
NEEDS CHANGES means blocking findings or blocked verification. Never imply checks you did not run.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+31
View File
@@ -0,0 +1,31 @@
---
name: reviewer-correctness
description: Correctness, credential safety, durability, and regression review.
family: review
tools: [read, bash]
---
Independently inspect the explicit repo and immutable full-SHA base..head range with git log and git diff.
Never substitute a range. If it is missing or ambiguous, stop and report blocked verification.
Do not read other reviewers’ reports. Report only concrete issues introduced by this range.
Stay read-only. Do not edit, commit, push, change task status, run scripts, launch children, or delegate.
The lead starts reviewers in parallel and merges their findings. The coordinator owns the final verdict.
Check functional behavior, VFS mistakes, credential injection, 403 approvals, durable replay, duplicate effects, and caller regressions.
Do not invent per-agent security walls. The host credential manager is the security boundary.
For changed APIs, tools, documents, or events, inspect test doubles and fixtures for stale contracts.
P0 means breakage, data loss, or exposed secrets. P1 means likely functional failure.
P2 blocks only when acceptance criteria require it. P3 means nonblocking polish.
Start with APPROVED or NEEDS CHANGES. The advisory architecture reviewer always uses APPROVED.
Report Reviewed range, Findings, Verification, and Residual risks.
Each finding states priority, title, path:line, impact, evidence, and a suggested fix.
NEEDS CHANGES means blocking findings or blocked verification. Never imply checks you did not run.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+31
View File
@@ -0,0 +1,31 @@
---
name: reviewer-simplicity
description: Independent simplicity and test-quality review.
family: review
tools: [read, bash]
---
Independently inspect the explicit repo and immutable full-SHA base..head range with git log and git diff.
Never substitute a range. If it is missing or ambiguous, stop and report blocked verification.
Do not read other reviewers’ reports. Report only concrete issues introduced by this range.
Stay read-only. Do not edit, commit, push, change task status, run scripts, launch children, or delegate.
The lead starts reviewers in parallel and merges their findings. The coordinator owns the final verdict.
Check that the range uses the least code that satisfies the criteria.
Flag dead code, needless abstractions, duplicated logic, speculative fallbacks, jargon, and tests that merely restate implementation.
Every behavior needs a test. Prefer fast core tests and actual regressions over unrealistic load or timing tests.
P0 means breakage, data loss, or exposed secrets. P1 means likely functional failure.
P2 blocks only when acceptance criteria require it. P3 means nonblocking polish.
Start with APPROVED or NEEDS CHANGES. The advisory architecture reviewer always uses APPROVED.
Report Reviewed range, Findings, Verification, and Residual risks.
Each finding states priority, title, path:line, impact, evidence, and a suggested fix.
NEEDS CHANGES means blocking findings or blocked verification. Never imply checks you did not run.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+29
View File
@@ -0,0 +1,29 @@
---
name: reviewer
description: Read-only review lead for an immutable commit range.
family: review
tools: [read, bash, subagent]
---
# Review lead
Review the coordinator’s explicit repo, immutable full-SHA base..head range, and acceptance criteria.
A missing or ambiguous range blocks verification. Never substitute another range.
If reviewer-* definitions exist, start every one with the same brief. Do not share reports among reviewers.
Wait for complete answers. A failed or idle child without an answer counts as missing.
Merge findings without additions or omissions. Name every reviewer who raised each issue. Show disagreements explicitly.
APPROVED requires approval from every reviewer. Otherwise report NEEDS CHANGES.
Without a team, inspect git log, git diff, and relevant unchanged logic yourself.
Remain read-only. Do not edit, commit, push, run project scripts, or execute mutation-capable checks.
Read supplied validation output instead of rerunning suites. Report only issues the range introduces.
P0 means breakage, data loss, or exposed secrets. P1 means likely functional failure.
P2 blocks only if acceptance criteria require it. P3 means nonblocking polish.
Report the reviewed range, reviewer verdicts, blocking and nonblocking findings, disagreements, observed verification, and residual risks.
Each finding needs priority, path:line, impact, evidence, and a suggested fix. Never imply checks you did not run.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+6
View File
@@ -0,0 +1,6 @@
---
name: coordinator
description: Coordinates one feature for @ultimate
family: lead
addTools: []
---
+21
View File
@@ -0,0 +1,21 @@
---
name: scout
description: Read-only reconnaissance of one code question with path:line evidence.
family: research
tools: [read, bash]
---
# Read-only reconnaissance
Answer only the assigned question. Inspect the explicit repo with read and /bin programs through bash.
Use rg, ls, git log, git show, and git diff for inspection. Never change files or state.
Do not edit, commit, run project scripts, or delegate.
Return a short conclusion, then path:line evidence and brief quotations where needed.
Name everything you cannot find or verify.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+42
View File
@@ -0,0 +1,42 @@
---
name: ultimate
description: 'The standing agent: starts branches, routes decisions, and maintains world memory.'
family: default
resident: true
tools: [read, bash, message, note, notify, automation, branch, compact, remember, memory, todo, goal, edit, write]
---
# Standing agent
You are Ultimate, the user’s standing agent in Sausage. Keep this conversation for the user and branch reports.
Read /src/sausage/AGENTS.md, agent files, skills, and relevant extension documents before answers about the platform.
Read the how-sausage-works skill when the user asks for a walkthrough.
## Branches and decisions
Write configuration and memory yourself. Send product code to a coordinator through branch.
Every branch brief names its repo path, goal, scope, acceptance criteria, and relevant memory slugs.
A small clear change needs targeted checks. A feature needs an assessment before implementation unless the user explicitly waived it.
After authorization, the coordinator plans, implements, verifies, reviews, and submits. Never declare success from an idle branch.
Route decisions through message with numbered questions. Let coordinators ask their own decisions after you state your recommendation.
Ask the user unless their instructions, memory, or active goal already settle the decision.
During a hands-off goal, record each decision and its authority with goal. Credentials and approvals remain the user’s responsibility.
Report a completed submission with its commit, checks, and divergence. Never push without authorization.
## Memory and open work
Use memory to read facts and remember to apply add, update, and delete operations.
Record durable decisions, not state that the prompt already supplies. Never store secrets in memory.
Use todo for open work, not unanswered questions. Use message for branch replies and notify for requested updates.
On a failed run, inspect the supplied failure evidence before you resume the agent or request user action.
Use compact with a self-contained handoff when context grows large.
A dream is a read-only review of evidence and memory. Approved proposals become ordinary branches. Do not switch storage chapters.
Use automation for schedules, event conditions, or supervised scripts. Verify its state with automation list.
When the user requests specialization, read integrate-sausage and adopt-sausage-workflow before proposals.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+26
View File
@@ -0,0 +1,26 @@
---
name: watcher
description: Writes and repairs one supervised automation script.
family: fast
tools: [read, write, edit, bash, automation]
---
# Automation script owner
Own the executable run file and automation id supplied in the brief. Use a WASI program or a supported shell in /bin.
Prefer an existing blocking watch command over polling. Never poll faster than every 30 seconds.
Store seen ids and persistent state in the supplied AUTOMATION_STATE directory.
Print FIRE followed by the event description when the condition holds. Other stdout is progress. Never print credentials.
A one-shot script exits after FIRE. A recurring script must not repeat seen events.
If the recurring task explicitly permits completion, print DONE with a reason and exit zero when no work remains.
Use the explicit repo path and automation file path from the brief. Report edits that remain uncommitted.
Try the script briefly before automation resume with your id.
On failure, inspect the supplied log, fix the cause, and resume. If the task cannot work, escalate with evidence.
Every repair turn ends with automation resume or escalate. Keep reports short.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+35
View File
@@ -0,0 +1,35 @@
---
name: worker
description: Implements one bounded task and reports check evidence in a focused commit.
family: fast
fast: true
isolate: worktree
denyTools: []
---
# Bounded implementer
Implement exactly the assigned task in your worktree. Read the brief, repository instructions, and nearby code first.
Reuse existing helpers. Do not expand scope, redesign the plan, delegate, or push. Report missing context as BLOCKED.
Run the brief’s checks with /bin programs. Use targeted tests during iteration and required suites before the commit.
Every behavior needs a test. Retain tests for the core path and actual bugs.
Inspect the complete diff. Remove only your own temporary code. Stage only task files.
Create one focused commit with --no-gpg-sign. Never commit failed checks.
## Report
Status: DONE | PARTIAL | BLOCKED
Summary: the outcome or blocker.
Acceptance criteria: each criterion with observed command output or path:line evidence.
Validation: exact commands and results. Name all unverified checks.
Commit: the full SHA and subject, or none.
Divergence: differences from the brief, or None.
Risks: remaining risks, or none.
DONE requires evidence for every criterion.
## World contract
The world is one WASI sandbox. All agents see the same world. Use the programs in /bin through bash.
Use an explicit repo path for repository operations. Worktrees live at /worktrees/<id>.
The host injects credentials. A 403 requests approval through the host. Never read, store, or print secrets.
Agents write CLIs and servers. Do not add host imports beyond the process interface.
+148
View File
@@ -0,0 +1,148 @@
/** Node-only test runner. The implementation imports no Node APIs. Run from the repository root. */
import assert from 'node:assert/strict'
import { readFileSync, readdirSync } from 'node:fs'
import { parseDefinition, parseOverlay, parseFrontmatter, mergeOverlay, parseFamilies, discoverAgents, definitionRuns, toolsFor } from './agents.ts'
let count = 0
function test(name: string, run: () => void) { run(); console.log(`ok ${++count} - ${name}`) }
const definition = (fields = '', body = 'Original instructions') => `---\nname: scout\ndescription: Search\n${fields}\n---\n${body}\n`
const families = parseFamilies(readFileSync('agents/models.json', 'utf8'))
const models = [...new Set(Object.values(families).map(family => family.model))]
const discover = (files: { path: string; text: string }[]) => discoverAgents(files, '/src/sausage', '/repos/app', families, models)
const base = (fields = '') => ({ path: '/src/sausage/agents/scout.md', text: definition(fields) })
const project = (text: string, filename = 'scout') => ({ path: `/repos/app/.sausage/agents/${filename}.md`, text })
test('Pi parse case: folded description and block tool list', () => {
assert.deepEqual(parseDefinition('---\nname: scout\ndescription: >\n A read-only scout\n that searches.\nmodel: openai/gpt-6.1-sol\nthinking: max\ntools:\n - read\n - bash\n---\nSearch\n', 'scout.md'), {
name: 'scout', description: 'A read-only scout that searches.', path: 'scout.md', instructions: 'Search', model: models[0], thinking: 'max', tools: ['read', 'bash'],
})
})
test('flow lists, quotes, booleans, and carriage returns', () => {
const parsed = parseDefinition(definition('tools: [read, "bash"]\nsummaries: false\nresident: true\nfast: false\nicon: NotebookPen\ntint: grey').replaceAll('\n', '\r\n'), 'test')
assert.deepEqual(parsed.tools, ['read', 'bash']); assert.equal(parsed.summaries, false)
assert.equal(parsed.resident, true); assert.equal(parsed.fast, false)
assert.equal(parseFrontmatter("---\nname: 'it''s'\n---\nBody").frontmatter.name, "it's")
const multiline = parseDefinition(definition('tools:\n [\n read,\n bash,\n ]\nfamily: research'), 'test')
assert.deepEqual(multiline.tools, ['read', 'bash']); assert.equal(multiline.family, 'research')
assert.deepEqual(parseDefinition(definition('tools: [read, "unknown,tool"]'), 'test').tools, ['read', 'unknown,tool'])
})
const errors: [string, string][] = [
['tools: [read, bash]\nthinking: nonsense', 'thinking must be one of'],
['description: " "', 'description is required'],
['name: a scout', 'name must be a nonempty identifier'],
['tools: read', 'tools must be a list of tool names'],
['tools: [read, 1]', 'tools must be a list of tool names'],
['model: " "', 'model must be a nonempty string'],
['tool: [read]', 'Unknown field: tool'],
['isolate: remote', 'isolate must be none or worktree'],
['summaries: never', 'summaries must be a boolean'],
['resident: no', 'resident must be a boolean'],
['fast: 1', 'fast must be a boolean'],
['family: Bad', 'family must be a family name'],
['icon: bad/icon', 'icon must be a lucide icon name'],
['tint: brown', 'tint must be one of'],
]
for (const [field, error] of errors) test(`reject ${field}`, () => {
const text = field.startsWith('description:') ? definition().replace('description: Search', field) : field.startsWith('name:') ? definition().replace('name: scout', field) : definition(field)
assert.throws(() => parseDefinition(text, 'test'), { message: new RegExp(error) })
})
test('reject absent fields, empty instructions, and malformed blocks', () => {
for (const text of ['Text', '---\n---\nBody', '---\n- name\n---\nBody', '---\nname: scout\n']) assert.throws(() => parseDefinition(text, 'test'), /Expected frontmatter/)
assert.throws(() => parseDefinition(definition().replace('description: Search\n', ''), 'test'), /description is required/)
assert.throws(() => parseDefinition(definition('', ''), 'test'), /Instructions are required/)
})
test('denyTools accepts unknown names and rejects invalid values for definitions and overlays', () => {
assert.deepEqual(parseDefinition(definition('denyTools: [instance, unknown-tool]'), 'test').denyTools, ['instance', 'unknown-tool'])
assert.deepEqual(parseDefinition(definition('denyTools: []'), 'test').denyTools, [])
for (const value of ['instance', '[instance, 1]', 'null', '{instance: true}']) {
assert.throws(() => parseDefinition(definition(`denyTools: ${value}`), 'test'), /denyTools must be a list/)
assert.throws(() => parseOverlay(definition(`addTools: []\ndenyTools: ${value}`), 'test'), /denyTools must be a list/)
}
})
test('ordered overlays add tools and append bodies without changing source', () => {
const found = discover([project('---\nname: scout\naddTools: [write, read]\n---\nFirst', '01'), base('tools: [read, bash]'), project('---\nname: scout\naddTools: [message]\n---\nSecond', '02')])
assert.deepEqual(found.errors, []); assert.deepEqual(found.replaced, [])
const agent = found.definitions[0]
assert.equal(agent.source, 'builtin'); assert.equal(agent.path, base().path)
assert.deepEqual(agent.tools, ['read', 'bash', 'write', 'message'])
assert.equal(agent.instructions, 'Original instructions\n\n## Project additions\n\nFirst\n\n## Project additions\n\nSecond')
assert.deepEqual(agent.overlays, [project('', '01').path, project('', '02').path])
})
test('overlay settings replace while denials accumulate through empty overlay', () => {
const found = discover([base('tools: [read]\ndenyTools: [unknown-tool]\nthinking: low'), project('---\nname: scout\naddTools: []\ndenyTools: [unknown-tool, bash]\nmodel: anthropic/claude-opus-5-5\nthinking: high\nfast: true\ndescription: Our scout\n---', '01'), project('---\nname: scout\naddTools: []\ndenyTools: []\n---', '02')])
assert.deepEqual(found.errors, [])
assert.deepEqual(found.definitions[0].denyTools, ['unknown-tool', 'bash'])
assert.equal(found.definitions[0].thinking, 'high'); assert.equal(found.definitions[0].fast, true)
assert.equal(found.definitions[0].description, 'Our scout')
assert.deepEqual(found.definitions[0].tools, ['read'])
})
test('inherited tools retain deferred additions and do not mutate parent', () => {
const parent = ['read', 'bash', 'write', 'subagent']
const child = mergeOverlay(parseDefinition(definition('denyTools: [write, subagent]'), 'test'), parseOverlay('---\nname: scout\naddTools: [message]\n---', 'overlay'))
assert.equal(child.tools, undefined); assert.deepEqual(child.addTools, ['message'])
assert.deepEqual(toolsFor(child, parent), ['read', 'bash', 'message'])
assert.deepEqual(toolsFor(parseDefinition(definition('tools: [read, write]\ndenyTools: [write]'), 'test'), parent), ['read'])
assert.deepEqual(parent, ['read', 'bash', 'write', 'subagent'])
})
test('overlay errors do not replace a valid base', () => {
const found = discover([base(), project('---\nname: scout\naddTools: [x]\ntools: [read]\n---', 'listed'), project('---\nname: scout\naddTools: []\ncolour: red\n---', 'unknown'), project('---\nname: nobody\naddTools: [x]\n---', 'orphan')])
assert.deepEqual(Object.fromEntries(found.errors.map(error => [error.name, error.error])), {
listed: 'Error: An overlay adds tools with addTools, not tools', unknown: 'Error: Unknown field: colour', orphan: 'Error: addTools needs an earlier definition named nobody',
})
assert.equal(found.definitions[0].instructions, 'Original instructions')
assert.throws(() => parseOverlay('---\nname: scout\naddTools: read\n---', 'test'), /addTools must be a list/)
})
test('replacement uses name, not filename, and respects exact roots', () => {
const found = discover([base(), project(definition('family: research', 'Replacement'), 'different'), { path: '/repos/other/.sausage/agents/scout.md', text: definition() }, { path: '/src/sausage/agents/nested/no.md', text: definition() }])
assert.deepEqual(found.errors, []); assert.equal(found.definitions.length, 1)
assert.equal(found.definitions[0].source, 'project'); assert.equal(found.definitions[0].instructions, 'Replacement')
assert.deepEqual(found.replaced, [{ name: 'scout', path: project('', 'different').path, source: 'project' }])
})
test('family fallback, thinking override, model suffix, and unknown models', () => {
const run = (fields: string) => definitionRuns(parseDefinition(definition(fields), 'test'), families, models)
assert.equal(run('family: missing').family, 'default')
assert.equal(run('family: constructor').family, 'default')
assert.equal(run('family: research').thinking, 'high')
assert.equal(run('family: research\nthinking: low').thinking, 'low')
assert.equal(run('model: openai/gpt-6.1-sol').thinking, undefined)
assert.equal(run('model: openai/gpt-6.1-sol:high').thinking, 'high')
assert.equal(run('model: openai/gpt-6.1-sol:high\nthinking: low').thinking, 'low')
for (const model of ['anthropic/unknown', 'unknown']) assert.match(discover([base(`model: ${model}`)]).errors[0].error, /model: Model .* not found/)
const found = discover([base('model: openai/gpt-6.1-sol'), project('---\nname: scout\nfamily: lead\naddTools: []\n---')])
assert.equal(found.definitions[0].model, undefined); assert.equal(found.definitions[0].runs.family, 'lead')
const suffix = discover([base('thinking: low'), project('---\nname: scout\nmodel: openai/gpt-6.1-sol:high\naddTools: []\n---')])
assert.equal(suffix.definitions[0].runs.thinking, 'high')
})
test('family world file requires valid refs and default', () => {
for (const text of ['null', '[]', '{}', '{"default":{"model":"bad"}}', '{"Default":{"model":"openai/a"}}', '{"default":{"model":"openai/a","thinking":"bad"}}']) assert.throws(() => parseFamilies(text))
assert.deepEqual(Object.keys(families), ['default', 'fast', 'review', 'research', 'design', 'lead'])
assert.equal(families.review.thinking, 'xhigh')
assert.equal(families.design.thinking, 'high')
})
test('all shipped definitions and overlays parse and discover', () => {
const files = readdirSync('agents').filter(name => name.endsWith('.md')).map(name => ({ path: `/src/sausage/agents/${name}`, text: readFileSync(`agents/${name}`, 'utf8') }))
assert.equal(files.length, 12)
const available = ['read', 'bash', 'edit', 'write', 'message', 'note', 'notify', 'automation', 'branch', 'compact', 'remember', 'memory', 'todo', 'goal', 'subagent', 'plan', 'workspace']
for (const file of files) {
const agent = 'addTools' in parseFrontmatter(file.text).frontmatter ? parseOverlay(file.text, file.path) : parseDefinition(file.text, file.path)
assert.ok(!/\b(pnpm|vitest|clone|instance|restart|web_search|mcp|setup|codemode|debug)\b|~\/\.ultimate/.test(agent.instructions), file.path)
const fields = 'settings' in agent ? agent.settings : agent
for (const tool of [...(fields.tools ?? []), ...(fields.denyTools ?? []), ...('addTools' in agent ? agent.addTools ?? [] : [])]) assert.ok(available.includes(tool), `${file.path}: ${tool}`)
}
const found = discover(files)
assert.deepEqual(found.errors, []); assert.equal(found.definitions.length, 11)
assert.equal(found.definitions.find(agent => agent.name === 'coordinator')!.runs.family, 'lead')
for (const agent of found.definitions.filter(agent => agent.name.startsWith('reviewer'))) assert.equal(agent.runs.thinking, 'xhigh')
})
test('all applicable skills parse and avoid unavailable tools', () => {
let skills = 0
for (const name of readdirSync('skills')) {
if (name === 'README.md') continue
const parsed = parseFrontmatter(readFileSync(`skills/${name}/SKILL.md`, 'utf8'))
assert.equal(parsed.frontmatter.name, name)
assert.equal(typeof parsed.frontmatter.description, 'string'); assert.ok(parsed.body)
assert.ok(!/\b(pnpm|vitest|clone|instance|restart|web_search|mcp|setup|codemode|debug)\b|~\/\.ultimate/.test(parsed.body), name)
skills++
}
assert.equal(skills, 4)
})
console.log(`${count} checks passed`)
+171
View File
@@ -0,0 +1,171 @@
/** Pure parsing and ordered overlays for world agent files. The caller supplies all file contents and model configuration. */
export const thinkingLevels = ['off', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max'] as const
export type Thinking = typeof thinkingLevels[number]
export type Family = { model: string; thinking?: Thinking }
export type Families = Record<string, Family>
export interface Definition {
name: string
description: string
path: string
instructions: string
model?: string
thinking?: Thinking
family?: string
tools?: string[]
addTools?: string[]
denyTools?: string[]
isolate?: 'none' | 'worktree'
resident?: boolean
fast?: boolean
summaries?: boolean
icon?: string
tint?: string
overlays?: string[]
}
export type File = { path: string; text: string }
const fields = ['name', 'description', 'model', 'thinking', 'family', 'tools', 'denyTools', 'isolate', 'resident', 'fast', 'summaries', 'icon', 'tint']
const familyName = (name: string) => /^[a-z][a-z0-9-]{0,23}$/.test(name)
const tools = (value: unknown): value is string[] => Array.isArray(value) && value.every(v => typeof v === 'string')
/** Scalars and flow lists, plus the block lists and folded description used in Ultimate's parser tests. Not general YAML. */
function scalar(input: string): unknown {
const text = input.trim()
if (text.startsWith('[')) {
if (!text.endsWith(']')) throw new Error('Unclosed flow list')
const items = text.slice(1, -1).match(/\s*(?:"(?:\\.|[^"\\])*"|'(?:''|[^'])*'|[^,]+)/g) ?? []
return items.filter(item => item.trim()).map(item => scalar(item))
}
if (text.startsWith('"')) return JSON.parse(text)
if (text.startsWith("'")) {
if (!text.endsWith("'")) throw new Error('Unclosed scalar')
return text.slice(1, -1).replaceAll("''", "'")
}
if (text === 'true') return true
if (text === 'false') return false
if (text === 'null' || text === '~' || text === '') return null
if (/^-?\d+(\.\d+)?$/.test(text)) return Number(text)
if (text.startsWith('{')) return {} // Mapping values cannot satisfy any supported field.
return text.replace(/\s+#.*$/, '')
}
export function parseFrontmatter(text: string): { frontmatter: Record<string, unknown>; body: string } {
const lines = text.replaceAll('\r\n', '\n').split('\n')
const end = lines.indexOf('---', 1)
if (lines[0] !== '---' || end < 0) throw new Error('Expected frontmatter fields between --- lines')
const frontmatter: Record<string, unknown> = {}
for (let i = 1; i < end; i++) {
if (!lines[i].trim() || lines[i].trimStart().startsWith('#')) continue
const match = /^([A-Za-z][A-Za-z0-9]*):\s*(.*)$/.exec(lines[i])
if (!match) throw new Error('Expected frontmatter fields between --- lines')
const [, key, raw] = match
let value = raw
if (value.trim().startsWith('[') && !value.trim().endsWith(']')) {
while (i + 1 < end && !value.trim().endsWith(']')) value += ' ' + lines[++i].trim()
}
if (raw === '>' || raw === '|' || raw === '') {
const block: string[] = []
while (i + 1 < end && /^\s+\S/.test(lines[i + 1])) block.push(lines[++i].trim())
if (raw === '>') frontmatter[key] = block.join(' ')
else if (raw === '|') frontmatter[key] = block.join('\n')
else frontmatter[key] = block[0]?.startsWith('[') ? scalar(block.join(' ')) : block.length ? block.map(line => scalar(line.replace(/^-\s+/, ''))) : null
} else frontmatter[key] = scalar(value)
}
if (!Object.keys(frontmatter).length) throw new Error('Expected frontmatter fields between --- lines')
return { frontmatter, body: lines.slice(end + 1).join('\n').trim() }
}
function settings(data: Record<string, unknown>, overlay: boolean) {
const unknown = Object.keys(data).find(key => !fields.includes(key) && !(overlay && key === 'addTools'))
if (unknown) throw new Error(`Unknown field: ${unknown}`)
if (typeof data.name !== 'string' || !/^[a-zA-Z0-9_-]+$/.test(data.name)) throw new Error('name must be a nonempty identifier')
for (const key of ['description', 'model']) {
if (data[key] !== undefined && (typeof data[key] !== 'string' || !data[key].trim())) throw new Error(key === 'description' ? 'description is required' : 'model must be a nonempty string')
}
if (data.family !== undefined && (typeof data.family !== 'string' || !familyName(data.family))) throw new Error('family must be a family name: lowercase letters, digits and -')
if (data.thinking !== undefined && !thinkingLevels.includes(data.thinking as Thinking)) throw new Error(`thinking must be one of ${thinkingLevels.join(', ')}`)
for (const key of ['tools', 'denyTools', 'addTools']) {
if (data[key] !== undefined && !tools(data[key])) throw new Error(`${key} must be a list of tool names`)
}
if (data.isolate !== undefined && data.isolate !== 'none' && data.isolate !== 'worktree') throw new Error('isolate must be none or worktree')
for (const key of ['resident', 'fast', 'summaries']) {
if (data[key] !== undefined && typeof data[key] !== 'boolean') throw new Error(`${key} must be a boolean`)
}
if (data.icon !== undefined && (typeof data.icon !== 'string' || !/^[A-Za-z][A-Za-z0-9]*(-[a-z0-9]+)*$/.test(data.icon))) throw new Error('icon must be a lucide icon name, like Gem or notebook-pen')
if (data.tint !== undefined && !['violet', 'blue', 'green', 'amber', 'red', 'pink', 'teal', 'grey'].includes(data.tint as string)) throw new Error('tint must be one of violet, blue, green, amber, red, pink, teal, grey')
return { ...data, ...(typeof data.description === 'string' ? { description: data.description.trim() } : {}) }
}
export function parseDefinition(text: string, path: string): Definition {
const { frontmatter, body } = parseFrontmatter(text)
const parsed = settings(frontmatter, false)
if (!parsed.description) throw new Error('description is required')
if (!body) throw new Error('Instructions are required')
return { ...parsed, path, instructions: body } as unknown as Definition
}
export function parseOverlay(text: string, path: string) {
const { frontmatter, body } = parseFrontmatter(text)
if ('tools' in frontmatter) throw new Error('An overlay adds tools with addTools, not tools')
const parsed = settings(frontmatter, true)
if (!tools(parsed.addTools)) throw new Error('addTools must be a list of tool names')
const { name, addTools, ...rest } = parsed
return { name: name as string, addTools, settings: rest as Partial<Definition>, path, instructions: body }
}
export function mergeOverlay(base: Definition, overlay: ReturnType<typeof parseOverlay>): Definition {
const { model: _pin, ...unpinned } = base
const suffix = overlay.settings.model?.match(/:(off|minimal|low|medium|high|xhigh|max)$/)
const resolved = suffix ? { ...overlay.settings, model: overlay.settings.model!.slice(0, -suffix[0].length), thinking: overlay.settings.thinking ?? suffix[1] as Thinking } : overlay.settings
return {
...(overlay.settings.family && !overlay.settings.model ? unpinned : base),
...resolved,
...(overlay.settings.denyTools ? { denyTools: [...new Set([...(base.denyTools ?? []), ...overlay.settings.denyTools])] } : {}),
...(base.tools ? { tools: [...new Set([...base.tools, ...overlay.addTools])] } : overlay.addTools.length ? { addTools: [...new Set([...(base.addTools ?? []), ...overlay.addTools])] } : {}),
instructions: overlay.instructions ? `${base.instructions}\n\n## Project additions\n\n${overlay.instructions}` : base.instructions,
overlays: [...(base.overlays ?? []), overlay.path],
}
}
export function parseFamilies(text: string): Families {
const value = JSON.parse(text)
if (!value || typeof value !== 'object' || Array.isArray(value)) throw new Error('families must be an object')
for (const [name, spec] of Object.entries(value) as [string, Family][]) {
if (!familyName(name) || !spec || typeof spec !== 'object' || !modelRef(spec.model)) throw new Error(`Invalid family: ${name}`)
if (spec.thinking !== undefined && !thinkingLevels.includes(spec.thinking)) throw new Error(`Invalid thinking for family: ${name}`)
}
if (!value.default) throw new Error('default family is required')
return value
}
function modelRef(model: unknown): model is string { return typeof model === 'string' && /^[^/\s]+\/\S+$/.test(model) }
export function definitionRuns(definition: Definition, families: Families, models: readonly string[]) {
const family = Object.hasOwn(families, definition.family ?? 'default') ? definition.family ?? 'default' : 'default'
let model = definition.model ?? families[family].model
let thinking = definition.thinking ?? (definition.model ? undefined : families[family].thinking)
const suffix = /:(off|minimal|low|medium|high|xhigh|max)$/.exec(model)
if (suffix) { model = model.slice(0, -suffix[0].length); thinking = definition.thinking ?? suffix[1] as Thinking }
if (!modelRef(model) || !models.includes(model)) throw new Error(`model: Model "${model}" not found`)
const slash = model.indexOf('/')
return { model: { provider: model.slice(0, slash), modelId: model.slice(slash + 1) }, thinking, ...(definition.model ? { via: 'pin' as const } : { via: 'family' as const, family }) }
}
/** Sorted built-ins precede sorted project files. Invalid files produce errors without replacing an earlier definition. */
export function discoverAgents(files: readonly File[], builtinRoot: string, repo: string, families: Families, models: readonly string[]) {
const definitions = new Map<string, Definition & { source: 'builtin' | 'project' }>()
const errors: { name: string; path: string; source: string; error: string }[] = []
const replaced: { name: string; path: string; source: string }[] = []
for (const [source, root] of [['builtin', builtinRoot + '/agents/'], ['project', repo + '/.sausage/agents/']] as const) {
for (const file of files.filter(file => file.path.startsWith(root) && !file.path.slice(root.length).includes('/') && file.path.endsWith('.md')).toSorted((a, b) => a.path.localeCompare(b.path))) {
try {
let definition: Definition
if ('addTools' in parseFrontmatter(file.text).frontmatter) {
const overlay = parseOverlay(file.text, file.path)
const base = definitions.get(overlay.name)
if (!base) throw new Error(`addTools needs an earlier definition named ${overlay.name}`)
definition = mergeOverlay(base, overlay)
} else definition = parseDefinition(file.text, file.path)
definitionRuns(definition, families, models)
const previous = definitions.get(definition.name)
if (source === 'project' && previous?.source === 'builtin' && !definition.overlays) replaced.push({ name: definition.name, path: file.path, source })
definitions.set(definition.name, { ...definition, source: definition.overlays ? previous!.source : source })
} catch (error) { errors.push({ name: file.path.split('/').at(-1)!.slice(0, -3), path: file.path, source, error: String(error) }) }
}
}
return { definitions: [...definitions.values()].map(definition => ({ ...definition, runs: definitionRuns(definition, families, models) })), errors, replaced }
}
/** Tool denials also apply to inherited tools. Inputs stay unchanged. */
export function toolsFor(definition: Definition, parent: readonly string[]): string[] {
return [...new Set([...(definition.tools ?? parent), ...(definition.addTools ?? [])])].filter(tool => !definition.denyTools?.includes(tool))
}
+31 -3
View File
@@ -1,7 +1,7 @@
# Host JavaScript
The console uses pi-durable in V8. Rust owns the SQLite connection.
The console runs V8 on a dedicated current-thread Tokio runtime.
The headless harness uses pi-durable in V8. Rust owns the SQLite connection.
The harness runs V8 on a dedicated current-thread Tokio runtime.
This satisfies Deno's runtime requirement without a change to the CLI runtime.
The database lives at `<dir>/host/harness.sqlite`, outside the world.
WAL preserves atomic transactions after SIGKILL. `synchronous=FULL` flushes each commit before SQLite reports success.
@@ -28,7 +28,13 @@ npm run --prefix host/js bundle
CARGO_TARGET_DIR=target/harness nix develop -c cargo test -p sausage --test harness
```
The lock file pins npm dependencies. The checked-in bundles let Cargo build without npm or network access for JavaScript.
The lock file pins npm dependencies. The bundle command requires Nushell, GNU patch, and npm.
It applies `patches/pi-durable-1.5.0.patch` before esbuild runs.
This is Ultimate's patch at `7ec0b4f`, without the Node execution environment hunks (D65).
The patch adds transactional `beforeInput`, input markers, scheduler hold/release, and the failed-run boundary fix.
The step verifies an existing application before another bundle build. A mismatched or partially applied patch fails the command.
`npm ci` restores the unpatched dependency. The next bundle command applies the patch again.
The checked-in bundles let Cargo build without npm or network access for JavaScript.
Cargo still needs its normal Rust dependencies and the V8 archive.
For Nix, use a separate npm derivation with `importNpmLock` and the checked-in lock file.
@@ -52,3 +58,25 @@ A decided task admits one plain-language input and completes in one SQLite commi
The input starts a new turn when the conversation is idle. Neither the input nor the task contains a secret.
The task and its input survive SIGKILL. The input uses a stable request id to prevent duplicate admission.
Hosts with a shared credentials handle can use `run_module_with_credentials`.
## Channel input and shutdown
`run_module_with_input` accepts a receiver from `harness::input::channel()` and runs without console input or output.
Its caller supplies the world, process interface, shared credentials handle, and bundled module.
`Input::Submit` contains a conversation id, JSON content, and a request id.
The harness admits that input into the specified conversation. The request id preserves submission deduplication.
Headless admission does not wait for a model answer.
The harness groups queued follow-ups into one new run.
`Input::Stop` waits for admitted foreground work, then closes the harness cleanly.
The JavaScript `serve` function implements this channel protocol.
The main module exports `storage`, including its `scanSubmissions` method.
`run_console` connects stdin and stop signals to the same channel.
The console submits to the durable root conversation (#1).
EOF closes only the stdin client. `sausage run` stays active until SIGINT or SIGTERM.
The signal client sends `Input::Stop`. The harness closes its watches, credential coordinator, and SQLite connection.
The host then stops its process services and HTTP listener.
Console output still streams assistant messages, without replay of the historical transcript.
`harness_foundation.rs` runs the three Ultimate input tests and the scheduler phase-boundary test through bundled V8 modules.
It also tests the channel with closed stdin, CLI EOF and SIGTERM, submission scans, and the patched bundle hooks.
+15
View File
@@ -0,0 +1,15 @@
# Apply the versioned dist patch, or verify that all its hunks already exist.
let args = [-p1 -d node_modules/@earendil-works/pi-durable -i ../../../patches/pi-durable-1.5.0.patch --batch --fuzz=0]
let forward = (^patch ...$args --forward --dry-run | complete)
if $forward.exit_code == 0 {
^patch ...$args --forward
if $env.LAST_EXIT_CODE != 0 { error make {msg: "pi-durable patch failed"} }
} else {
let reverse = (^patch ...$args --force --reverse --dry-run | complete)
if $reverse.exit_code != 0 {
print $forward.stdout
print --stderr $forward.stderr
error make {msg: "pi-durable 1.5.0 patch does not match the installed package"}
}
print "pi-durable patch already applied"
}
+162 -94
View File
@@ -30494,8 +30494,8 @@ var CachedStorage = class {
#submissions = new Lru(256);
/** Per record with a read in flight: the reads, and how many published commits wrote it since the first began. */
#reads = /* @__PURE__ */ new Map();
constructor(storage) {
this.#storage = storage;
constructor(storage2) {
this.#storage = storage2;
}
/** The backend's own scope for a publication (`Storage.publishing`), when it has one. */
get publishing() {
@@ -30686,10 +30686,10 @@ var CachedStorage = class {
return this.#reads.size;
}
};
function scopeOf(storage) {
if (storage.publishing === void 0)
function scopeOf(storage2) {
if (storage2.publishing === void 0)
return void 0;
return (publish2, context) => storage.publishing(publish2, context);
return (publish2, context) => storage2.publishing(publish2, context);
}
var Lru = class {
#map = /* @__PURE__ */ new Map();
@@ -30877,8 +30877,8 @@ function tableKeys(writes, keys) {
var GatedStorage = class {
#storage;
#touch;
constructor(storage, touch) {
this.#storage = storage;
constructor(storage2, touch) {
this.#storage = storage2;
this.#touch = touch;
}
commit(batches, receive, context) {
@@ -30954,24 +30954,24 @@ var ROOT_CONVERSATION_ID = 1;
// node_modules/@earendil-works/pi-durable/dist/session/forks.js
var SCAN_PAGE_SIZE = 256;
async function prepareForkDocumentCopies(storage, parentConversationId, at, childConversationId, context) {
const entry2 = await storage.entry(parentConversationId, at, context);
async function prepareForkDocumentCopies(storage2, parentConversationId, at, childConversationId, context) {
const entry2 = await storage2.entry(parentConversationId, at, context);
if (entry2 === void 0)
throw new Error(`Entry ${at} is not visible from conversation ${parentConversationId}`);
const copies = [];
const copiedAddresses = /* @__PURE__ */ new Set();
await collectCopies(storage, { kind: "conversation", conversationId: entry2.entry.conversationId }, entry2.commitSeq, "asOf", childConversationId, copies, copiedAddresses, context);
await collectCopies(storage, { kind: "conversation", conversationId: parentConversationId }, "current", "current", childConversationId, copies, copiedAddresses, context);
await collectCopies(storage2, { kind: "conversation", conversationId: entry2.entry.conversationId }, entry2.commitSeq, "asOf", childConversationId, copies, copiedAddresses, context);
await collectCopies(storage2, { kind: "conversation", conversationId: parentConversationId }, "current", "current", childConversationId, copies, copiedAddresses, context);
return copies;
}
async function collectCopies(storage, scope, at, policy, childConversationId, copies, copiedAddresses, context) {
async function collectCopies(storage2, scope, at, policy, childConversationId, copies, copiedAddresses, context) {
let cursor;
do {
const page2 = await storage.scanDocuments({ scope, at }, SCAN_PAGE_SIZE, cursor, context);
const page2 = await storage2.scanDocuments({ scope, at }, SCAN_PAGE_SIZE, cursor, context);
for (const source of page2.items) {
if (source.scope.kind !== "conversation" || source.fork !== policy)
continue;
const id = await storage.mintId();
const id = await storage2.mintId();
const identity = {
id,
kind: source.kind,
@@ -31140,6 +31140,9 @@ var Transaction = class {
this.#assertOpen();
return record;
}
beforeInput(conversationId, input) {
return this.#read("beforeInput", () => this.#host.beforeInput(this, conversationId, input, this.#context));
}
appendEntry(first, second, third) {
if (typeof first === "number")
return this.#appendEntry(first, second);
@@ -31612,7 +31615,7 @@ var Transaction = class {
return { observed: publications, stored: stored ?? publications };
}
async #assemble() {
const storage = this.#host.storage;
const storage2 = this.#host.storage;
const plans = this.#plans;
for (const document of this.#documents) {
const plan = planDocument(document);
@@ -31667,7 +31670,7 @@ var Transaction = class {
}
let cursor;
do {
const page2 = await storage.scanDocuments({ scope: { kind: "task", taskId }, at: "current" }, INTERNAL_SCAN_PAGE_SIZE, cursor, this.#context);
const page2 = await storage2.scanDocuments({ scope: { kind: "task", taskId }, at: "current" }, INTERNAL_SCAN_PAGE_SIZE, cursor, this.#context);
for (const record of page2.items)
retire(record);
cursor = page2.next;
@@ -31691,7 +31694,7 @@ var Transaction = class {
plan.conversationId = task.publicationConversationId;
}
for (const { id, change } of this.#submissionChanges) {
const current = this.#submissions.get(id) ?? await storage.submission(id, this.#context);
const current = this.#submissions.get(id) ?? await storage2.submission(id, this.#context);
if (current === void 0)
throw new Error(`Submission ${id} does not exist`);
const next = applySubmissionChange(current, change);
@@ -32262,12 +32265,12 @@ var SessionImpl = class {
get failed() {
return this.#failed.promise;
}
constructor(storage, now = Date.now, documentRetentionMs = () => void 0, setTimer, readOnly = false, persistWithinMs = () => void 0) {
constructor(storage2, now = Date.now, documentRetentionMs = () => void 0, setTimer, readOnly = false, persistWithinMs = () => void 0) {
this.#failed.promise.catch(() => {
});
this.#readOnly = readOnly;
this.#cached = readOnly ? void 0 : new CachedStorage(storage);
this.#storage = new GuardedStorage(this.#cached ?? storage, this);
this.#cached = readOnly ? void 0 : new CachedStorage(storage2);
this.#storage = new GuardedStorage(this.#cached ?? storage2, this);
this.#gated = new GatedStorage(this.#storage, (key) => this.touch(key));
this.#writer = new Writer({
commit: (batches, receive, context) => this.#storage.commit(batches, receive, context),
@@ -32322,6 +32325,7 @@ var SessionImpl = class {
if (this.#documents.get(id)?.record.id === recordId)
this.#dropDocument(id);
},
beforeInput: (tx, id, input, context) => this.beforeInput(tx, id, input, context),
conversationCreated: (tx, record) => this.conversationCreated(tx, record),
taskDocuments: (taskId) => this.#taskDocuments.get(taskId)?.values(),
latestEntryId: (conversationId, context) => this.latestEntryId(conversationId, context),
@@ -32646,6 +32650,9 @@ var SessionImpl = class {
*/
report(_error, _info = LISTENER, _context) {
}
async beforeInput(_tx, _id, _input, _context) {
return [];
}
/**
* Runs inside every transaction that creates or forks a conversation, after the conversation record is staged. A
* plain Session stages nothing; a Harness stages its built-in documents.
@@ -33561,8 +33568,8 @@ var GuardedStorage = class {
/** Backend calls underway, also those off the Session line, which close waits for before it closes the backend. */
#underway = /* @__PURE__ */ new Set();
#closing = false;
constructor(storage, latch) {
this.#storage = storage;
constructor(storage2, latch) {
this.#storage = storage2;
this.#latch = latch;
}
/** The backend's own scope for a publication (`Storage.publishing`), when it has one; not a call to guard. */
@@ -34072,21 +34079,21 @@ function defineTask(definition) {
var SCAN_PAGE_SIZE2 = 256;
var EXCLUDED_STOP_REASONS = /* @__PURE__ */ new Set(["aborted", "error", "deferred"]);
var MISSING_RESULT_TEXT = "Tool result unavailable: history ends before this call completed.";
async function captureContextBounds(session, storage, conversationId, context, at, visible = false) {
async function captureContextBounds(session, storage2, conversationId, context, at, visible = false) {
let tail;
if (at === void 0) {
tail = await session.latestEntryId(conversationId, context);
if (tail === void 0)
return void 0;
} else {
if (!visible && await storage.entry(conversationId, at, context) === void 0) {
if (!visible && await storage2.entry(conversationId, at, context) === void 0) {
throw new Error(`Entry ${at} is not visible from conversation ${conversationId}`);
}
tail = at;
}
return { head: await session.latestHeadMarker(conversationId, tail, context), tail };
}
async function readContextFrom(session, storage, conversationId, context, at, previous) {
async function readContextFrom(session, storage2, conversationId, context, at, previous) {
const visible = at !== void 0 && previous !== void 0 && rangeEntry(previous, at) !== void 0;
const bounds = await session.readOnLine((line) => captureContextBounds(session, line, conversationId, context, at, visible));
if (bounds === void 0)
@@ -34095,7 +34102,7 @@ async function readContextFrom(session, storage, conversationId, context, at, pr
const current = at === void 0;
let range;
if (previous === void 0 || previous.bounds.head?.id !== bounds.head?.id) {
const entries2 = freezeJson(await scanRange(storage, conversationId, rangeQuery(bounds), context));
const entries2 = freezeJson(await scanRange(storage2, conversationId, rangeQuery(bounds), context));
range = deriveRange(bounds, entries2, current);
} else if (bounds.tail === previous.bounds.tail) {
range = previous;
@@ -34103,7 +34110,7 @@ async function readContextFrom(session, storage, conversationId, context, at, pr
range = deriveRange(bounds, previous.entries.filter((entry2) => entry2.id <= bounds.tail), false);
} else {
const minEntryId = idFromNumber(previous.bounds.tail + 1);
const added = await scanRange(storage, conversationId, { minEntryId, maxEntryId: bounds.tail }, context);
const added = await scanRange(storage2, conversationId, { minEntryId, maxEntryId: bounds.tail }, context);
range = extendRange(previous, bounds, freezeJson(added), current);
}
if (range.current && session.knownLatestEntryId(conversationId) !== range.bounds.tail) {
@@ -34120,14 +34127,14 @@ async function readContextFrom(session, storage, conversationId, context, at, pr
range
};
}
async function readContext(session, storage, conversationId, context, at) {
async function readContext(session, storage2, conversationId, context, at) {
const bounds = await session.readOnLine((line) => captureContextBounds(session, line, conversationId, context, at));
return deriveContext(storage, conversationId, bounds, context);
return deriveContext(storage2, conversationId, bounds, context);
}
async function deriveContext(storage, conversationId, bounds, context) {
async function deriveContext(storage2, conversationId, bounds, context) {
if (bounds === void 0)
return emptyView();
return deriveRange(bounds, await scanRange(storage, conversationId, rangeQuery(bounds), context), false).view;
return deriveRange(bounds, await scanRange(storage2, conversationId, rangeQuery(bounds), context), false).view;
}
function emptyView() {
return { head: void 0, entries: [], contributions: [], messages: [] };
@@ -34245,11 +34252,11 @@ function rangeQuery(bounds) {
const head = bounds.head;
return head === void 0 ? { maxEntryId: bounds.tail } : { minEntryId: head.head, maxEntryId: bounds.tail };
}
async function scanRange(storage, conversationId, range, context) {
async function scanRange(storage2, conversationId, range, context) {
const entries2 = [];
let cursor;
do {
const page2 = await storage.scanEntries({ conversationId, ...range }, SCAN_PAGE_SIZE2, cursor, context);
const page2 = await storage2.scanEntries({ conversationId, ...range }, SCAN_PAGE_SIZE2, cursor, context);
entries2.push(...page2.items);
cursor = page2.next;
} while (cursor !== void 0);
@@ -34403,10 +34410,14 @@ var InboxDoc = defineDoc({
initial: () => ({ items: [] }),
checkpointWhen: (value2) => value2.items.length === 0
});
async function prepareBoundary(tx, conversationId, modes) {
async function prepareBoundary(tx, conversationId, modes, at, now, incoming) {
const head = (await tx.latestHeadMarker(conversationId))?.head;
const inbox = await tx.doc(InboxDoc, conversationId);
const first = inbox.items.find((item) => item.mode === "steer" || item.mode === "followUp");
const input = first === void 0 ? incoming : inputDraft(first.content, first.entry, now);
const entries2 = input !== void 0 && (at === "final" || hasQueuedReset(inbox.items)) ? await tx.beforeInput(conversationId, input) : [];
return {
entries: entries2,
conversationId,
inbox,
steeringMode: modes.steeringMode,
@@ -34442,6 +34453,9 @@ async function applyBoundary(tx, boundary, at, now) {
}
const placed = [];
for (const index of users) {
if (placed.length === 0)
for (const draft of boundary.entries)
await tx.appendEntry(conversationId, draft);
const item = items[index];
const entry2 = await appendInput(tx, conversationId, item.content, item.entry, now);
tx.placeSubmission(item.id, entry2.id);
@@ -34454,10 +34468,11 @@ async function applyBoundary(tx, boundary, at, now) {
return { users: placed, first, reset };
}
function appendInput(tx, conversationId, content, entry2, now) {
return tx.appendEntry(conversationId, inputDraft(content, entry2, now));
}
function inputDraft(content, entry2, now) {
const message = { role: "user", content, timestamp: now };
if (entry2 === void 0)
return tx.appendEntry(UserEntry, conversationId, { model: [message] });
return tx.appendEntry(conversationId, { ...pickInputEntry(entry2), model: [message] });
return { ...entry2 === void 0 ? { kind: UserEntry.kind } : pickInputEntry(entry2), input: true, model: [message] };
}
function pickInputEntry({ kind, data }) {
const picked = data === void 0 ? { kind } : { kind, data };
@@ -35177,6 +35192,7 @@ var TaskScheduler = class {
#unsubscribeRegistry = () => {
};
#enabled = false;
#held = false;
#readOnly;
#closing = false;
#dirty = false;
@@ -35205,6 +35221,14 @@ var TaskScheduler = class {
this.#context = options.context;
this.#readOnly = options.readOnly === true;
}
/** Start no run invocation until `release()`; running invocations and abort handlers go on. Overrides `resume()`. */
hold() {
this.#held = true;
}
release() {
this.#held = false;
this.#kick();
}
/**
* Load live tasks and change surviving `running` tasks back to `pending`, counting the restart. Dispatches nothing;
* a task cut off `settings.restartLimit` times ends when it is next reserved. Read-only, it only loads them: a
@@ -35394,13 +35418,13 @@ var TaskScheduler = class {
return marked.result;
}
async waitForTask(id, context) {
const found = await this.#session.readOnLine(async (storage) => {
const found = await this.#session.readOnLine(async (storage2) => {
if (this.#closing)
throw closedError(this.#session);
if (await this.#region(() => this.#liveRecord(id) !== void 0)) {
return { promise: this.#taskWaiters.add(id, context) };
}
const record = await storage.task(id, context);
const record = await storage2.task(id, context);
if (record === void 0)
throw new Error(`Task ${id} does not exist`);
return { promise: Promise.resolve(record) };
@@ -36659,6 +36683,8 @@ var TaskScheduler = class {
for (const record of plan.ready) {
const id = record.id;
const mode = record.abortRequested ? "abort" : "run";
if (mode === "run" && this.#held)
continue;
const cutOff = record.restarts !== void 0 && record.restarts >= this.#restartLimit();
if (cutOff && (mode === "abort" || !this.#limitsRestarts(record.kind))) {
const message = mode === "abort" ? `Abort handler of task ${id} (${record.kind}) was cut off ${record.restarts} times` : `Task ${id} (${record.kind}) was cut off ${record.restarts} times without progress`;
@@ -36844,7 +36870,7 @@ var TaskScheduler = class {
const tasks = [];
for (const record of this.#live.values())
tasks.push({ record, state: this.#inspectTask(record, snapshot) });
const scheduling = this.#closing ? "closing" : this.#enabled ? "running" : "paused";
const scheduling = this.#closing ? "closing" : this.#enabled && !this.#held ? "running" : "paused";
return { scheduling, tasks };
}
#inspectTask(record, snapshot) {
@@ -36999,6 +37025,10 @@ var TaskScheduler = class {
* invocation continues.
*/
#progress(tx, current, state2, extension, context) {
if (this.#held) {
tx.setTask(withState(current, { status: "pending", checkpoint: current.state.checkpoint }));
return false;
}
state2.snapshot = this.#registry.snapshot();
const next = state2.snapshot.task(current.kind);
if (next === state2.task)
@@ -37366,7 +37396,7 @@ var TaskScheduler = class {
watchDoc: ((...args) => this.#watchDoc(invocation, args)),
snapshot: ((...args) => this.#read(invocation, () => sessionMethod(this.#session, "snapshot")(...args))),
snapshotAsOf: ((...args) => this.#read(invocation, () => sessionMethod(this.#session, "snapshotAsOf")(...args))),
getTask: ((id, context) => this.#read(invocation, () => this.#session.readOnLine((storage) => storage.task(id, context)))),
getTask: ((id, context) => this.#read(invocation, () => this.#session.readOnLine((storage2) => storage2.task(id, context)))),
waitForTask: ((id, context) => this.#read(invocation, () => this.waitForTask(id, withAbortSignal(invocation.controller.signal, context)))),
ownedTasks: (context) => this.#read(invocation, () => this.#session.readOnLine(async () => {
if (invocation.ended)
@@ -37386,7 +37416,7 @@ var TaskScheduler = class {
})),
abortOwned: (id, context) => this.#read(invocation, async () => {
const bound = withAbortSignal(invocation.controller.signal, context);
const record = await this.#session.readOnLine((storage) => storage.task(id, bound));
const record = await this.#session.readOnLine((storage2) => storage2.task(id, bound));
if (record?.owner !== invocation.taskId) {
throw new Error(`Task ${id} is not owned by task ${invocation.taskId}`);
}
@@ -37395,10 +37425,10 @@ var TaskScheduler = class {
await this.abort(id, bound, true);
await this.waitForTask(id, bound);
}),
outcomes: ((ids, context) => this.#read(invocation, () => this.#session.readOnLine(async (storage) => {
outcomes: ((ids, context) => this.#read(invocation, () => this.#session.readOnLine(async (storage2) => {
const outcomes = [];
for (const id of ids) {
const state2 = (await storage.task(id, context))?.state;
const state2 = (await storage2.task(id, context))?.state;
if (state2?.status !== "terminal")
throw new Error(`Task ${id} is not terminal`);
outcomes.push(state2.outcome);
@@ -37416,11 +37446,11 @@ var TaskScheduler = class {
const [token, id, context] = args.length === 2 ? [void 0, args[0], args[1]] : [args[0], args[1], args[2]];
return this.#read(invocation, async () => {
const kept = this.#contexts.get(invocation.conversationId);
const entry2 = (kept === void 0 ? void 0 : rangeEntry(kept.range, id)) ?? (await this.#session.readOnLine((storage) => storage.entry(invocation.conversationId, id, context)))?.entry;
const entry2 = (kept === void 0 ? void 0 : rangeEntry(kept.range, id)) ?? (await this.#session.readOnLine((storage2) => storage2.entry(invocation.conversationId, id, context)))?.entry;
return token === void 0 || entry2?.kind === token.kind ? entry2 : void 0;
});
}),
entries: (query, limit2, cursor, context) => this.#read(invocation, () => this.#session.readOnLine((storage) => storage.scanEntries(conversationEntries(invocation.conversationId, query), limit2, cursor, context))),
entries: (query, limit2, cursor, context) => this.#read(invocation, () => this.#session.readOnLine((storage2) => storage2.scanEntries(conversationEntries(invocation.conversationId, query), limit2, cursor, context))),
context: (conversationId, context, options) => this.#read(invocation, async () => {
const read2 = await readContextFrom(this.#session, this.#storage, conversationId, context, options?.at, this.#contexts.get(conversationId)?.range);
const { view: view2 } = read2;
@@ -39732,7 +39762,7 @@ async function prepareFailure(tx, runtime) {
const policy = settings.afterFailedRun;
if (policy !== "continue")
return { policy };
return { policy, boundary: await prepareBoundary(tx, runtime.conversationId, settings) };
return { policy, boundary: await prepareBoundary(tx, runtime.conversationId, settings, "final", runtime.now()) };
}
async function endFailedRun(tx, runtime, live, failure, end) {
const conversationId = runtime.conversationId;
@@ -39900,7 +39930,7 @@ async function answer(runtime, message, context) {
const afterRun = await asksAfterRun(runtime, context);
const conversationId = runtime.conversationId;
await runtime.commit(async (tx) => {
const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
const boundary = await prepareBoundary(tx, conversationId, runtime.settings, "final", runtime.now());
const live = await tx.doc(LiveDoc, conversationId);
const entry2 = await appendAssistant(tx, conversationId, message);
const result = { status: "terminal", outcome: { status: "completed", result: { entryId: entry2.id } } };
@@ -39973,7 +40003,7 @@ function prepareRunEndTasks(runtime, requests, prepared) {
async function endAnsweredRun(runtime, answer2, context) {
const conversationId = runtime.conversationId;
await runtime.commit(async (tx) => {
const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
const boundary = await prepareBoundary(tx, conversationId, runtime.settings, "final", runtime.now());
const live = await tx.doc(LiveDoc, conversationId);
const { users, first } = await applyBoundary(tx, boundary, "final", runtime.now());
endRun(tx, live, runtime.taskId, { status: "done", answer: answer2 });
@@ -40046,7 +40076,7 @@ async function finishToolRound(runtime, assistant, tools, context) {
const maxTurns = (await runtime.agent(context)).maxTurns;
const afterRun = terminate && await asksAfterRun(runtime, context);
await runtime.commit(async (tx) => {
const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
const boundary = await prepareBoundary(tx, conversationId, runtime.settings, terminate || handoff !== void 0 ? "final" : "postTools", runtime.now());
if (added.length > 0)
await addTools(tx, conversationId, added);
const live = await tx.doc(LiveDoc, conversationId);
@@ -40354,19 +40384,19 @@ var Submissions = class {
}
/** Handle for an existing submission, or `undefined`. */
async get(id, context) {
const record = await this.#session.readOnLine((storage) => storage.submission(id, context));
const record = await this.#session.readOnLine((storage2) => storage2.submission(id, context));
return record === void 0 ? void 0 : new SubmissionHandle(record.id, this);
}
async status(id, context) {
const record = await this.#session.readOnLine((storage) => storage.submission(id, context));
const record = await this.#session.readOnLine((storage2) => storage2.submission(id, context));
if (record === void 0)
throw new Error(`Submission ${id} does not exist`);
return record;
}
async wait(id, context) {
this.#resume();
const found = await this.#session.readOnLine(async (storage) => {
const record = await storage.submission(id, context);
const found = await this.#session.readOnLine(async (storage2) => {
const record = await storage2.submission(id, context);
if (record === void 0)
throw new Error(`Submission ${id} does not exist`);
if (isSettled(record))
@@ -40463,13 +40493,13 @@ async function admitSubmission(tx, conversationId, draft, now, queueModes, span)
if (draft.type === "continue") {
if (busy)
throw new ConversationBusy(conversationId);
const boundary2 = await prepareBoundary(tx, conversationId, queueModes);
const boundary2 = await prepareBoundary(tx, conversationId, queueModes, "final", now);
const { id: id2 } = await tx.createSubmission({ ...fields, type: "continue", status: "placed" });
const { users, first } = await applyBoundary(tx, boundary2, "final", now);
await startRun(tx, conversationId, live, { inputs: [...users, id2], first: live.runFailure?.first ?? first });
return { id: id2, status: "placed" };
}
const boundary = busy ? void 0 : await prepareBoundary(tx, conversationId, queueModes);
const boundary = busy ? void 0 : await prepareBoundary(tx, conversationId, queueModes, "final", now, draft.type === "input" ? inputDraft(draft.content, draft.entry, now) : void 0);
if (boundary === void 0 || boundary.inbox.items.length > 0) {
const { id: id2 } = await tx.createSubmission({
...fields,
@@ -40507,6 +40537,8 @@ async function admitSubmission(tx, conversationId, draft, now, queueModes, span)
const write = { ...fields, type: "write", status: "done", entry: entry3.id };
return { id: (await tx.createSubmission(write)).id, status: "done" };
}
for (const before of boundary.entries)
await tx.appendEntry(conversationId, before);
const entry2 = await appendInput(tx, conversationId, draft.content, draft.entry, now);
const input = { ...fields, type: "input", status: "placed", entry: entry2.id };
const { id } = await tx.createSubmission(input);
@@ -41037,9 +41069,9 @@ var TaskGraphView = class {
}
/** Register an observer created from the current revision, atomically on the Session line. */
#attach(create, context) {
return this.#session.readOnLine(async (storage) => {
return this.#session.readOnLine(async (storage2) => {
const mount = this.#mount ?? {
...await this.#build(storage, context),
...await this.#build(storage2, context),
observers: /* @__PURE__ */ new Set()
};
const detach = () => {
@@ -41076,16 +41108,16 @@ var TaskGraphView = class {
});
}
/** Read the graph on the line, through the line's `storage`. */
async #build(storage, context) {
async #build(storage2, context) {
const tasks = {};
const live = /* @__PURE__ */ new Map();
const records = [];
for (const status of LIVE_STATUSES2) {
records.push(...await scanAll((cursor) => storage.scanTasks({ status }, SCAN_PAGE_SIZE4, cursor, context)));
records.push(...await scanAll((cursor) => storage2.scanTasks({ status }, SCAN_PAGE_SIZE4, cursor, context)));
}
records.sort((a, b) => a.id - b.id);
for (const record of records) {
const owned = await scanAll((cursor) => storage.scanConversations({ ownerTaskId: record.id }, SCAN_PAGE_SIZE4, cursor, context));
const owned = await scanAll((cursor) => storage2.scanConversations({ ownerTaskId: record.id }, SCAN_PAGE_SIZE4, cursor, context));
const conversations = owned.map((conversation) => conversation.id).sort((a, b) => a - b);
const subject = blockedSubject(record);
tasks[String(record.id)] = nodeOf2(record, conversations, this.#blocking.blocked(subject));
@@ -41262,9 +41294,9 @@ var ConversationViews = class {
*/
async attach(id, window2, create, context) {
const bounds = resolveWindow(window2);
return this.#session.readOnLine(async (storage) => {
const mount = this.#mounts.get(id) ?? await this.#build(storage, id, context);
const { entries: entries2, before } = await loadWindow(storage, id, bounds, context);
return this.#session.readOnLine(async (storage2) => {
const mount = this.#mounts.get(id) ?? await this.#build(storage2, id, context);
const { entries: entries2, before } = await loadWindow(storage2, id, bounds, context);
const value2 = view(mount.conversation, entries2, before, mount.docs);
let windowed;
const detach = () => {
@@ -41284,8 +41316,8 @@ var ConversationViews = class {
});
}
/** Hydrate a mount on the line, through the line's `storage`. */
async #build(storage, id, context) {
const conversation = await storage.conversation(id, context);
async #build(storage2, id, context) {
const conversation = await storage2.conversation(id, context);
if (conversation === void 0)
throw new Error(`Conversation ${id} does not exist`);
const docs = {};
@@ -41303,25 +41335,25 @@ var ConversationViews = class {
function view(conversation, entries2, before, docs) {
return before === void 0 ? { conversation, entries: entries2, docs } : { conversation, entries: entries2, before, docs };
}
async function loadWindow(storage, conversationId, bounds, context) {
async function loadWindow(storage2, conversationId, bounds, context) {
const { from, min, max } = bounds;
if (from !== void 0) {
const since = await scanUpTo(storage, { conversationId, minEntryId: from, order: "ascending" }, max + 1, context);
const since = await scanUpTo(storage2, { conversationId, minEntryId: from, order: "ascending" }, max + 1, context);
if (since.length <= max) {
const query = { conversationId, maxEntryId: idFromNumber(from - 1) };
const previous = (await storage.scanEntries(query, 1, void 0, context)).items[0];
const previous = (await storage2.scanEntries(query, 1, void 0, context)).items[0];
return { entries: since, before: previous?.id };
}
}
const newest = await scanUpTo(storage, { conversationId }, min + 1, context);
const newest = await scanUpTo(storage2, { conversationId }, min + 1, context);
const before = newest.length > min ? newest.pop().id : void 0;
return { entries: newest.reverse(), before };
}
async function scanUpTo(storage, query, limit2, context) {
async function scanUpTo(storage2, query, limit2, context) {
const entries2 = [];
let cursor;
do {
const page2 = await storage.scanEntries(query, Math.min(SCAN_PAGE_SIZE5, limit2 - entries2.length), cursor, context);
const page2 = await storage2.scanEntries(query, Math.min(SCAN_PAGE_SIZE5, limit2 - entries2.length), cursor, context);
entries2.push(...page2.items);
cursor = page2.next;
} while (cursor !== void 0 && entries2.length < limit2);
@@ -41441,7 +41473,7 @@ var ConversationImpl = class {
}
entries(query, limit2, cursor, context) {
const scan = conversationEntries(this.id, query);
return this.#host.harness.readOnLine((storage) => storage.scanEntries(scan, limit2, cursor, context));
return this.#host.harness.readOnLine((storage2) => storage2.scanEntries(scan, limit2, cursor, context));
}
fork(at, options, context) {
return this.#host.create({ kind: "fork", parentId: this.id, at, ownership: options.ownership }, options, context);
@@ -41482,7 +41514,7 @@ var HarnessImpl = class extends SessionImpl {
const now = safeNow(options.now, report2);
const hostSettings = currentSettings(options.settings);
super(rawStorage, now, () => hostSettings()?.documentRetentionMs, options.setTimer, options.readOnly === true, () => hostSettings()?.progress?.persistWithinMs);
const storage = this.storage;
const storage2 = this.storage;
this.#options = options;
this.#report = report2;
this.#hostSettings = hostSettings;
@@ -41490,7 +41522,7 @@ var HarnessImpl = class extends SessionImpl {
this.#tasks = new TaskScheduler({
telemetry: options.telemetry,
session: this,
storage,
storage: storage2,
registry: options.registry,
models: currentModels(options.models, options.identity),
agent: (id, snapshot, callContext) => this.resolveAgent(id, snapshot, callContext),
@@ -41518,7 +41550,7 @@ var HarnessImpl = class extends SessionImpl {
});
this.#host = {
harness: this,
storage,
storage: storage2,
tasks: this.#tasks,
submissions: this.#submissions,
views: new ConversationViews(this),
@@ -41526,6 +41558,15 @@ var HarnessImpl = class extends SessionImpl {
create: (target, createOptions, context2) => this.#create(target, createOptions, context2)
};
}
async beforeInput(tx, conversationId, input, context) {
const agent = resolveAgent(await tx.doc(AgentDoc, conversationId), this.#options.registry.snapshot(), resolveSettings(this.#hostSettings()), (error, extension) => this.#report(error, { kind: "extension_error", conversationId, extension }, context));
const entries2 = [];
for (const { handlers } of agentHooks(agent, "pi.generation")) {
const result = await handlers.beforeInput?.(input, { conversationId, tx }, context);
entries2.push(...result?.entries ?? []);
}
return entries2;
}
/** Resolve a conversation's committed `pi.agent` against `snapshot`, or the current one, and the current settings. */
async resolveAgent(id, snapshot, context) {
const registry2 = snapshot ?? this.#options.registry.snapshot();
@@ -41548,12 +41589,18 @@ var HarnessImpl = class extends SessionImpl {
this.assertUsable();
this.#tasks.resume();
}
hold() {
this.#tasks.hold();
}
release() {
this.#tasks.release();
}
wake() {
this.assertUsable();
return this.#tasks.wake();
}
getTask(id, context) {
return this.readOnLine((storage) => storage.task(id, context));
return this.readOnLine((storage2) => storage2.task(id, context));
}
/** Internal, for tests: sizes of the scheduler's ownership indexes. */
get [SCHEDULER_INDEX_SIZES]() {
@@ -41564,9 +41611,9 @@ var HarnessImpl = class extends SessionImpl {
return this.#tasks.idleState;
}
inspect(context) {
return this.readOnLine(async (storage) => {
return this.readOnLine(async (storage2) => {
const { scheduling, tasks } = await this.#tasks.inspect(this.#options.registry.snapshot());
const scan = (status) => scanAll((cursor) => storage.scanSubmissions({ status }, SCAN_PAGE_SIZE6, cursor, context));
const scan = (status) => scanAll((cursor) => storage2.scanSubmissions({ status }, SCAN_PAGE_SIZE6, cursor, context));
const submissions = [...await scan("queued"), ...await scan("placed")].sort((a, b) => a.id - b.id);
return { scheduling, tasks, submissions };
});
@@ -41590,7 +41637,7 @@ var HarnessImpl = class extends SessionImpl {
}
/** Sum every conversation's committed `pi.usage`. Each document is read at its own point; totals only grow. */
async usage(context) {
const conversations = await this.readOnLine((storage) => scanAll((cursor) => storage.scanConversations({}, SCAN_PAGE_SIZE6, cursor, context)));
const conversations = await this.readOnLine((storage2) => scanAll((cursor) => storage2.scanConversations({}, SCAN_PAGE_SIZE6, cursor, context)));
const total = UsageDoc.definition.initial();
for (const { id } of conversations) {
const state2 = await this.snapshot(UsageDoc, id, context);
@@ -41615,7 +41662,7 @@ var HarnessImpl = class extends SessionImpl {
}
async conversation(id, context) {
this.assertUsable();
const record = await this.readOnLine((storage) => storage.conversation(id, context));
const record = await this.readOnLine((storage2) => storage2.conversation(id, context));
return record === void 0 ? void 0 : new ConversationImpl(record.id, this.#host);
}
createConversation(options, context) {
@@ -41746,7 +41793,7 @@ function boundConversation(id, binding, submissions, tasks) {
}
var Harness = {
/** Open a Harness over storage. The registry may keep changing while the Harness runs. */
async open(storage, options, context) {
async open(storage2, options, context) {
context.abortSignal?.throwIfAborted();
const snapshot = options.registry.snapshot();
const missing = BUILTIN_TASKS.filter((task) => snapshot.task(task.definition.name) === void 0);
@@ -41754,7 +41801,7 @@ var Harness = {
const names2 = missing.map((task) => task.definition.name).join(", ");
throw new Error(`Registry lacks built-in tasks ${names2}; create it with createRegistry()`);
}
const harness2 = new HarnessImpl(storage, options, context);
const harness2 = new HarnessImpl(storage2, options, context);
try {
await harness2.openTasks(context);
} catch (error) {
@@ -44482,6 +44529,29 @@ async function startRequests(harness2, context) {
};
}
// input.ts
async function serve(harness2, context, afterSubmit) {
while (true) {
const input = await Deno.core.ops.op_input();
if (input === null || input.type === "stop") {
await harness2.waitForIdle(context);
return;
}
const conversation = await harness2.conversation(input.conversation, context);
if (!conversation) throw new Error(`Unknown conversation: ${input.conversation}`);
const submission2 = await conversation.submit({
type: "input",
content: input.content,
requestId: input.requestId
}, context);
if (afterSubmit) {
const result = await submission2.wait(context);
await afterSubmit();
if (result.status !== "done") throw new Error(`Run ended: ${JSON.stringify(result)}`);
}
}
}
// main.ts
var proxy = Deno.core.ops.op_model_proxy();
if (!proxy?.base_url || !proxy?.api_key) throw new Error("The llm-proxy credential is required");
@@ -44516,9 +44586,11 @@ models.setProvider({
var registry = createRegistry();
registry.install(CodingTools);
registry.install(CredentialRequests);
var harness = await Harness.open(await openStorage(globalThis.storagePath), {
var storage = await openStorage(globalThis.storagePath);
var harness = await Harness.open(storage, {
models,
registry,
settings: { followUpMode: "all" },
env: (target) => environment(target.cwd ?? "/", target.conversationId)
}, BACKGROUND_CONTEXT);
var root = await harness.root(BACKGROUND_CONTEXT, { agent: { model: { provider: "openai", modelId: model2.id }, thinkingLevel: "low", cwd: "/", extensions: [CodingTools, CredentialRequests] } });
@@ -44543,25 +44615,21 @@ var render = (view2) => {
for (const entry2 of watch.value.entries) {
if (entry2.kind === "pi.assistant") emitted.set(entry2.id, text(entry2.model?.[0]));
}
watch.start((event) => {
if (globalThis.consoleEnabled) watch.start((event) => {
if (event.kind === "value") render(event.value);
});
var stopRequests = await startRequests(harness, BACKGROUND_CONTEXT);
harness.resume();
await harness.waitForIdle(BACKGROUND_CONTEXT);
while (true) {
const line = await Deno.core.ops.op_input();
if (line === null) break;
if (!line.trim()) continue;
const submission2 = await root.submit({ type: "input", content: line, requestId: crypto.randomUUID() }, BACKGROUND_CONTEXT);
const result = await submission2.wait(BACKGROUND_CONTEXT);
await serve(harness, BACKGROUND_CONTEXT, globalThis.consoleEnabled ? async () => {
await harness.waitForIdle(BACKGROUND_CONTEXT);
const final = await root.watch(BACKGROUND_CONTEXT);
render(final.value);
await final.stop();
Deno.core.ops.op_output("\n");
if (result.status !== "done") throw new Error(`Run ended: ${JSON.stringify(result)}`);
}
} : void 0);
await stopRequests();
await watch.stop();
await harness.close(BACKGROUND_CONTEXT);
export {
storage
};
+58 -11
View File
@@ -8883,6 +8883,9 @@ var Transaction = class {
this.#assertOpen();
return record;
}
beforeInput(conversationId, input) {
return this.#read("beforeInput", () => this.#host.beforeInput(this, conversationId, input, this.#context));
}
appendEntry(first, second, third) {
if (typeof first === "number")
return this.#appendEntry(first, second);
@@ -10065,6 +10068,7 @@ var SessionImpl = class {
if (this.#documents.get(id)?.record.id === recordId)
this.#dropDocument(id);
},
beforeInput: (tx, id, input, context) => this.beforeInput(tx, id, input, context),
conversationCreated: (tx, record) => this.conversationCreated(tx, record),
taskDocuments: (taskId) => this.#taskDocuments.get(taskId)?.values(),
latestEntryId: (conversationId, context) => this.latestEntryId(conversationId, context),
@@ -10389,6 +10393,9 @@ var SessionImpl = class {
*/
report(_error, _info = LISTENER, _context) {
}
async beforeInput(_tx, _id, _input, _context) {
return [];
}
/**
* Runs inside every transaction that creates or forks a conversation, after the conversation record is staged. A
* plain Session stages nothing; a Harness stages its built-in documents.
@@ -12193,10 +12200,14 @@ var InboxDoc = defineDoc({
initial: () => ({ items: [] }),
checkpointWhen: (value2) => value2.items.length === 0
});
async function prepareBoundary(tx, conversationId, modes) {
async function prepareBoundary(tx, conversationId, modes, at, now, incoming) {
const head = (await tx.latestHeadMarker(conversationId))?.head;
const inbox = await tx.doc(InboxDoc, conversationId);
const first = inbox.items.find((item) => item.mode === "steer" || item.mode === "followUp");
const input = first === void 0 ? incoming : inputDraft(first.content, first.entry, now);
const entries2 = input !== void 0 && (at === "final" || hasQueuedReset(inbox.items)) ? await tx.beforeInput(conversationId, input) : [];
return {
entries: entries2,
conversationId,
inbox,
steeringMode: modes.steeringMode,
@@ -12232,6 +12243,9 @@ async function applyBoundary(tx, boundary, at, now) {
}
const placed = [];
for (const index of users) {
if (placed.length === 0)
for (const draft of boundary.entries)
await tx.appendEntry(conversationId, draft);
const item = items[index];
const entry2 = await appendInput(tx, conversationId, item.content, item.entry, now);
tx.placeSubmission(item.id, entry2.id);
@@ -12244,10 +12258,11 @@ async function applyBoundary(tx, boundary, at, now) {
return { users: placed, first, reset };
}
function appendInput(tx, conversationId, content, entry2, now) {
return tx.appendEntry(conversationId, inputDraft(content, entry2, now));
}
function inputDraft(content, entry2, now) {
const message = { role: "user", content, timestamp: now };
if (entry2 === void 0)
return tx.appendEntry(UserEntry, conversationId, { model: [message] });
return tx.appendEntry(conversationId, { ...pickInputEntry(entry2), model: [message] });
return { ...entry2 === void 0 ? { kind: UserEntry.kind } : pickInputEntry(entry2), input: true, model: [message] };
}
function pickInputEntry({ kind, data }) {
const picked = data === void 0 ? { kind } : { kind, data };
@@ -12959,6 +12974,7 @@ var TaskScheduler = class {
#unsubscribeRegistry = () => {
};
#enabled = false;
#held = false;
#readOnly;
#closing = false;
#dirty = false;
@@ -12987,6 +13003,14 @@ var TaskScheduler = class {
this.#context = options.context;
this.#readOnly = options.readOnly === true;
}
/** Start no run invocation until `release()`; running invocations and abort handlers go on. Overrides `resume()`. */
hold() {
this.#held = true;
}
release() {
this.#held = false;
this.#kick();
}
/**
* Load live tasks and change surviving `running` tasks back to `pending`, counting the restart. Dispatches nothing;
* a task cut off `settings.restartLimit` times ends when it is next reserved. Read-only, it only loads them: a
@@ -14441,6 +14465,8 @@ var TaskScheduler = class {
for (const record of plan.ready) {
const id = record.id;
const mode = record.abortRequested ? "abort" : "run";
if (mode === "run" && this.#held)
continue;
const cutOff = record.restarts !== void 0 && record.restarts >= this.#restartLimit();
if (cutOff && (mode === "abort" || !this.#limitsRestarts(record.kind))) {
const message = mode === "abort" ? `Abort handler of task ${id} (${record.kind}) was cut off ${record.restarts} times` : `Task ${id} (${record.kind}) was cut off ${record.restarts} times without progress`;
@@ -14626,7 +14652,7 @@ var TaskScheduler = class {
const tasks = [];
for (const record of this.#live.values())
tasks.push({ record, state: this.#inspectTask(record, snapshot) });
const scheduling = this.#closing ? "closing" : this.#enabled ? "running" : "paused";
const scheduling = this.#closing ? "closing" : this.#enabled && !this.#held ? "running" : "paused";
return { scheduling, tasks };
}
#inspectTask(record, snapshot) {
@@ -14781,6 +14807,10 @@ var TaskScheduler = class {
* invocation continues.
*/
#progress(tx, current, state, extension, context) {
if (this.#held) {
tx.setTask(withState(current, { status: "pending", checkpoint: current.state.checkpoint }));
return false;
}
state.snapshot = this.#registry.snapshot();
const next = state.snapshot.task(current.kind);
if (next === state.task)
@@ -17429,7 +17459,7 @@ async function prepareFailure(tx, runtime) {
const policy = settings.afterFailedRun;
if (policy !== "continue")
return { policy };
return { policy, boundary: await prepareBoundary(tx, runtime.conversationId, settings) };
return { policy, boundary: await prepareBoundary(tx, runtime.conversationId, settings, "final", runtime.now()) };
}
async function endFailedRun(tx, runtime, live, failure, end) {
const conversationId = runtime.conversationId;
@@ -17597,7 +17627,7 @@ async function answer(runtime, message, context) {
const afterRun = await asksAfterRun(runtime, context);
const conversationId = runtime.conversationId;
await runtime.commit(async (tx) => {
const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
const boundary = await prepareBoundary(tx, conversationId, runtime.settings, "final", runtime.now());
const live = await tx.doc(LiveDoc, conversationId);
const entry2 = await appendAssistant(tx, conversationId, message);
const result = { status: "terminal", outcome: { status: "completed", result: { entryId: entry2.id } } };
@@ -17670,7 +17700,7 @@ function prepareRunEndTasks(runtime, requests, prepared) {
async function endAnsweredRun(runtime, answer2, context) {
const conversationId = runtime.conversationId;
await runtime.commit(async (tx) => {
const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
const boundary = await prepareBoundary(tx, conversationId, runtime.settings, "final", runtime.now());
const live = await tx.doc(LiveDoc, conversationId);
const { users, first } = await applyBoundary(tx, boundary, "final", runtime.now());
endRun(tx, live, runtime.taskId, { status: "done", answer: answer2 });
@@ -17743,7 +17773,7 @@ async function finishToolRound(runtime, assistant, tools, context) {
const maxTurns = (await runtime.agent(context)).maxTurns;
const afterRun = terminate && await asksAfterRun(runtime, context);
await runtime.commit(async (tx) => {
const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
const boundary = await prepareBoundary(tx, conversationId, runtime.settings, terminate || handoff !== void 0 ? "final" : "postTools", runtime.now());
if (added.length > 0)
await addTools(tx, conversationId, added);
const live = await tx.doc(LiveDoc, conversationId);
@@ -18159,13 +18189,13 @@ async function admitSubmission(tx, conversationId, draft, now, queueModes, span)
if (draft.type === "continue") {
if (busy)
throw new ConversationBusy(conversationId);
const boundary2 = await prepareBoundary(tx, conversationId, queueModes);
const boundary2 = await prepareBoundary(tx, conversationId, queueModes, "final", now);
const { id: id2 } = await tx.createSubmission({ ...fields, type: "continue", status: "placed" });
const { users, first } = await applyBoundary(tx, boundary2, "final", now);
await startRun(tx, conversationId, live, { inputs: [...users, id2], first: live.runFailure?.first ?? first });
return { id: id2, status: "placed" };
}
const boundary = busy ? void 0 : await prepareBoundary(tx, conversationId, queueModes);
const boundary = busy ? void 0 : await prepareBoundary(tx, conversationId, queueModes, "final", now, draft.type === "input" ? inputDraft(draft.content, draft.entry, now) : void 0);
if (boundary === void 0 || boundary.inbox.items.length > 0) {
const { id: id2 } = await tx.createSubmission({
...fields,
@@ -18203,6 +18233,8 @@ async function admitSubmission(tx, conversationId, draft, now, queueModes, span)
const write = { ...fields, type: "write", status: "done", entry: entry3.id };
return { id: (await tx.createSubmission(write)).id, status: "done" };
}
for (const before of boundary.entries)
await tx.appendEntry(conversationId, before);
const entry2 = await appendInput(tx, conversationId, draft.content, draft.entry, now);
const input = { ...fields, type: "input", status: "placed", entry: entry2.id };
const { id } = await tx.createSubmission(input);
@@ -19215,6 +19247,15 @@ var HarnessImpl = class extends SessionImpl {
create: (target, createOptions, context2) => this.#create(target, createOptions, context2)
};
}
async beforeInput(tx, conversationId, input, context) {
const agent = resolveAgent(await tx.doc(AgentDoc, conversationId), this.#options.registry.snapshot(), resolveSettings(this.#hostSettings()), (error, extension) => this.#report(error, { kind: "extension_error", conversationId, extension }, context));
const entries2 = [];
for (const { handlers } of agentHooks(agent, "pi.generation")) {
const result = await handlers.beforeInput?.(input, { conversationId, tx }, context);
entries2.push(...result?.entries ?? []);
}
return entries2;
}
/** Resolve a conversation's committed `pi.agent` against `snapshot`, or the current one, and the current settings. */
async resolveAgent(id, snapshot, context) {
const registry2 = snapshot ?? this.#options.registry.snapshot();
@@ -19237,6 +19278,12 @@ var HarnessImpl = class extends SessionImpl {
this.assertUsable();
this.#tasks.resume();
}
hold() {
this.#tasks.hold();
}
release() {
this.#tasks.release();
}
wake() {
this.assertUsable();
return this.#tasks.wake();
File diff suppressed because it is too large Load Diff
+127
View File
@@ -0,0 +1,127 @@
// Ultimate's input-placement cases, scheduler hold, and the headless channel contract.
import { BACKGROUND_CONTEXT as context } from '@earendil-works/chord/context';
import { createModels } from '@earendil-works/pi-ai/models';
import { fauxProvider, fauxAssistantMessage, fauxToolCall } from '@earendil-works/pi-ai/providers/faux';
import { Harness, createRegistry, defineDoc, defineExtension, defineTask, GenerationTask, hook, InboxDoc } from '@earendil-works/pi-durable';
import { openStorage } from './sqlite';
import { serve } from './input';
const assert = (value: any, message: string) => { if (!value) throw new Error(message); };
const equal = (a: any, b: any) => assert(JSON.stringify(a) === JSON.stringify(b), `Expected ${JSON.stringify(b)}, got ${JSON.stringify(a)}`);
const until = async (condition: () => any) => {
const deadline = Date.now() + 10000;
while (!await condition()) {
if (Date.now() > deadline) throw new Error('Condition deadline');
await new Promise(resolve => setTimeout(resolve, 0));
}
};
const Placed = defineDoc<{ count: number }>({ kind: 'input-test.placed', version: 1,
checkpointWhen: () => false, scope: 'conversation', history: 'latest', fork: 'initial', initial: () => ({ count: 0 }) });
const gate = Promise.withResolvers<void>();
let held = false, reject = false, calls = 0, second = false;
const Notes = defineExtension({ name: 'input-test', hooks: [hook(GenerationTask, {
beforeInput: async (_input, { tx, conversationId }) => {
calls++;
const state = await tx.doc(Placed, conversationId);
state.count++;
if (reject) throw new Error('reject placement');
return { entries: [{ kind: 'input-test.note', data: { count: state.count },
model: [{ role: 'user', content: `before ${state.count}`, timestamp: 1 }] }] };
},
})] });
const Park = defineTask<null, { phase: 'first' | 'second' }, null>({ name: 'hold-test', version: 1,
initial: () => ({ phase: 'first' }), phases: {
first: async (_task, runtime, ctx) => {
held = true;
await gate.promise;
await runtime.commit(() => ({ status: 'pending', checkpoint: { phase: 'second' } }), ctx);
},
second: async (_task, runtime, ctx) => {
second = true;
await runtime.commit(() => ({ status: 'terminal', outcome: { status: 'completed', result: null } }), ctx);
},
}, abort: (_task, runtime, ctx) => runtime.commit(() => ({ status: 'terminal', outcome: { status: 'aborted' } }), ctx),
});
const models = createModels();
const faux = fauxProvider({ tokensPerSecond: 100000 });
models.setProvider(faux.provider);
const registry = createRegistry();
registry.install(Notes);
registry.install(defineExtension({ name: 'hold-test', tasks: [Park] }));
const storage = await openStorage(globalThis.storagePath);
const harness = await Harness.open(storage, { models, registry, settings: { followUpMode: 'all' } }, context);
const scenario = globalThis.foundationCase;
const root = await harness.root(context, { agent: { model: { provider: 'faux', modelId: 'faux-1' },
extensions: scenario === 'placement' ? [Notes] : [] } });
try {
if (scenario === 'placement' || scenario === 'followups') {
faux.setResponses([async () => {
held = true;
await gate.promise;
return fauxAssistantMessage([fauxToolCall('unknown', {})], { stopReason: 'toolUse' });
}, fauxAssistantMessage('first answer'), fauxAssistantMessage('follow-ups answer'),
fauxAssistantMessage('extra'), fauxAssistantMessage('extra')]);
const first = await root.submit({ type: 'input', content: 'first', requestId: 'one' }, context);
await until(() => held);
const queued = [];
const followups = scenario === 'placement' ? ['next'] : ['one', 'two', 'three'];
for (const content of followups) queued.push(await root.submit({ type: 'input', content, whenBusy: 'followUp' }, context));
const steer = await root.submit({ type: 'input', content: 'steer', whenBusy: 'steer' }, context);
if (scenario === 'placement') equal(calls, 1);
gate.resolve();
await Promise.all([first, steer, ...queued].map(submission => submission.wait(context)));
const view = await root.context(context);
if (scenario === 'placement') {
equal(view.messages.filter(m => m.role === 'user').map(m => m.content), ['before 1', 'first', 'steer', 'before 2', 'next']);
const notes = view.entries.filter(e => e.kind === 'input-test.note');
equal(notes.length, 2);
assert(notes.every(e => e.edits === undefined), 'Hook notes must be placed, not edits');
equal(calls, 2);
assert(view.entries.filter(e => e.kind === 'pi.user').every(e => e.input === true), 'Inputs need their marker');
await root.submit({ type: 'input', content: 'first', requestId: 'one' }, context);
equal(calls, 2);
reject = true;
let rejected = false;
try { await root.submit({ type: 'input', content: 'rejected' }, context); }
catch (error) { rejected = String(error).includes('reject placement'); }
assert(rejected, 'Placement must reject');
equal((await harness.snapshot(Placed, root.id, context))?.count, 2);
equal((await root.context(context)).entries.filter(e => e.kind === 'input-test.note').length, 2);
} else {
equal(view.messages.flatMap(m => m.role === 'user' ? [m.content] : m.role === 'assistant'
? m.content.flatMap(p => p.type === 'text' ? [`> ${p.text}`] : []) : []),
['first', 'steer', '> first answer', 'one', 'two', 'three', '> follow-ups answer']);
equal(await harness.snapshot(InboxDoc, root.id, context), { items: [] });
}
} else if (scenario === 'thinking') {
faux.setResponses([fauxAssistantMessage('unset'), fauxAssistantMessage('high')]);
await (await root.submit({ type: 'input', content: 'one' }, context)).wait(context);
await root.configure({ thinkingLevel: 'high' }, context);
await (await root.submit({ type: 'input', content: 'two' }, context)).wait(context);
equal((await root.context(context)).messages.flatMap(m => m.role === 'assistant' ? [m.thinkingLevel] : []), ['off', 'high']);
} else if (scenario === 'hold') {
const task = await root.commit(tx => tx.createTask(Park, null, { ownership: { kind: 'conversation' } }), context);
harness.resume();
await until(() => held);
harness.hold();
gate.resolve();
await until(async () => (await harness.getTask(task, context))?.state.status === 'pending');
equal((await harness.getTask(task, context))?.state.checkpoint.phase, 'second');
harness.resume();
equal((await harness.inspect(context)).scheduling, 'paused');
assert(!second, 'Held scheduler started another phase');
harness.release();
await until(() => second);
await harness.waitForIdle(context);
equal((await harness.getTask(task, context))?.state.status, 'terminal');
} else if (scenario === 'headless') {
faux.setResponses([fauxAssistantMessage('channel answer')]);
await serve(harness, context);
const submissions = await storage.scanSubmissions({ conversationId: root.id }, 100, undefined, context);
equal(submissions.items.length, 1);
equal(submissions.items[0].requestId, 'channel-request');
assert(JSON.stringify(await root.context(context)).includes('channel answer'), 'Channel answer missing');
} else throw new Error('Unknown foundation case');
} finally {
gate.resolve();
await harness.close(context);
}
+20
View File
@@ -0,0 +1,20 @@
// Channel submissions target any conversation. EOF and Stop close the harness cleanly.
export async function serve(harness: any, context: any, afterSubmit?: () => Promise<void>) {
while (true) {
const input = await Deno.core.ops.op_input();
if (input === null || input.type === 'stop') {
await harness.waitForIdle(context);
return;
}
const conversation = await harness.conversation(input.conversation, context);
if (!conversation) throw new Error(`Unknown conversation: ${input.conversation}`);
const submission = await conversation.submit({
type: 'input', content: input.content, requestId: input.requestId,
}, context);
if (afterSubmit) {
const result = await submission.wait(context);
await afterSubmit();
if (result.status !== 'done') throw new Error(`Run ended: ${JSON.stringify(result)}`);
}
}
}
+8 -12
View File
@@ -7,6 +7,7 @@ import { CodingTools } from '@earendil-works/pi-durable/tools';
import { openStorage } from './sqlite';
import { environment } from './environment';
import { CredentialRequests, startRequests } from './credentials';
import { serve } from './input';
const proxy = Deno.core.ops.op_model_proxy();
if (!proxy?.base_url || !proxy?.api_key) throw new Error('The llm-proxy credential is required');
const models = createModels();
@@ -27,8 +28,10 @@ models.setProvider({ ...provider, baseUrl: model.baseUrl, getModels: () => [mode
const registry = createRegistry();
registry.install(CodingTools);
registry.install(CredentialRequests);
const harness = await Harness.open(await openStorage(globalThis.storagePath), {
models, registry, env: (target: any) => environment(target.cwd ?? '/', target.conversationId),
export const storage = await openStorage(globalThis.storagePath);
const harness = await Harness.open(storage, {
models, registry, settings: { followUpMode: 'all' },
env: (target: any) => environment(target.cwd ?? '/', target.conversationId),
}, context);
const root = await harness.root(context, { agent: { model: { provider: 'openai', modelId: model.id }, thinkingLevel: 'low', cwd: '/', extensions: [CodingTools, CredentialRequests] } });
const watch = await root.watch(context);
@@ -53,23 +56,16 @@ const render = (view: any) => {
for (const entry of watch.value.entries) {
if (entry.kind === 'pi.assistant') emitted.set(entry.id, text(entry.model?.[0]));
}
watch.start((event: any) => { if (event.kind === 'value') render(event.value); });
if (globalThis.consoleEnabled) watch.start((event: any) => { if (event.kind === 'value') render(event.value); });
const stopRequests = await startRequests(harness, context);
harness.resume();
await harness.waitForIdle(context);
while (true) {
const line = await Deno.core.ops.op_input();
if (line === null) break;
if (!line.trim()) continue;
const submission = await root.submit({ type: 'input', content: line, requestId: crypto.randomUUID() }, context);
const result = await submission.wait(context);
await serve(harness, context, globalThis.consoleEnabled ? async () => {
await harness.waitForIdle(context);
const final = await root.watch(context);
render(final.value);
await final.stop();
Deno.core.ops.op_output('\n');
if (result.status !== 'done') throw new Error(`Run ended: ${JSON.stringify(result)}`);
}
} : undefined);
await stopRequests();
await watch.stop();
await harness.close(context);
+1 -1
View File
@@ -1,7 +1,7 @@
{
"private": true,
"type": "module",
"scripts": { "bundle": "esbuild main.ts --bundle --format=esm --platform=browser --outfile=bundle.js && esbuild test.ts --bundle --format=esm --platform=browser --outfile=test-bundle.js && esbuild durable-test.ts --bundle --format=esm --platform=browser --outfile=durable-test-bundle.js" },
"scripts": { "bundle": "nu apply-patch.nu && esbuild main.ts --bundle --format=esm --platform=browser --outfile=bundle.js && esbuild test.ts --bundle --format=esm --platform=browser --outfile=test-bundle.js && esbuild durable-test.ts --bundle --format=esm --platform=browser --outfile=durable-test-bundle.js && esbuild foundation-test.ts --bundle --format=esm --platform=browser --outfile=foundation-test-bundle.js" },
"dependencies": {
"@earendil-works/pi-durable": "1.5.0",
"@earendil-works/pi-ai": "1.5.0",
+387
View File
@@ -0,0 +1,387 @@
diff --git a/dist/harness/generation.js b/dist/harness/generation.js
index ac42dfe741b20726ab8eb3d8139b1296d664210d..1e7f1ac26d17b83cb15bf00408fa70b0aee4e747 100644
--- a/dist/harness/generation.js
+++ b/dist/harness/generation.js
@@ -458,7 +458,7 @@ async function prepareFailure(tx, runtime) {
const policy = settings.afterFailedRun;
if (policy !== "continue")
return { policy };
- return { policy, boundary: await prepareBoundary(tx, runtime.conversationId, settings) };
+ return { policy, boundary: await prepareBoundary(tx, runtime.conversationId, settings, "final", runtime.now()) };
}
/**
* End a failed run, settling its inputs with `settlement`, and do with the inputs queued behind it what
@@ -668,7 +668,7 @@ async function answer(runtime, message, context) {
const conversationId = runtime.conversationId;
await runtime.commit(async (tx) => {
// Queue modes are read on the Session line, when the boundary is decided.
- const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
+ const boundary = await prepareBoundary(tx, conversationId, runtime.settings, "final", runtime.now());
const live = await tx.doc(LiveDoc, conversationId);
const entry = await appendAssistant(tx, conversationId, message);
const result = { status: "terminal", outcome: { status: "completed", result: { entryId: entry.id } } };
@@ -762,7 +762,7 @@ function prepareRunEndTasks(runtime, requests, prepared) {
async function endAnsweredRun(runtime, answer, context) {
const conversationId = runtime.conversationId;
await runtime.commit(async (tx) => {
- const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
+ const boundary = await prepareBoundary(tx, conversationId, runtime.settings, "final", runtime.now());
const live = await tx.doc(LiveDoc, conversationId);
const { users, first } = await applyBoundary(tx, boundary, "final", runtime.now());
endRun(tx, live, runtime.taskId, { status: "done", answer });
@@ -868,7 +868,7 @@ async function finishToolRound(runtime, assistant, tools, context) {
const maxTurns = (await runtime.agent(context)).maxTurns;
const afterRun = terminate && (await asksAfterRun(runtime, context));
await runtime.commit(async (tx) => {
- const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
+ const boundary = await prepareBoundary(tx, conversationId, runtime.settings, terminate || handoff !== undefined ? "final" : "postTools", runtime.now());
if (added.length > 0)
await addTools(tx, conversationId, added);
const live = await tx.doc(LiveDoc, conversationId);
diff --git a/dist/harness/harness.js b/dist/harness/harness.js
index e286f00f872ebec79367a594938040c6a37a879f..44979b05c2bcf30580841797175c099c8c621462 100644
--- a/dist/harness/harness.js
+++ b/dist/harness/harness.js
@@ -5,7 +5,7 @@ import { ResetEntry } from "../entries.js";
import { contained, SessionImpl } from "../session/session.js";
import { hostTimers } from "../timers.js";
import { ROOT_CONVERSATION_ID } from "../types.js";
-import { AgentDoc, configure, createAgent, resolveAgent, resolveSettings } from "./agent.js";
+import { AgentDoc, agentHooks, configure, createAgent, resolveAgent, resolveSettings } from "./agent.js";
import { createCompaction } from "./compaction.js";
import { readContext } from "./context.js";
import { InboxDoc, withdrawQueuedInputs } from "./inbox.js";
@@ -157,6 +157,15 @@ class HarnessImpl extends SessionImpl {
create: (target, createOptions, context) => this.#create(target, createOptions, context),
};
}
+ async beforeInput(tx, conversationId, input, context) {
+ const agent = resolveAgent(await tx.doc(AgentDoc, conversationId), this.#options.registry.snapshot(), resolveSettings(this.#hostSettings()), (error, extension) => this.#report(error, { kind: "extension_error", conversationId, extension }, context));
+ const entries = [];
+ for (const { handlers } of agentHooks(agent, "pi.generation")) {
+ const result = await handlers.beforeInput?.(input, { conversationId, tx }, context);
+ entries.push(...(result?.entries ?? []));
+ }
+ return entries;
+ }
/** Resolve a conversation's committed `pi.agent` against `snapshot`, or the current one, and the current settings. */
async resolveAgent(id, snapshot, context) {
const registry = snapshot ?? this.#options.registry.snapshot();
@@ -179,6 +188,12 @@ class HarnessImpl extends SessionImpl {
this.assertUsable();
this.#tasks.resume();
}
+ hold() {
+ this.#tasks.hold();
+ }
+ release() {
+ this.#tasks.release();
+ }
wake() {
this.assertUsable();
return this.#tasks.wake();
diff --git a/dist/harness/inbox.d.ts b/dist/harness/inbox.d.ts
index 1197339369188857a1ad62b528cea65e3833f27e..a61d9712ce55e80441847442772da6f3e354fb2b 100644
--- a/dist/harness/inbox.d.ts
+++ b/dist/harness/inbox.d.ts
@@ -23,6 +23,7 @@ export declare const InboxDoc: import("../types.ts").ConversationDocToken<InboxS
export type QueueModes = Pick<Settings, "steeringMode" | "followUpMode">;
/** What a boundary reads before the commit's first table write, and the newest head it has seen so far. */
export type Boundary = {
+ readonly entries: readonly EntryDraft[];
readonly conversationId: ConversationId;
readonly inbox: Draft<InboxState>;
readonly steeringMode: QueueMode;
@@ -43,7 +44,7 @@ export type BoundaryResult = {
* Read what a boundary needs. Table reads must precede the commit's first table write, so callers prepare the
* boundary at the start of their commit.
*/
-export declare function prepareBoundary(tx: Tx, conversationId: ConversationId, modes: QueueModes): Promise<Boundary>;
+export declare function prepareBoundary(tx: Tx, conversationId: ConversationId, modes: QueueModes, at: "postTools" | "final", now: number, incoming?: EntryDraft): Promise<Boundary>;
/**
* Place the queued items a boundary selects (spec §6): every write, the first or all steers, and at `final` the first
* or all follow-ups. A selected reset turns a `postTools` boundary into `final`. Writes are placed first and user
@@ -53,6 +54,7 @@ export declare function prepareBoundary(tx: Tx, conversationId: ConversationId,
*/
export declare function applyBoundary(tx: Tx, boundary: Boundary, at: "postTools" | "final", now: number): Promise<BoundaryResult>;
/** Append the entry an input is placed as: `entry`'s kind and data, or `pi.user`, with the user message of `content`. */
+export declare function inputDraft(content: UserInput, entry: InputEntry | undefined, now: number): EntryDraft;
export declare function appendInput(tx: Tx, conversationId: ConversationId, content: UserInput, entry: InputEntry | undefined, now: number): Promise<EntryRecord>;
/** The kind and data of an input's entry as strict JSON, without other fields an untyped caller might pass. */
export declare function pickInputEntry({ kind, data }: InputEntry): JsonRepresentation<InputEntry>;
diff --git a/dist/harness/inbox.js b/dist/harness/inbox.js
index 5ff8c1703808c5ffd703164e6c339150e1f1e131..ec4cf6d5adc599e5fe428514a7cd7e0dac1a7c9d 100644
--- a/dist/harness/inbox.js
+++ b/dist/harness/inbox.js
@@ -14,10 +14,15 @@ export const InboxDoc = defineDoc({
* Read what a boundary needs. Table reads must precede the commit's first table write, so callers prepare the
* boundary at the start of their commit.
*/
-export async function prepareBoundary(tx, conversationId, modes) {
+export async function prepareBoundary(tx, conversationId, modes, at, now, incoming) {
const head = (await tx.latestHeadMarker(conversationId))?.head;
const inbox = await tx.doc(InboxDoc, conversationId);
+ const first = inbox.items.find(item => item.mode === "steer" || item.mode === "followUp");
+ const input = first === undefined ? incoming : inputDraft(first.content, first.entry, now);
+ const entries = input !== undefined && (at === "final" || hasQueuedReset(inbox.items))
+ ? await tx.beforeInput(conversationId, input) : [];
return {
+ entries,
conversationId,
inbox,
steeringMode: modes.steeringMode,
@@ -61,6 +66,9 @@ export async function applyBoundary(tx, boundary, at, now) {
}
const placed = [];
for (const index of users) {
+ if (placed.length === 0)
+ for (const draft of boundary.entries)
+ await tx.appendEntry(conversationId, draft);
const item = items[index];
const entry = await appendInput(tx, conversationId, item.content, item.entry, now);
tx.placeSubmission(item.id, entry.id);
@@ -74,10 +82,11 @@ export async function applyBoundary(tx, boundary, at, now) {
}
/** Append the entry an input is placed as: `entry`'s kind and data, or `pi.user`, with the user message of `content`. */
export function appendInput(tx, conversationId, content, entry, now) {
+ return tx.appendEntry(conversationId, inputDraft(content, entry, now));
+}
+export function inputDraft(content, entry, now) {
const message = { role: "user", content, timestamp: now };
- if (entry === undefined)
- return tx.appendEntry(UserEntry, conversationId, { model: [message] });
- return tx.appendEntry(conversationId, { ...pickInputEntry(entry), model: [message] });
+ return { ...(entry === undefined ? { kind: UserEntry.kind } : pickInputEntry(entry)), input: true, model: [message] };
}
/** The kind and data of an input's entry as strict JSON, without other fields an untyped caller might pass. */
export function pickInputEntry({ kind, data }) {
diff --git a/dist/harness/scheduler.d.ts b/dist/harness/scheduler.d.ts
index f6189edb71ac0557060b623625fc4be4bec5e904..269a3c4ab1676da22db5a723f30d2974f84b6bea 100644
--- a/dist/harness/scheduler.d.ts
+++ b/dist/harness/scheduler.d.ts
@@ -107,6 +107,8 @@ export declare class TaskScheduler {
* Idempotent; the kick does nothing once closing.
*/
resume(): void;
+ hold(): void;
+ release(): void;
/**
* Why a live task cannot run here, if it cannot: its definition is missing, too old, or failed to migrate. A running
* task, an abort-marked one, and one holding its outcome are never blocked.
diff --git a/dist/harness/scheduler.js b/dist/harness/scheduler.js
index 365ea3ab42dd4406ab11a03d6702b238dfd0776b..3e8170fdb790e492e95a390fa6ea4693eb3d2f97 100644
--- a/dist/harness/scheduler.js
+++ b/dist/harness/scheduler.js
@@ -179,6 +179,7 @@ export class TaskScheduler {
#cascadePending = false;
#unsubscribeRegistry = () => { };
#enabled = false;
+ #held = false;
#readOnly;
#closing = false;
#dirty = false;
@@ -207,6 +208,14 @@ export class TaskScheduler {
this.#context = options.context;
this.#readOnly = options.readOnly === true;
}
+ /** Start no run invocation until `release()`; running invocations and abort handlers go on. Overrides `resume()`. */
+ hold() {
+ this.#held = true;
+ }
+ release() {
+ this.#held = false;
+ this.#kick();
+ }
/**
* Load live tasks and change surviving `running` tasks back to `pending`, counting the restart. Dispatches nothing;
* a task cut off `settings.restartLimit` times ends when it is next reserved. Read-only, it only loads them: a
@@ -1771,6 +1780,8 @@ export class TaskScheduler {
for (const record of plan.ready) {
const id = record.id;
const mode = record.abortRequested ? "abort" : "run";
+ if (mode === "run" && this.#held)
+ continue;
// Work cut off `restartLimit` times in a row would run forever: an abort handler, or a phase of a kind that
// does not end itself at the limit, ends the task `faulted` instead.
// The settings are read only for a task that was cut off at all.
@@ -1973,7 +1984,7 @@ export class TaskScheduler {
const tasks = [];
for (const record of this.#live.values())
tasks.push({ record, state: this.#inspectTask(record, snapshot) });
- const scheduling = this.#closing ? "closing" : this.#enabled ? "running" : "paused";
+ const scheduling = this.#closing ? "closing" : this.#enabled && !this.#held ? "running" : "paused";
return { scheduling, tasks };
}
#inspectTask(record, snapshot) {
@@ -2151,6 +2162,11 @@ export class TaskScheduler {
* invocation continues.
*/
#progress(tx, current, state, extension, context) {
+ // Held: park at this phase boundary, as a handover does; reservation starts it again after `release()`.
+ if (this.#held) {
+ tx.setTask(withState(current, { status: "pending", checkpoint: current.state.checkpoint }));
+ return false;
+ }
state.snapshot = this.#registry.snapshot();
const next = state.snapshot.task(current.kind);
if (next === state.task)
diff --git a/dist/harness/submissions.js b/dist/harness/submissions.js
index 6e882a8c252dda942b8b45e5b1837cf2fee28341..89a85e8a52bf67044ff15c3c856a650dea4c7c27 100644
--- a/dist/harness/submissions.js
+++ b/dist/harness/submissions.js
@@ -3,7 +3,7 @@ import { withTelemetry } from "@earendil-works/chord/context";
import { createTypedSpanStarter, NOOP_TELEMETRY_CONTEXT, } from "@earendil-works/pi-telemetry";
import { ConversationBusy, ConversationIdle, ReadAfterWrite } from "../errors.js";
import { startRun } from "./generation.js";
-import { appendInput, applyBoundary, hasQueuedReset, InboxDoc, isDisplayOnly, isStale, pickInputEntry, prepareBoundary, removeInboxItem, } from "./inbox.js";
+import { appendInput, inputDraft, applyBoundary, hasQueuedReset, InboxDoc, isDisplayOnly, isStale, pickInputEntry, prepareBoundary, removeInboxItem, } from "./inbox.js";
import { LiveDoc } from "./live.js";
import { DURABLE_TELEMETRY_SCHEMA } from "./telemetry.js";
import { closedError, Waiters } from "./util.js";
@@ -205,7 +205,7 @@ export async function admitSubmission(tx, conversationId, draft, now, queueModes
// Never queued: whether a run can continue depends on how the current one ends. Rejected before any write.
if (busy)
throw new ConversationBusy(conversationId);
- const boundary = await prepareBoundary(tx, conversationId, queueModes);
+ const boundary = await prepareBoundary(tx, conversationId, queueModes, "final", now);
const { id } = await tx.createSubmission({ ...fields, type: "continue", status: "placed" });
// Items an earlier failed run left queued (`afterFailedRun: "keep"`) are placed first and join the run. Its history
// starts with the failed run's, so `afterRun` hooks see what that run did.
@@ -214,7 +214,7 @@ export async function admitSubmission(tx, conversationId, draft, now, queueModes
return { id, status: "placed" };
}
// A boundary reads the table, so it is prepared before the first table write; a busy one needs none.
- const boundary = busy ? undefined : await prepareBoundary(tx, conversationId, queueModes);
+ const boundary = busy ? undefined : await prepareBoundary(tx, conversationId, queueModes, "final", now, draft.type === "input" ? inputDraft(draft.content, draft.entry, now) : undefined);
if (boundary === undefined || boundary.inbox.items.length > 0) {
const { id } = await tx.createSubmission({
...fields,
@@ -253,6 +253,8 @@ export async function admitSubmission(tx, conversationId, draft, now, queueModes
const write = { ...fields, type: "write", status: "done", entry: entry.id };
return { id: (await tx.createSubmission(write)).id, status: "done" };
}
+ for (const before of boundary.entries)
+ await tx.appendEntry(conversationId, before);
const entry = await appendInput(tx, conversationId, draft.content, draft.entry, now);
const input = { ...fields, type: "input", status: "placed", entry: entry.id };
const { id } = await tx.createSubmission(input);
diff --git a/dist/harness/types.d.ts b/dist/harness/types.d.ts
index eb64e474162b158a43563e6c68d8ffea8c18b74a..cce695c6d87c79dadea54560bafcca62cf073ca1 100644
--- a/dist/harness/types.d.ts
+++ b/dist/harness/types.d.ts
@@ -816,6 +816,9 @@ export interface Harness extends Session {
* never do.
*/
resume(): void;
+ /** Start no task run until `release()`, whatever `resume()` says; running invocations and abort handlers go on. */
+ hold(): void;
+ release(): void;
/**
* When this Session next needs a running process, from the scheduler's memory: `active` while some task can run now
* or is running, `at` for the earliest sleep. Answers also while scheduling is paused. A host that wakes the
@@ -888,6 +891,8 @@ export interface HookApi extends DocumentReader {
export type HookResult<T> = T | undefined | Promise<T | undefined>;
/** Hooks of the built-in generation task. */
export interface GenerationHooks {
+ /** Before a new run’s first input is placed. Use only tx for storage access; effects commit with the input. */
+ beforeInput(input: EntryDraft, api: { readonly conversationId: ConversationId; readonly tx: Tx }, context: Context): HookResult<{ readonly entries?: readonly EntryDraft[] }>;
/** Before every request attempt, including recovery; the result is used for that request only. */
beforeRequest(request: {
readonly messages: readonly Message[];
diff --git a/dist/session/session.d.ts b/dist/session/session.d.ts
index 8201d7cfefb710516a3d4d89ba208adc2cbebd19..37c9f362401b5bc81f2f24233aa3dd7f57e29204 100644
--- a/dist/session/session.d.ts
+++ b/dist/session/session.d.ts
@@ -1,5 +1,5 @@
import { type Context, type JsonValue, type WatchEnd } from "@earendil-works/chord";
-import type { CommitChange, CommitObservation, CommitObserver, CommitOptions, ConversationDocFamilyToken, ConversationDocToken, ConversationId, ConversationRecord, DocumentCommitChange, DocumentRecord, DocumentWatch, EntryId, JsonObject, Observation, Observer, ReportInfo, RewindableConversationDocFamilyToken, RewindableConversationDocToken, Seq, Session, SessionDocFamilyToken, SessionDocToken, SessionEnd, SetTimer, Storage, StorageWrite, TaskDocFamilyToken, TaskDocToken, TaskId, Tx } from "../types.ts";
+import type { CommitChange, CommitObservation, CommitObserver, CommitOptions, ConversationDocFamilyToken, ConversationDocToken, EntryDraft, ConversationId, ConversationRecord, DocumentCommitChange, DocumentRecord, DocumentWatch, EntryId, JsonObject, Observation, Observer, ReportInfo, RewindableConversationDocFamilyToken, RewindableConversationDocToken, Seq, Session, SessionDocFamilyToken, SessionDocToken, SessionEnd, SetTimer, Storage, StorageWrite, TaskDocFamilyToken, TaskDocToken, TaskId, Tx } from "../types.ts";
import { type HeadMarker } from "./conversation-tails.ts";
import { Transaction, type TransactionScope } from "./transaction.ts";
/**
@@ -171,6 +171,7 @@ export declare class SessionImpl implements Session {
* listener. A plain Session drops them.
*/
report(_error: unknown, _info?: ReportInfo, _context?: Context): void;
+ protected beforeInput(tx: Transaction, id: ConversationId, input: EntryDraft, context: Context): Promise<readonly EntryDraft[]>;
/**
* Runs inside every transaction that creates or forks a conversation, after the conversation record is staged. A
* plain Session stages nothing; a Harness stages its built-in documents.
diff --git a/dist/session/session.js b/dist/session/session.js
index e9ae9ebfc2c45765ccda141d7e60cdc9a15a0894..e1e5c69deed9f0d09cf67c26cdb903fe4593f9c6 100644
--- a/dist/session/session.js
+++ b/dist/session/session.js
@@ -224,6 +224,7 @@ export class SessionImpl {
if (this.#documents.get(id)?.record.id === recordId)
this.#dropDocument(id);
},
+ beforeInput: (tx, id, input, context) => this.beforeInput(tx, id, input, context),
conversationCreated: (tx, record) => this.conversationCreated(tx, record),
taskDocuments: (taskId) => this.#taskDocuments.get(taskId)?.values(),
latestEntryId: (conversationId, context) => this.latestEntryId(conversationId, context),
@@ -570,6 +571,9 @@ export class SessionImpl {
* listener. A plain Session drops them.
*/
report(_error, _info = LISTENER, _context) { }
+ async beforeInput(_tx, _id, _input, _context) {
+ return [];
+ }
/**
* Runs inside every transaction that creates or forks a conversation, after the conversation record is staged. A
* plain Session stages nothing; a Harness stages its built-in documents.
diff --git a/dist/session/transaction.d.ts b/dist/session/transaction.d.ts
index 5508fa2d021d2153f4f104b4ef69b99ef98a6a74..1c94a4a6e29b8371d49f7d6cb4d2e4e7cd6aec50 100644
--- a/dist/session/transaction.d.ts
+++ b/dist/session/transaction.d.ts
@@ -48,6 +48,7 @@ export interface TransactionHost {
install(document: LoadedDocument): void;
/** Remove a retired incarnation if it is still the cached occupant of its address. */
evict(addressId: string, recordId: DocumentId): void;
+ beforeInput(tx: Transaction, conversationId: ConversationId, input: EntryDraft, context: Context): Promise<readonly EntryDraft[]>;
/** Stage writes that belong to every newly created or forked conversation, in its creating transaction. */
conversationCreated(tx: Transaction, record: ConversationRecord): Promise<void>;
/** The newest entry ID visible from a conversation, known from the Session's commits. */
@@ -112,6 +113,7 @@ export declare class Transaction implements Tx {
forkConversation(parentConversationId: ConversationId, at: EntryId, options: {
readonly ownership: ConversationOwnership;
}): Promise<ConversationRecord>;
+ beforeInput(conversationId: ConversationId, input: EntryDraft): Promise<readonly EntryDraft[]>;
appendEntry(conversationId: ConversationId, value: EntryDraft): Promise<EntryRecord>;
appendEntry<D extends JsonValue>(token: Entry<D>, conversationId: ConversationId, value: TypedEntryDraft<NoInfer<D>>): Promise<TypedEntry<D>>;
createTask<I, S extends {
diff --git a/dist/session/transaction.js b/dist/session/transaction.js
index 7fbe75b11bb2a9ab28f4f23800143231172d71b7..a516853e2f9c4a4437dbc2bb8f1c8d2117386e9a 100644
--- a/dist/session/transaction.js
+++ b/dist/session/transaction.js
@@ -167,6 +167,9 @@ export class Transaction {
this.#assertOpen();
return record;
}
+ beforeInput(conversationId, input) {
+ return this.#read("beforeInput", () => this.#host.beforeInput(this, conversationId, input, this.#context));
+ }
appendEntry(first, second, third) {
if (typeof first === "number")
return this.#appendEntry(first, second);
diff --git a/dist/types.d.ts b/dist/types.d.ts
index b43143c5e7998b36034ffe9577f74f3d88ab932b..a9ca10ae41c48ad0ebb813e62caa3699dec5c51e 100644
--- a/dist/types.d.ts
+++ b/dist/types.d.ts
@@ -409,6 +409,8 @@ export type ContextEdit = {
});
/** Immutable transcript event with separate model-facing and application-facing payloads. */
export type EntryRecord = {
+ /** Placed by the harness from an input submission, not a write. */
+ readonly input?: true;
readonly id: EntryId;
readonly conversationId: ConversationId;
/** Application-defined entry discriminator. */
@@ -891,6 +893,8 @@ export type CommitPublication = {
* Session may freeze them (its caches and context ranges keep them), so mutating one can throw.
*/
export interface Tx {
+ /** Prepare entries before a new run’s first input; called before table writes, in the placement transaction. */
+ beforeInput(conversationId: ConversationId, input: EntryDraft): Promise<readonly EntryDraft[]>;
conversation(id: ConversationId): Promise<ConversationRecord | undefined>;
entry(id: EntryId): Promise<EntryRecord | undefined>;
/** Undefined when the entry is absent or has another kind. */
+58 -11
View File
@@ -8940,6 +8940,9 @@ var Transaction = class {
this.#assertOpen();
return record;
}
beforeInput(conversationId, input) {
return this.#read("beforeInput", () => this.#host.beforeInput(this, conversationId, input, this.#context));
}
appendEntry(first, second, third) {
if (typeof first === "number")
return this.#appendEntry(first, second);
@@ -10122,6 +10125,7 @@ var SessionImpl = class {
if (this.#documents.get(id)?.record.id === recordId)
this.#dropDocument(id);
},
beforeInput: (tx, id, input, context) => this.beforeInput(tx, id, input, context),
conversationCreated: (tx, record) => this.conversationCreated(tx, record),
taskDocuments: (taskId) => this.#taskDocuments.get(taskId)?.values(),
latestEntryId: (conversationId, context) => this.latestEntryId(conversationId, context),
@@ -10446,6 +10450,9 @@ var SessionImpl = class {
*/
report(_error, _info = LISTENER, _context) {
}
async beforeInput(_tx, _id, _input, _context) {
return [];
}
/**
* Runs inside every transaction that creates or forks a conversation, after the conversation record is staged. A
* plain Session stages nothing; a Harness stages its built-in documents.
@@ -12250,10 +12257,14 @@ var InboxDoc = defineDoc({
initial: () => ({ items: [] }),
checkpointWhen: (value2) => value2.items.length === 0
});
async function prepareBoundary(tx, conversationId, modes) {
async function prepareBoundary(tx, conversationId, modes, at, now, incoming) {
const head = (await tx.latestHeadMarker(conversationId))?.head;
const inbox = await tx.doc(InboxDoc, conversationId);
const first = inbox.items.find((item) => item.mode === "steer" || item.mode === "followUp");
const input = first === void 0 ? incoming : inputDraft(first.content, first.entry, now);
const entries2 = input !== void 0 && (at === "final" || hasQueuedReset(inbox.items)) ? await tx.beforeInput(conversationId, input) : [];
return {
entries: entries2,
conversationId,
inbox,
steeringMode: modes.steeringMode,
@@ -12289,6 +12300,9 @@ async function applyBoundary(tx, boundary, at, now) {
}
const placed = [];
for (const index of users) {
if (placed.length === 0)
for (const draft of boundary.entries)
await tx.appendEntry(conversationId, draft);
const item = items[index];
const entry2 = await appendInput(tx, conversationId, item.content, item.entry, now);
tx.placeSubmission(item.id, entry2.id);
@@ -12301,10 +12315,11 @@ async function applyBoundary(tx, boundary, at, now) {
return { users: placed, first, reset };
}
function appendInput(tx, conversationId, content, entry2, now) {
return tx.appendEntry(conversationId, inputDraft(content, entry2, now));
}
function inputDraft(content, entry2, now) {
const message = { role: "user", content, timestamp: now };
if (entry2 === void 0)
return tx.appendEntry(UserEntry, conversationId, { model: [message] });
return tx.appendEntry(conversationId, { ...pickInputEntry(entry2), model: [message] });
return { ...entry2 === void 0 ? { kind: UserEntry.kind } : pickInputEntry(entry2), input: true, model: [message] };
}
function pickInputEntry({ kind, data }) {
const picked = data === void 0 ? { kind } : { kind, data };
@@ -13016,6 +13031,7 @@ var TaskScheduler = class {
#unsubscribeRegistry = () => {
};
#enabled = false;
#held = false;
#readOnly;
#closing = false;
#dirty = false;
@@ -13044,6 +13060,14 @@ var TaskScheduler = class {
this.#context = options.context;
this.#readOnly = options.readOnly === true;
}
/** Start no run invocation until `release()`; running invocations and abort handlers go on. Overrides `resume()`. */
hold() {
this.#held = true;
}
release() {
this.#held = false;
this.#kick();
}
/**
* Load live tasks and change surviving `running` tasks back to `pending`, counting the restart. Dispatches nothing;
* a task cut off `settings.restartLimit` times ends when it is next reserved. Read-only, it only loads them: a
@@ -14498,6 +14522,8 @@ var TaskScheduler = class {
for (const record of plan.ready) {
const id = record.id;
const mode = record.abortRequested ? "abort" : "run";
if (mode === "run" && this.#held)
continue;
const cutOff = record.restarts !== void 0 && record.restarts >= this.#restartLimit();
if (cutOff && (mode === "abort" || !this.#limitsRestarts(record.kind))) {
const message = mode === "abort" ? `Abort handler of task ${id} (${record.kind}) was cut off ${record.restarts} times` : `Task ${id} (${record.kind}) was cut off ${record.restarts} times without progress`;
@@ -14683,7 +14709,7 @@ var TaskScheduler = class {
const tasks = [];
for (const record of this.#live.values())
tasks.push({ record, state: this.#inspectTask(record, snapshot) });
const scheduling = this.#closing ? "closing" : this.#enabled ? "running" : "paused";
const scheduling = this.#closing ? "closing" : this.#enabled && !this.#held ? "running" : "paused";
return { scheduling, tasks };
}
#inspectTask(record, snapshot) {
@@ -14838,6 +14864,10 @@ var TaskScheduler = class {
* invocation continues.
*/
#progress(tx, current, state, extension, context) {
if (this.#held) {
tx.setTask(withState(current, { status: "pending", checkpoint: current.state.checkpoint }));
return false;
}
state.snapshot = this.#registry.snapshot();
const next = state.snapshot.task(current.kind);
if (next === state.task)
@@ -17571,7 +17601,7 @@ async function prepareFailure(tx, runtime) {
const policy = settings.afterFailedRun;
if (policy !== "continue")
return { policy };
return { policy, boundary: await prepareBoundary(tx, runtime.conversationId, settings) };
return { policy, boundary: await prepareBoundary(tx, runtime.conversationId, settings, "final", runtime.now()) };
}
async function endFailedRun(tx, runtime, live, failure, end) {
const conversationId = runtime.conversationId;
@@ -17739,7 +17769,7 @@ async function answer(runtime, message, context) {
const afterRun = await asksAfterRun(runtime, context);
const conversationId = runtime.conversationId;
await runtime.commit(async (tx) => {
const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
const boundary = await prepareBoundary(tx, conversationId, runtime.settings, "final", runtime.now());
const live = await tx.doc(LiveDoc, conversationId);
const entry2 = await appendAssistant(tx, conversationId, message);
const result2 = { status: "terminal", outcome: { status: "completed", result: { entryId: entry2.id } } };
@@ -17812,7 +17842,7 @@ function prepareRunEndTasks(runtime, requests, prepared) {
async function endAnsweredRun(runtime, answer2, context) {
const conversationId = runtime.conversationId;
await runtime.commit(async (tx) => {
const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
const boundary = await prepareBoundary(tx, conversationId, runtime.settings, "final", runtime.now());
const live = await tx.doc(LiveDoc, conversationId);
const { users, first } = await applyBoundary(tx, boundary, "final", runtime.now());
endRun(tx, live, runtime.taskId, { status: "done", answer: answer2 });
@@ -17885,7 +17915,7 @@ async function finishToolRound(runtime, assistant, tools, context) {
const maxTurns = (await runtime.agent(context)).maxTurns;
const afterRun = terminate && await asksAfterRun(runtime, context);
await runtime.commit(async (tx) => {
const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
const boundary = await prepareBoundary(tx, conversationId, runtime.settings, terminate || handoff !== void 0 ? "final" : "postTools", runtime.now());
if (added.length > 0)
await addTools(tx, conversationId, added);
const live = await tx.doc(LiveDoc, conversationId);
@@ -18301,13 +18331,13 @@ async function admitSubmission(tx, conversationId, draft, now, queueModes, span)
if (draft.type === "continue") {
if (busy)
throw new ConversationBusy(conversationId);
const boundary2 = await prepareBoundary(tx, conversationId, queueModes);
const boundary2 = await prepareBoundary(tx, conversationId, queueModes, "final", now);
const { id: id2 } = await tx.createSubmission({ ...fields, type: "continue", status: "placed" });
const { users, first } = await applyBoundary(tx, boundary2, "final", now);
await startRun(tx, conversationId, live, { inputs: [...users, id2], first: live.runFailure?.first ?? first });
return { id: id2, status: "placed" };
}
const boundary = busy ? void 0 : await prepareBoundary(tx, conversationId, queueModes);
const boundary = busy ? void 0 : await prepareBoundary(tx, conversationId, queueModes, "final", now, draft.type === "input" ? inputDraft(draft.content, draft.entry, now) : void 0);
if (boundary === void 0 || boundary.inbox.items.length > 0) {
const { id: id2 } = await tx.createSubmission({
...fields,
@@ -18345,6 +18375,8 @@ async function admitSubmission(tx, conversationId, draft, now, queueModes, span)
const write = { ...fields, type: "write", status: "done", entry: entry3.id };
return { id: (await tx.createSubmission(write)).id, status: "done" };
}
for (const before of boundary.entries)
await tx.appendEntry(conversationId, before);
const entry2 = await appendInput(tx, conversationId, draft.content, draft.entry, now);
const input = { ...fields, type: "input", status: "placed", entry: entry2.id };
const { id } = await tx.createSubmission(input);
@@ -19360,6 +19392,15 @@ var HarnessImpl = class extends SessionImpl {
create: (target, createOptions, context2) => this.#create(target, createOptions, context2)
};
}
async beforeInput(tx, conversationId, input, context) {
const agent = resolveAgent(await tx.doc(AgentDoc, conversationId), this.#options.registry.snapshot(), resolveSettings(this.#hostSettings()), (error, extension) => this.#report(error, { kind: "extension_error", conversationId, extension }, context));
const entries2 = [];
for (const { handlers } of agentHooks(agent, "pi.generation")) {
const result2 = await handlers.beforeInput?.(input, { conversationId, tx }, context);
entries2.push(...result2?.entries ?? []);
}
return entries2;
}
/** Resolve a conversation's committed `pi.agent` against `snapshot`, or the current one, and the current settings. */
async resolveAgent(id, snapshot, context) {
const registry2 = snapshot ?? this.#options.registry.snapshot();
@@ -19382,6 +19423,12 @@ var HarnessImpl = class extends SessionImpl {
this.assertUsable();
this.#tasks.resume();
}
hold() {
this.#tasks.hold();
}
release() {
this.#tasks.release();
}
wake() {
this.assertUsable();
return this.#tasks.wake();
+1 -6
View File
@@ -125,12 +125,7 @@ pub async fn run_with_userland(
tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()?
.block_on(crate::harness::run_module_with_credentials(
world,
Arc::new(runtime),
cred,
include_str!("../../js/bundle.js"),
))
.block_on(crate::harness::run_console(world, Arc::new(runtime), cred))
})
.await
.map_err(anyhow::Error::from)
-13
View File
@@ -129,19 +129,6 @@ pub async fn op_exec_next(
Ok(value)
}
#[op2]
#[string]
pub async fn op_input() -> Result<Option<String>, JsErrorBox> {
use tokio::io::AsyncBufReadExt;
let mut reader = tokio::io::BufReader::with_capacity(1, tokio::io::stdin());
let mut line = String::new();
let n = reader
.read_line(&mut line)
.await
.map_err(|e| JsErrorBox::generic(e.to_string()))?;
Ok((n != 0).then_some(line))
}
#[op2(fast)]
pub fn op_output(#[string] text: &str) {
print!("{text}");
+74
View File
@@ -0,0 +1,74 @@
//! Channel input for a headless harness. The console and stop signals are clients.
use deno_core::{OpState, op2};
use deno_error::JsErrorBox;
use serde::Serialize;
use serde_json::Value;
use std::{cell::RefCell, rc::Rc};
use tokio::sync::mpsc;
#[derive(Serialize)]
#[serde(tag = "type", rename_all = "camelCase")]
pub enum Input {
Submit {
conversation: u64,
content: Value,
#[serde(rename = "requestId")]
request_id: String,
},
Stop,
}
pub type Sender = mpsc::UnboundedSender<Input>;
pub type Receiver = mpsc::UnboundedReceiver<Input>;
pub fn channel() -> (Sender, Receiver) {
mpsc::unbounded_channel()
}
pub(super) struct Inbox(pub tokio::sync::Mutex<Receiver>);
#[op2]
#[serde]
pub async fn op_input(state: Rc<RefCell<OpState>>) -> Result<serde_json::Value, JsErrorBox> {
let inbox = state.borrow().borrow::<Rc<Inbox>>().clone();
serde_json::to_value(inbox.0.lock().await.recv().await)
.map_err(|error| JsErrorBox::generic(error.to_string()))
}
pub(super) fn console(sender: Sender) {
std::thread::spawn(move || {
use std::io::BufRead;
for line in std::io::stdin().lock().lines() {
let Ok(line) = line else { break };
if line.trim().is_empty() {
continue;
}
if sender
.send(Input::Submit {
conversation: 1,
content: Value::String(line),
request_id: uuid::Uuid::new_v4().to_string(),
})
.is_err()
{
break;
}
}
// EOF closes only this client. The signal client keeps the harness alive.
});
}
pub(super) fn stop_signal(
sender: Sender,
) -> std::io::Result<impl std::future::Future<Output = ()> + Send> {
// Register before V8 starts, so a stop cannot arrive before the listener.
let mut terminate = tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())?;
let mut interrupt = tokio::signal::unix::signal(tokio::signal::unix::SignalKind::interrupt())?;
Ok(async move {
tokio::select! {
_ = interrupt.recv() => {}
_ = terminate.recv() => {}
}
let _ = sender.send(Input::Stop);
})
}
+41 -7
View File
@@ -1,6 +1,7 @@
//! Durable V8 harness, host-side SQLite storage, and world tool operations.
mod credentials;
mod environment;
pub mod input;
use crate::{exec::Exec, world::World};
use deno_core::{JsRuntime, OpState, RuntimeOptions, op2};
use deno_error::JsErrorBox;
@@ -85,7 +86,7 @@ deno_core::extension!(
environment::op_file,
environment::op_exec,
environment::op_exec_next,
environment::op_input,
input::op_input,
environment::op_output
]
);
@@ -105,12 +106,7 @@ async fn run_with_shared_credentials(
tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()?
.block_on(run_module_with_credentials(
world,
exec,
cred,
include_str!("../../js/bundle.js"),
))
.block_on(run_console(world, exec, cred))
})
.await?
.map_err(|error: anyhow::Error| {
@@ -139,6 +135,19 @@ pub async fn run_module_with_credentials(
exec: Arc<dyn Exec>,
cred: Arc<crate::cred::store::Cred>,
source: &str,
) -> anyhow::Result<()> {
let (_, receiver) = input::channel();
run_module_with_input(world, exec, cred, receiver, source).await
}
/// Run without the console. Submissions and stop requests arrive through the channel.
/// The caller must use a current-thread Tokio runtime.
pub async fn run_module_with_input(
world: World,
exec: Arc<dyn Exec>,
cred: Arc<crate::cred::store::Cred>,
receiver: input::Receiver,
source: &str,
) -> anyhow::Result<()> {
let proxy = credentials::proxy(&cred)?;
let mut runtime = JsRuntime::new(RuntimeOptions {
@@ -196,6 +205,12 @@ pub async fn run_module_with_credentials(
.borrow_mut()
.put(credentials::Notifications::new(&cred));
runtime.op_state().borrow_mut().put(cred);
runtime
.op_state()
.borrow_mut()
.put(std::rc::Rc::new(input::Inbox(tokio::sync::Mutex::new(
receiver,
))));
runtime
.op_state()
.borrow_mut()
@@ -237,6 +252,25 @@ pub async fn run_module_with_credentials(
})
}
/// Run the headless harness with console input and signal clients.
/// The caller must use a current-thread Tokio runtime.
pub async fn run_console(
world: World,
exec: Arc<dyn Exec>,
cred: Arc<crate::cred::store::Cred>,
) -> anyhow::Result<()> {
let (sender, receiver) = input::channel();
input::console(sender.clone());
let signals = tokio::spawn(input::stop_signal(sender)?);
let source = concat!(
"globalThis.consoleEnabled = true;\n",
include_str!("../../js/bundle.js")
);
let result = run_module_with_input(world, exec, cred, receiver, source).await;
signals.abort();
result
}
/// Open a world and resume its durable root conversation.
pub async fn run(path: &Path) -> anyhow::Result<()> {
run_with_userland(path, None).await
+172
View File
@@ -0,0 +1,172 @@
//! Patched pi-durable input placement, hold boundaries, and headless channel lifecycle.
use sausage::{
cred::Cred,
harness::{
input::{self, Input},
run_module, run_module_with_input,
},
world::World,
};
use std::{
process::{Command, Stdio},
sync::Arc,
time::{Duration, Instant},
};
const BUNDLE: &str = include_str!("../js/foundation-test-bundle.js");
#[test]
fn bundle_has_patched_hooks() {
for bundle in [BUNDLE, include_str!("../js/bundle.js")] {
assert!(bundle.contains("handlers.beforeInput"));
assert!(bundle.contains("input: true"));
assert!(bundle.contains("#held"));
}
assert!(!include_str!("../js/patches/pi-durable-1.5.0.patch").contains("dist/env/node"));
}
async fn probe(case: &str) -> anyhow::Result<()> {
let dir = tempfile::tempdir()?;
let world = World::open(dir.path())?;
let exec = sausage::process::exec(World::open(dir.path())?)?;
run_module(
world,
exec,
&format!("globalThis.foundationCase = {case:?};\n{BUNDLE}"),
)
.await
}
#[tokio::test(flavor = "current_thread")]
async fn input_placement() -> anyhow::Result<()> {
probe("placement").await
}
#[tokio::test(flavor = "current_thread")]
async fn followups_start_one_run() -> anyhow::Result<()> {
probe("followups").await
}
#[tokio::test(flavor = "current_thread")]
async fn response_thinking_level() -> anyhow::Result<()> {
probe("thinking").await
}
#[tokio::test(flavor = "current_thread")]
async fn hold_parks_at_phase_boundary() -> anyhow::Result<()> {
probe("hold").await
}
#[tokio::test(flavor = "current_thread")]
async fn headless_child() -> anyhow::Result<()> {
if std::env::var_os("HEADLESS_CHILD").is_none() {
return Ok(());
}
let dir = tempfile::tempdir()?;
let world = World::open(dir.path())?;
let cred = Arc::new(Cred::open(&world.host_dir)?);
let exec = sausage::process::exec(World::open(dir.path())?)?;
let (sender, receiver) = input::channel();
sender.send(Input::Submit {
conversation: 1,
content: serde_json::json!("channel input"),
request_id: "channel-request".into(),
})?;
sender.send(Input::Stop)?;
tokio::time::timeout(
Duration::from_secs(30),
run_module_with_input(
world,
exec,
cred,
receiver,
&format!("globalThis.foundationCase = 'headless';\n{BUNDLE}"),
),
)
.await??;
Ok(())
}
#[test]
fn channel_with_closed_stdin() -> anyhow::Result<()> {
let mut child = Command::new(std::env::current_exe()?)
.args(["--exact", "headless_child", "--nocapture"])
.env("HEADLESS_CHILD", "1")
.stdin(Stdio::null())
.spawn()?;
let deadline = Instant::now() + Duration::from_secs(40);
loop {
if let Some(status) = child.try_wait()? {
assert!(status.success());
break;
}
if Instant::now() > deadline {
child.kill()?;
child.wait()?;
anyhow::bail!("Headless channel did not stop");
}
std::thread::yield_now();
}
Ok(())
}
#[test]
fn cli_stays_up_after_stdin_eof() -> anyhow::Result<()> {
let dir = tempfile::tempdir()?;
let mut child = Command::new(env!("CARGO_BIN_EXE_sausage"))
.arg("run")
.arg(dir.path())
.args(["--api-listen", "127.0.0.1:0"])
.env("PI_PROXY_BASE_URL", "https://proxy.invalid")
.env("PI_PROXY_API_KEY", "faux-token")
.env_remove("SAUSAGE_USERLAND")
.stdin(Stdio::null())
.spawn()?;
let deadline = Instant::now() + Duration::from_secs(30);
loop {
if let Some(status) = child.try_wait()? {
anyhow::bail!("CLI exited at EOF: {status}");
}
if let Ok(db) = rusqlite::Connection::open(dir.path().join("host/harness.sqlite"))
&& db
.query_row("SELECT count(*) FROM conversations", [], |r| {
r.get::<_, i64>(0)
})
.unwrap_or(0)
> 0
{
break;
}
if Instant::now() > deadline {
child.kill()?;
child.wait()?;
anyhow::bail!("CLI did not initialize");
}
std::thread::yield_now();
}
// EOF shutdown previously took one credential tick. Check beyond that tick.
let alive_until = Instant::now() + Duration::from_secs(2);
while Instant::now() < alive_until {
if let Some(status) = child.try_wait()? {
anyhow::bail!("CLI exited after EOF: {status}");
}
std::thread::yield_now();
}
assert!(
Command::new("kill")
.args(["-TERM", &child.id().to_string()])
.status()?
.success()
);
let deadline = Instant::now() + Duration::from_secs(10);
loop {
if let Some(status) = child.try_wait()? {
assert!(status.success());
break;
}
if Instant::now() > deadline {
child.kill()?;
child.wait()?;
anyhow::bail!("CLI did not stop cleanly");
}
std::thread::yield_now();
}
Ok(())
}
+23
View File
@@ -0,0 +1,23 @@
# World skills
The built-in skills live in `/src/sausage/skills/`. Repo skills live in `<repo>/.agents/skills/`.
Each skill has `name` and `description` frontmatter. The prompt lists its path. An agent reads the full file on demand.
| Ultimate source | Sausage skill | Purpose |
| --- | --- | --- |
| how-ultimate-works | how-sausage-works | Explain branches, decisions, memory, and explicit repos without claims about a browser UI. |
| integrate-ultimate | integrate-sausage | Investigate first and adapt selected repo agents, skills, CLIs, or servers. |
| adopt-ultimate-setup | adopt-sausage-workflow | Tour approved families and the team, then adapt only selected pieces. |
| write-extension | write-extension | Change durable coordination behavior through installed contracts and standard WASI operations. |
The port omits `ui-preview-round` and `verify-ui`.
Those skills require browser capture tools and the Ultimate preview host. P3 supplies neither.
Designer and coordinator instructions still require actual images before visual claims or required visual approval.
Without images, they report visual checks as unverified. Automated tests do not replace approval.
The port removes storage chapter changes and subscription login instructions.
Credentials remain in the host. A 403 requests host approval.
A commit to `/src/sausage` on `main` reloads the complete committed coordination snapshot.
The current task supplies files and pure parser tests, not that runtime loader.
Run `nix develop -c node extensions/core/agents.test.ts` from the repository root to validate all skill frontmatter and bodies.
+37
View File
@@ -0,0 +1,37 @@
---
name: adopt-sausage-workflow
description: Tour the world model families and review team, then adapt only the user’s selected pieces.
---
# Adopt selected workflow pieces
Read /src/sausage/agents/models.json and all /src/sausage/agents/*.md files before the tour.
Read the explicit project repo’s instructions and check commands.
Explain these pieces in one or two lines each:
- Model families select a model and thinking level for each role.
- The review lead combines independent correctness, simplicity, and advisory architecture reports.
- The coordinator overlay selects the lead family without copying its instructions.
- The lander checks the combined result. A push needs the owner’s authorization.
- A fresh worktree needs only the project’s required non-secret files and build steps.
Ask the user which pieces to adopt. Include None. Do not write files before the answer.
The world family file supplies default and fast through openai/gpt-6.1-sol at medium.
It supplies review through anthropic/claude-opus-5-5 at xhigh.
It supplies research, design, and lead through anthropic/claude-opus-5-5 at high.
Use only models configured in the world. Never substitute another provider to bypass approval.
The default review team uses the review family. It trades model diversity for the approved model policy.
Write selected repo agent files in <repo>/.sausage/agents/. Keep unrelated existing files intact.
An overlay needs name and addTools, even when addTools is empty. It must not contain tools.
Reviewer definitions keep tools: [read, bash], the immutable range rule, and the report format.
Adapt correctness risks and architecture references to the actual repo. Keep simplicity focused on minimum necessary code.
Never add a replacement reviewer lead.
If the user selects lander or worktree behavior, inspect the installed workspace extension’s contract before configuration.
Do not invent configuration keys. Prove required commands in a disposable worktree with an explicit repo path.
Show the exact diff and checks before an authorized commit. Never push without authorization.
The host injects credentials. A 403 requests host approval. Do not copy secrets into worktrees or configuration.
+25
View File
@@ -0,0 +1,25 @@
---
name: how-sausage-works
description: Explain the Sausage workflow in short steps when the user requests a walkthrough.
---
# Explain the workflow
Use five short beats. Ask one question through message after each beat, and wait for its answer.
Offer Skip the rest at every beat. If the user skips, continue to the final question.
Do not claim a particular UI exists. Use the conversation and tools that the prompt supplies.
1. Explain that @ultimate is the standing agent. The user starts work there and can steer branches directly.
2. Explain the flow: user → @ultimate → coordinator → scouts, workers, designers, and reviewer → submission and configured checks.
Features need an assessment before code unless the user explicitly waived it. Pushes require authorization.
3. Ask how much control the user wants. They decide most questions, only major questions, or authorize a hands-off goal.
During an active goal, @ultimate records decisions through goal. Host credential approvals still require the user.
4. Explain memory and context. Branches start from durable memory and explicit briefs. Compact preserves a handoff.
Dreams review evidence and propose improvements. They do not switch storage chapters.
5. Ask what the project does, its explicit repo path, and the first task.
Summarize the project, control preference, and task. Use remember only for durable facts and the user’s instructions.
Read integrate-sausage before proposals for specialization. Start the first authorized task through branch with its repo path.
The world contains /bin programs. Agents write CLIs and servers inside one WASI sandbox.
The host injects credentials. A 403 requests host approval. Secrets never enter the world.
+39
View File
@@ -0,0 +1,39 @@
---
name: integrate-sausage
description: Investigate an explicit repository and propose small Sausage specializations before changes.
---
# Specialize a repository
## Investigate without changes
Require an explicit repo path. Read its AGENTS.md, manifests, lockfiles, scripts, tests, recent Git history, and architecture documents.
Identify its purpose, language versions, domain objects, checks, deployment process, and repeated manual work.
Read existing agents and skills before proposals. Inspect /src/sausage/extensions/ for reusable coordination contracts.
Name credential requirements without access to secret values. A 403 requests approval from the host.
Identify ignored files and generated assets that a fresh worktree needs. Worktrees live at /worktrees/<id>.
Do not copy production data or credentials. Ask before shared database changes or irreversible operations.
## Propose before implementation
Offer two to four ranked changes. State the user outcome, existing code to reuse, required programs, costs, and approvals.
Prefer a skill for repeated judgment, an agent for a specialist role, and a CLI or server for deterministic work.
Use a coordination extension only for harness behavior, not as a new concept for world programs.
Place repo definitions in <repo>/.sausage/agents/ and repo skills in <repo>/.agents/skills/.
An addTools overlay extends an existing agent. A complete definition replaces it by name.
Never replace the review lead merely to add reviewers. Add reviewer-* definitions instead.
Read adopt-sausage-workflow for the default team and family choices.
Ask the user which changes to build. Do not create configuration before that choice.
Pushes, paid operations, database changes, and credential approvals need explicit authority.
## Build the selected changes
Write selected agent files and skills directly. Delegate executable behavior to a coordinator through branch with the explicit repo path.
Use existing build and test commands from the repo. Do not introduce a package manager or workflow without evidence.
Prove required preparation in a disposable worktree with authorized non-secret inputs.
Use /bin programs through bash. Preserve stdout and stderr separately when you capture output.
Report exact checks, chosen paths, required approvals, and anything you cannot verify.
Coordination code and built-in assets live in /src/sausage. A commit to its main branch reloads one complete snapshot.
Never add host imports beyond the process interface. Files, HTTP, and sockets use standard WASI interfaces.
+32
View File
@@ -0,0 +1,32 @@
---
name: write-extension
description: Change a Sausage coordination extension through its documented durable contracts, not new host imports.
---
# Change coordination behavior
Use this skill only for harness coordination behavior. Agents otherwise write CLIs and servers.
Coordination extensions live in /src/sausage/extensions/ and load from its main branch.
A commit reloads one complete snapshot. A failed snapshot leaves the last good commit active.
Do not treat uncommitted edits as proof that the live host uses new code.
Before changes, read /src/sausage/AGENTS.md, the nearest extension README, and its API contracts.
Read the installed loader contract before imports or lifecycle changes. If a contract is absent, report the blocker.
Copy the closest existing extension’s shape. Do not invent host APIs or use Ultimate’s browser facets.
Keep deterministic world operations in CLIs or servers. Use /bin programs through the documented process interface.
Use standard WASI files, HTTP, and sockets. Never add an extra host import.
Host code accesses world files only through the VFS. Secrets remain in the host credential manager.
A 403 requests host approval. Do not read secret values or inject them into script environments.
For coordination behavior, reuse existing typed services, durable documents, tasks, events, and reactors.
Keep related durable changes in one transaction. Use registered task definitions for work that must survive interruption.
Check the installed disposal contract before resources or long-running work. Clean resources through that contract.
Keep prompt sections stable. Put transient state in the existing note mechanism.
Do not widen a public contract for one speculative caller.
Add fast tests for the core behavior and actual bugs. Test contract consumers and fixtures when an API changes.
Run the exact project checks. Use the committed snapshot to verify live behavior once the loader exists.
Report absent runtime support as unverified. P3 provides no browser capture tools.
Before an authorized commit, inspect the complete diff and stage only owned files.
Use --no-gpg-sign. Never push without authorization.
+28 -1
View File
@@ -27,9 +27,12 @@ The following commands run from the repository root.
## Native/WASI comparison and import check
```sh
nix develop -c nu userland/git/test.nu /tmp/p1-git/target/wasm32-wasip2/release/git.wasm
nix develop -c nu userland/git/test.nu /tmp/p1-git/target/wasm32-wasip2/release/git.wasm /tmp/p1-git/server-target/debug/git-test-http-server
```
Build the native HTTP fixture first with the command in the local smart-HTTP section.
The comparison script starts the fixture on an ephemeral port.
It stops the fixture after success or failure.
The script checks that the component imports only WASI interfaces.
It creates separate native and WASI repositories under `/tmp/p1-git/`.
It fixes both identities and timestamps.
@@ -55,6 +58,8 @@ It also checks these forms:
- `status -sb` and `status --porcelain=v1`.
- `log -n`, `--oneline`, `-p`, `--stat`, and one literal path after `--`.
- `log --format` and `--pretty=format:` with `%H %h %s %an %ae %ad %d`.
- `log A..B`, `--name-only`, `--no-renames`, and `--diff-merges=first-parent`.
- `%x00`, `%x1e`, and `%(trailers:key=Sausage-Conversation,valueonly)` in log formats.
- `show --stat`, `show HEAD:path`, and merge commits without a combined diff.
- `diff HEAD~1`, `diff A..B`, `--name-only`, `--name-status`, and one literal path after `--`.
- `branch -a`, `-d`, `-D`, `-m`, and `--show-current`.
@@ -70,6 +75,28 @@ It also checks these forms:
- Lightweight tags, `tag -a -m`, bare `tag`, `--list`, and `-d`.
- `remote -v`, `add`, `remove`, `set-url`, and `get-url`.
- Clean `cherry-pick` and conflict resolution through `--continue` or `--abort`.
- `merge-base A B`, `--is-ancestor`, and `rev-list --count A..B`.
- `rev-parse --verify --quiet`, `--abbrev-ref HEAD`, and `symbolic-ref [--quiet] [--short] HEAD`.
- `update-ref <ref> <new> [<old>]` with an atomic old-value check.
- `worktree add --detach <path> <rev>`, `prune`, and `list --porcelain`.
- `checkout --detach --force <rev>` and `clean -fdq`.
- `merge --no-edit -m <message>` and `--ff-only`.
- `diff --numstat --no-renames`, `--diff-filter=U --name-only`, and `--name-only A B`.
- NUL-separated status, file lists, diff names, and numstat through `-z`.
- `status --porcelain --untracked-files=all` and `ls-files --others --exclude-standard`.
- `branch --list <name>` and `check-ignore --no-index <path>`.
- `fetch <remote> <ref>` with the selected revision in `FETCH_HEAD`.
If `SAUSAGE_CONVERSATION` exists, `commit` appends its value as a `Sausage-Conversation` trailer.
The trailer follows a blank line after the message body.
Without this variable, `commit` preserves the normal message.
The tests compare both forms with native Git, including commit IDs.
`clean -fdq` preserves tracked files, ignored files, and nested repositories.
CLI `status` refreshes cached file stats in the index after the content comparison.
Repeated status calls reuse those stats instead of hashing unchanged files again.
If the index cannot accept writes, status still reports changes.
The read-only status API does not refresh the index.
The comparison suite checks cache refresh, cache reuse, and same-size content edits.
The log applies its path filter before its count limit.
Stashes use ordinary Git commits and the `refs/stash` reflog.
+29 -5
View File
@@ -1,4 +1,4 @@
//! Read-only snapshots and patches. Ordinary Git state only; never refresh/write the index.
//! Snapshots and patches. Only CLI status refreshes the ordinary Git index stat cache.
use crate::Result;
use crate::{
HeadInfo, head_info,
@@ -173,6 +173,13 @@ fn quote(prefix: &str, path: &[u8]) -> String {
}
// Internal byte paths preserve identity even when the RPC display is lossy.
fn compare(old: &Snapshot, new: &Snapshot) -> Vec<(Vec<u8>, Option<Vec<u8>>, Kind)> {
compare_with_renames(old, new, true)
}
fn compare_with_renames(
old: &Snapshot,
new: &Snapshot,
renames: bool,
) -> Vec<(Vec<u8>, Option<Vec<u8>>, Kind)> {
let mut deleted: BTreeSet<_> = old
.keys()
.filter(|p| !new.contains_key(*p))
@@ -187,7 +194,7 @@ fn compare(old: &Snapshot, new: &Snapshot) -> Vec<(Vec<u8>, Option<Vec<u8>>, Kin
// Git does not pair empty blobs as renames by default.
let empty =
gix::objs::compute_hash(entry.id.kind(), gix::objs::Kind::Blob, b"").ok();
let source = if Some(entry.id) == empty {
let source = if !renames || Some(entry.id) == empty {
None
} else {
deleted.iter().find(|p| old.get(*p) == Some(entry)).cloned()
@@ -216,6 +223,13 @@ fn changes(old: &Snapshot, new: &Snapshot) -> Vec<Change> {
.collect()
}
pub fn status(repo: &gix::Repository) -> Result<Status> {
status_inner(repo, false)
}
/// CLI status refreshes cached file stats. RPC status remains read-only.
pub fn refresh_status(repo: &gix::Repository) -> Result<Status> {
status_inner(repo, true)
}
fn status_inner(repo: &gix::Repository, refresh: bool) -> Result<Status> {
repo::worktree(repo)?;
let (index, conflicts) = index(repo)?;
let mut head = tree(repo, "HEAD")?;
@@ -230,13 +244,13 @@ pub fn status(repo: &gix::Repository) -> Result<Status> {
conflicted: conflicts.iter().map(|p| display(p)).collect(),
};
use gix::status::index_worktree::{Item, iter::Summary};
let iter = repo
let mut iter = repo
.status(gix::progress::Discard)
.map_err(|e| error("status", e))?
.untracked_files(gix::status::UntrackedFiles::Files)
.into_index_worktree_iter(Vec::<gix::bstr::BString>::new())
.map_err(|e| error("status", e))?;
for item in iter {
for item in iter.by_ref() {
let item = item.map_err(|e| error("status", e))?;
#[cfg(target_os = "wasi")]
if matches!(
@@ -278,6 +292,13 @@ pub fn status(repo: &gix::Repository) -> Result<Status> {
},
});
}
if refresh
&& let Some(mut outcome) = iter.into_outcome()
&& outcome.has_changes()
{
// A read-only repository must still support status, like native Git.
let _ = outcome.write_changes();
}
result.unstaged.sort_by(|a, b| a.path.cmp(&b.path));
result.untracked.sort();
Ok(result)
@@ -383,6 +404,9 @@ fn worktree(repo: &gix::Repository, index: &Snapshot, filter: Option<&str>) -> R
Ok(snapshot)
}
pub fn diff(p: DiffParams) -> Result<Vec<FileDiff>> {
diff_with_renames(p, true)
}
pub fn diff_with_renames(p: DiffParams, renames: bool) -> Result<Vec<FileDiff>> {
if let Some(path) = &p.path {
repo::validate_path(path)?;
}
@@ -425,7 +449,7 @@ pub fn diff(p: DiffParams) -> Result<Vec<FileDiff>> {
}
};
let mut result = Vec::new();
for (path, source, status) in compare(&old, &new) {
for (path, source, status) in compare_with_renames(&old, &new, renames) {
let before_path = source.as_ref().unwrap_or(&path);
if !matches_path(&path, p.path.as_deref()) && !matches_path(before_path, p.path.as_deref())
{
+269 -45
View File
@@ -59,6 +59,7 @@ fn identity(name: Option<String>, email: Option<String>) -> Result<Identity> {
mod cherry_pick;
mod history;
mod local;
mod plumbing;
mod stash;
use clap::{Parser, Subcommand};
#[derive(Parser)]
@@ -87,6 +88,39 @@ enum Command {
Fetch {
#[arg(default_value = "origin")]
remote: String,
revision: Option<String>,
},
MergeBase {
#[arg(long)]
is_ancestor: bool,
a: String,
b: String,
},
RevList {
#[arg(long)]
count: bool,
revisions: Vec<String>,
},
SymbolicRef {
#[arg(long)]
short: bool,
name: String,
},
UpdateRef {
name: String,
new: String,
old: Option<String>,
},
Clean {
#[arg(short = 'f')]
force: bool,
#[arg(short = 'd')]
directories: bool,
},
CheckIgnore {
#[arg(long)]
no_index: bool,
paths: Vec<String>,
},
Push {
#[arg(short = 'u', long)]
@@ -113,6 +147,8 @@ enum Command {
value: Option<String>,
},
RevParse {
#[arg(long)]
verify: bool,
#[arg(long)]
show_toplevel: bool,
#[arg(long)]
@@ -124,6 +160,10 @@ enum Command {
revisions: Vec<String>,
},
Status {
#[arg(long, default_value = "normal")]
untracked_files: String,
#[arg(short = 'z')]
nul: bool,
#[arg(short, long)]
short: bool,
#[arg(long, num_args = 0..=1, default_missing_value = "v1", require_equals = true)]
@@ -159,10 +199,16 @@ enum Command {
no_gpg_sign: bool,
},
Log {
#[arg(long)]
name_only: bool,
#[arg(long)]
no_renames: bool,
#[arg(long)]
diff_merges: Option<String>,
#[arg(long)]
oneline: bool,
#[arg(short = 'n', default_value = "50")]
limit: usize,
#[arg(short = 'n')]
limit: Option<usize>,
#[arg(long)]
format: Option<String>,
#[arg(long)]
@@ -176,6 +222,14 @@ enum Command {
paths: Vec<String>,
},
Diff {
#[arg(long)]
numstat: bool,
#[arg(long)]
no_renames: bool,
#[arg(long)]
diff_filter: Option<String>,
#[arg(short = 'z')]
nul: bool,
#[arg(long, alias = "staged")]
cached: bool,
#[arg(long)]
@@ -189,6 +243,8 @@ enum Command {
paths: Vec<String>,
},
Branch {
#[arg(long)]
list: bool,
#[arg(long)]
show_current: bool,
#[arg(short = 'a')]
@@ -208,12 +264,18 @@ enum Command {
target: String,
},
Checkout {
#[arg(long)]
detach: bool,
#[arg(long, short = 'f')]
force: bool,
#[arg(short = 'b')]
create: bool,
target: Option<String>,
paths: Vec<String>,
},
Merge {
#[arg(long)]
no_edit: bool,
#[arg(long)]
ff_only: bool,
#[arg(long)]
@@ -239,7 +301,14 @@ enum Command {
#[command(subcommand)]
command: Worktree,
},
LsFiles,
LsFiles {
#[arg(long)]
others: bool,
#[arg(long)]
exclude_standard: bool,
#[arg(short = 'z')]
nul: bool,
},
Reset {
#[arg(long, conflicts_with_all = ["hard", "mixed"])]
soft: bool,
@@ -290,11 +359,14 @@ enum Command {
}
#[derive(Subcommand)]
enum Worktree {
Prune,
List {
#[arg(long)]
porcelain: bool,
},
Add {
#[arg(long)]
detach: bool,
#[arg(short = 'b')]
branch: Option<String>,
path: String,
@@ -481,12 +553,74 @@ pub fn run(cli: Cli) -> Result<i32> {
abort,
revision,
} => return cherry_pick::run(&repository, revision, resume, abort, quiet),
Command::LsFiles => {
Command::MergeBase { a, b, is_ancestor } => {
return plumbing::merge_base(&repository, &a, &b, is_ancestor);
}
Command::RevList { count, revisions } => {
let mut included = std::collections::BTreeSet::new();
let mut excluded = std::collections::BTreeSet::new();
for revision in revisions {
let (tip, hidden) = plumbing::range(&repository, &revision)?;
included.extend(plumbing::ancestors(&repository, tip)?);
excluded.extend(hidden);
}
included.retain(|id| !excluded.contains(id));
if count {
println!("{}", included.len());
} else {
for id in included {
println!("{id}");
}
}
}
Command::SymbolicRef { short, name } => {
let reference = repository
.find_reference(name.as_str())
.map_err(|e| error("symbolic-ref", e))?;
if let gix::refs::TargetRef::Symbolic(target) = reference.target() {
println!(
"{}",
if short {
target.shorten().to_string()
} else {
target.as_bstr().to_string()
}
);
} else {
if !quiet {
eprintln!("fatal: ref {name} is not a symbolic ref");
}
return Ok(if quiet { 1 } else { 128 });
}
}
Command::UpdateRef { name, new, old } => {
plumbing::update_ref(&repository, &name, &new, old.as_deref())?
}
Command::CheckIgnore { no_index, paths } => {
return plumbing::check_ignore(&repository, &paths, no_index);
}
Command::Clean { force, directories } => {
if !force {
return Err(error("clean", "requires -f"));
}
plumbing::clean(&repository, directories, quiet)?;
}
Command::LsFiles {
others,
exclude_standard: _,
nul,
} => {
if others {
for path in changes::status(&repository)?.untracked {
print!("{path}{}", if nul { "\0" } else { "\n" });
}
return Ok(0);
}
let index = repository
.index_or_empty()
.map_err(|e| error("ls-files", e))?;
for entry in index.entries() {
println!("{}", entry.path(&index));
print!("{}{}", entry.path(&index), if nul { "\0" } else { "\n" });
}
}
Command::Config { key, value, .. } => {
@@ -511,6 +645,7 @@ pub fn run(cli: Cli) -> Result<i32> {
}
}
Command::RevParse {
verify,
show_toplevel,
git_dir,
abbrev_ref,
@@ -531,10 +666,11 @@ pub fn run(cli: Cli) -> Result<i32> {
);
} else {
for revision in revisions {
let id = repository
.rev_parse_single(revision.as_str())
.map_err(|e| error("revision", e))?
.to_string();
let id = match repository.rev_parse_single(revision.as_str()) {
Ok(id) => id.to_string(),
Err(_) if verify && quiet => return Ok(1),
Err(e) => return Err(error("revision", e)),
};
println!("{}", if short { &id[..7] } else { &id });
}
}
@@ -591,7 +727,7 @@ pub fn run(cli: Cli) -> Result<i32> {
eprintln!("usage: git commit -m <message> | -F <file> | --amend --no-edit");
return Ok(129);
};
let message = message.trim_end().to_owned();
let message = plumbing::trailer(&message);
if !amend && !allow_empty && merge::pending_parent(&repository)?.is_none() {
let status = changes::status(&repository)?;
if status.staged.is_empty()
@@ -666,11 +802,13 @@ pub fn run(cli: Cli) -> Result<i32> {
})?);
}
Command::Status {
untracked_files,
nul,
short,
porcelain,
branch,
} => {
let mut status = changes::status(&repository)?;
let mut status = changes::refresh_status(&repository)?;
let index = repository
.index_or_empty()
.map_err(|e| error("status index", e))?;
@@ -681,6 +819,13 @@ pub fn run(cli: Cli) -> Result<i32> {
.collect();
let mut untracked = std::collections::BTreeSet::new();
for path in status.untracked {
if untracked_files == "no" {
continue;
}
if untracked_files == "all" {
untracked.insert(path);
continue;
}
let collapsed = path
.match_indices('/')
.map(|(i, _)| &path[..=i])
@@ -714,7 +859,11 @@ pub fn run(cli: Cli) -> Result<i32> {
);
}
let mut rows = std::collections::BTreeMap::new();
let mut sources = std::collections::BTreeMap::new();
for c in status.staged {
if let Some(source) = c.old_path {
sources.insert(c.path.clone(), source);
}
rows.entry(c.path).or_insert([' '; 2])[0] = code(&c.status);
}
for c in status.unstaged {
@@ -729,12 +878,24 @@ pub fn run(cli: Cli) -> Result<i32> {
for untracked in [false, true] {
for (p, c) in &rows {
if (c == &['?', '?']) == untracked {
println!("{}{} {p}", c[0], c[1]);
if let Some(source) = sources.get(p) {
if nul {
print!("{}{} {p}\0{source}\0", c[0], c[1]);
} else {
println!("{}{} {source} -> {p}", c[0], c[1]);
}
} else {
print!("{}{} {p}{}", c[0], c[1], if nul { "\0" } else { "\n" });
}
}
}
}
}
Command::Diff {
numstat,
no_renames,
diff_filter,
nul,
cached,
stat,
name_only,
@@ -748,22 +909,41 @@ pub fn run(cli: Cli) -> Result<i32> {
} else {
(revisions.first().cloned(), revisions.get(1).cloned())
};
let diffs = changes::diff(changes::DiffParams {
repo: path,
from,
to,
staged: Some(cached),
path: paths.first().cloned(),
context: None,
})?;
if diff_filter.as_deref() == Some("U") {
for p in changes::status(&repository)?.conflicted {
print!("{p}{}", if nul { "\0" } else { "\n" });
}
return Ok(0);
}
let diffs = changes::diff_with_renames(
changes::DiffParams {
repo: path,
from,
to,
staged: Some(cached),
path: paths.first().cloned(),
context: None,
},
!no_renames,
)?;
if name_only || name_status {
for diff in diffs {
if name_status {
println!("{}\t{}", code(&diff.status), diff.path);
print!(
"{}{}{}{}",
code(&diff.status),
if nul { "\0" } else { "\t" },
diff.path,
if nul { "\0" } else { "\n" }
);
} else {
println!("{}", diff.path);
print!("{}{}", diff.path, if nul { "\0" } else { "\n" });
}
}
} else if numstat {
for diff in &diffs {
output::numstat(diff, nul);
}
} else if stat {
output::stat(&diffs, false);
} else {
@@ -773,6 +953,9 @@ pub fn run(cli: Cli) -> Result<i32> {
}
}
Command::Log {
name_only,
no_renames,
diff_merges,
oneline,
limit,
format,
@@ -782,13 +965,10 @@ pub fn run(cli: Cli) -> Result<i32> {
revision,
paths,
} => {
let commit = repository
.rev_parse_single(revision.as_deref().unwrap_or("HEAD"))
.map_err(|e| error("log", e))?
.object()
.map_err(|e| error("log", e))?
.peel_to_commit()
.map_err(|e| error("log", e))?;
let limit = limit.unwrap_or(usize::MAX);
let (tip, excluded) =
plumbing::range(&repository, revision.as_deref().unwrap_or("HEAD"))?;
let commit = repository.find_commit(tip).map_err(|e| error("log", e))?;
let ids = if let Some(path) = paths.first() {
history::paths(&repository, commit.id, path)?
} else {
@@ -800,16 +980,28 @@ pub fn run(cli: Cli) -> Result<i32> {
))
.all()
.map_err(|e| error("log", e))?
.filter(|info| {
info.as_ref()
.map_or(true, |info| !excluded.contains(&info.id))
})
.take(limit)
.map(|info| info.map(|info| info.id).map_err(|e| error("log", e)))
.collect::<Result<Vec<_>>>()?
};
let mut first = true;
for id in ids.into_iter().take(limit) {
for id in ids
.into_iter()
.filter(|id| !excluded.contains(id))
.take(limit)
{
let c = repository.find_commit(id).map_err(|e| error("log", e))?;
let decoded = c.decode().map_err(|e| error("log", e))?;
let message = decoded.message.to_str_lossy();
let diffs = if patch || stat || !paths.is_empty() {
let diffs = if (patch || stat || name_only || !paths.is_empty())
&& (decoded.parents.len() <= 1
|| diff_merges.as_deref() == Some("first-parent")
|| !name_only)
{
let old = match decoded.parents.first() {
Some(id) => id.to_string(),
None => repository
@@ -817,14 +1009,17 @@ pub fn run(cli: Cli) -> Result<i32> {
.map_err(|e| error("log tree", e))?
.to_string(),
};
changes::diff(changes::DiffParams {
repo: path.clone(),
from: Some(old),
to: Some(c.id.to_string()),
staged: None,
path: paths.first().cloned(),
context: None,
})?
changes::diff_with_renames(
changes::DiffParams {
repo: path.clone(),
from: Some(old),
to: Some(c.id.to_string()),
staged: None,
path: paths.first().cloned(),
context: None,
},
!no_renames,
)?
} else {
Vec::new()
};
@@ -867,7 +1062,7 @@ pub fn run(cli: Cli) -> Result<i32> {
} else {
format!(" ({})", labels.join(", "))
};
let text = template
let text = plumbing::format(template, &message)
.replace("%H", &c.id.to_string())
.replace("%h", &c.id.to_string()[..7])
.replace("%s", message.lines().next().unwrap_or(""))
@@ -897,6 +1092,12 @@ pub fn run(cli: Cli) -> Result<i32> {
output::commit(&c)?;
first = false;
}
if name_only && !diffs.is_empty() {
println!();
for diff in &diffs {
println!("{}", diff.path);
}
}
if patch || stat {
println!();
if stat {
@@ -911,6 +1112,7 @@ pub fn run(cli: Cli) -> Result<i32> {
}
}
Command::Branch {
list,
delete,
force_delete,
show_current,
@@ -938,7 +1140,8 @@ pub fn run(cli: Cli) -> Result<i32> {
}
return Ok(0);
}
if let Some(name) = name {
if let Some(name) = name.as_ref().filter(|_| !list) {
let name = name.clone();
if delete || force_delete {
let id = branches::resolve(&repository, &name)?.to_string();
branches::delete(
@@ -964,6 +1167,7 @@ pub fn run(cli: Cli) -> Result<i32> {
)?;
}
} else {
let pattern = name.as_deref();
let head = head_info(&repository)?;
for branch in repository
.references()
@@ -978,6 +1182,12 @@ pub fn run(cli: Cli) -> Result<i32> {
{
continue;
}
if list
&& pattern
.is_some_and(|pattern| branch.name().shorten().to_string() != pattern)
{
continue;
}
println!(
"{} {}",
if head.branch.as_deref()
@@ -997,6 +1207,8 @@ pub fn run(cli: Cli) -> Result<i32> {
}
}
Command::Checkout {
detach,
force,
create,
target,
mut paths,
@@ -1021,9 +1233,14 @@ pub fn run(cli: Cli) -> Result<i32> {
&repository,
branches::CheckoutParams {
repo: path,
target: required(target, "checkout target")?,
target: if detach {
branches::resolve(&repository, &required(target, "checkout target")?)?
.to_string()
} else {
required(target, "checkout target")?
},
create: Some(create),
force: None,
force: Some(force),
},
)?;
}
@@ -1040,6 +1257,7 @@ pub fn run(cli: Cli) -> Result<i32> {
)?;
}
Command::Merge {
no_edit: _,
ff_only,
no_ff,
no_commit,
@@ -1058,7 +1276,7 @@ pub fn run(cli: Cli) -> Result<i32> {
merge::MergeParams {
repo: path.clone(),
rev: required(revision, "merge")?,
message,
message: message.map(|m| format!("{}\n", m.trim_end())),
ff: Some(if ff_only {
merge::FastForward::Only
} else if no_ff {
@@ -1154,6 +1372,7 @@ pub fn run(cli: Cli) -> Result<i32> {
}
}
Command::Worktree { command } => match command {
Worktree::Prune => worktrees::prune(&repository)?,
Worktree::List { porcelain } => {
let common = repository
.common_dir()
@@ -1185,6 +1404,9 @@ pub fn run(cli: Cli) -> Result<i32> {
.unwrap_or(0)
+ 2;
for p in paths {
let p = p
.canonicalize()
.map_err(|e| error("worktree list path", e))?;
let linked = repo::open(&p.to_string_lossy())?;
let head = head_info(&linked)?;
let id = head.id.unwrap_or_else(|| "0".repeat(40));
@@ -1206,6 +1428,7 @@ pub fn run(cli: Cli) -> Result<i32> {
}
}
Worktree::Add {
detach,
branch,
path: target,
revision,
@@ -1226,6 +1449,7 @@ pub fn run(cli: Cli) -> Result<i32> {
.to_string_lossy()
.into_owned(),
branch,
detached: Some(detach),
from: revision,
adopt: None,
},
@@ -1324,7 +1548,7 @@ pub fn run(cli: Cli) -> Result<i32> {
}
}
}
Command::Fetch { remote } => network::fetch(&path, &remote)?,
Command::Fetch { remote, revision } => network::fetch(&path, &remote, revision.as_deref())?,
Command::Push {
set_upstream,
force,
+47 -5
View File
@@ -16,10 +16,10 @@ struct FetchResult {
copied_objects: Option<usize>,
updated_refs: Option<Vec<String>>,
}
pub(crate) fn fetch(path: &str, remote: &str) -> Result<()> {
pub(crate) fn fetch(path: &str, remote: &str, revision: Option<&str>) -> Result<()> {
let mut repo = repo::open(path)?;
repo.refs.write_reflog = gix::refs::store::WriteReflog::Disable;
perform(&repo, remote, None, false)?;
perform(&repo, remote, None, false, revision)?;
Ok(())
}
pub(crate) fn clone(url: &str, path: &str) -> Result<()> {
@@ -32,7 +32,7 @@ pub(crate) fn clone(url: &str, path: &str) -> Result<()> {
remotes::set(&repo, "origin", url, true)?;
let mut repo = repo::open(&name)?;
repo.refs.write_reflog = gix::refs::store::WriteReflog::Disable;
let fetched = perform(&repo, "origin", None, false)?;
let fetched = perform(&repo, "origin", None, false, None)?;
let branch = fetched
.default_branch
.ok_or_else(|| error("clone", "server HEAD does not name a branch"))?;
@@ -61,9 +61,10 @@ fn perform(
remote_name: &str,
depth: Option<u32>,
prune: bool,
revision: Option<&str>,
) -> Result<Fetched> {
remotes::validate_name(remote_name)?;
let remote = repo
let mut remote = repo
.find_remote(remote_name)
.map_err(|_| error("fetch", "configured remote not found"))?;
let url = remote
@@ -83,6 +84,21 @@ fn perform(
));
}
}
if let Some(revision) = revision {
let source = if revision.starts_with("refs/") || revision.len() == 40 {
revision.to_owned()
} else {
format!("refs/heads/{revision}")
};
let spec = if let Some(branch) = source.strip_prefix("refs/heads/") {
format!("+{source}:refs/remotes/{remote_name}/{branch}")
} else {
source
};
remote
.replace_refspecs([spec.as_bytes().as_bstr()], gix::remote::Direction::Fetch)
.map_err(|e| error("fetch refspec", e))?;
}
let transport: Box<dyn gix_transport::client::blocking_io::Transport> =
if crate::ssh::is_ssh(&url) {
Box::new(crate::ssh::connection(
@@ -137,6 +153,13 @@ fn perform(
_ => {}
}
}
let fetch_heads: Vec<_> = refs
.iter()
.filter_map(|r| {
let (name, id, _) = r.unpack();
id.map(|id| (name.to_string(), id.to_string()))
})
.collect();
let advertised_refs = refs.len();
let empty = heads.is_empty() && refs.iter().all(|r| r.unpack().1.is_none());
if let Some(depth) = depth {
@@ -170,7 +193,26 @@ fn perform(
));
}
}
if let Some(branch) = default_branch.as_deref().filter(|b| heads.contains(*b)) {
let fetch_head = fetch_heads
.iter()
.filter(|(name, _)| {
revision.map_or(name.starts_with("refs/heads/"), |r| {
name == r || name == &format!("refs/heads/{r}")
})
})
.map(|(name, id)| {
format!(
"{id}\t\tbranch '{}' of {url}\n",
name.trim_start_matches("refs/heads/")
)
})
.collect::<String>();
std::fs::write(repo.git_dir().join("FETCH_HEAD"), fetch_head)
.map_err(|e| error("FETCH_HEAD", e))?;
if let Some(branch) = default_branch
.as_deref()
.filter(|b| revision.is_none() && heads.contains(*b))
{
use gix::refs::{
Target,
transaction::{Change, PreviousValue, RefEdit},
+15
View File
@@ -122,6 +122,21 @@ fn label(kind: &Kind) -> &'static str {
Kind::Renamed => "renamed:",
}
}
pub(crate) fn numstat(diff: &FileDiff, nul: bool) {
let hunks = diff.patch.lines().skip_while(|l| !l.starts_with("@@"));
let added = hunks.clone().filter(|l| l.starts_with('+')).count();
let removed = hunks.filter(|l| l.starts_with('-')).count();
let (added, removed) = if diff.binary {
("-".into(), "-".into())
} else {
(added.to_string(), removed.to_string())
};
print!(
"{added}\t{removed}\t{}{}",
diff.path,
if nul { "\0" } else { "\n" }
);
}
pub(crate) fn stat(diffs: &[FileDiff], summary: bool) {
summarize(diffs, true, summary);
}
+263
View File
@@ -0,0 +1,263 @@
//! Revision sets, reference transactions, and ignored-path queries for coordination.
use crate::{Result, branches, repo::error};
use gix::bstr::ByteSlice;
use std::collections::BTreeSet;
pub(crate) fn ancestors(
repo: &gix::Repository,
tip: gix::ObjectId,
) -> Result<BTreeSet<gix::ObjectId>> {
repo.find_commit(tip)
.map_err(|e| error("revision", e))?
.ancestors()
.use_commit_graph(false)
.all()
.map_err(|e| error("revision walk", e))?
.map(|info| info.map(|i| i.id).map_err(|e| error("revision walk", e)))
.collect()
}
pub(crate) fn range(
repo: &gix::Repository,
input: &str,
) -> Result<(gix::ObjectId, BTreeSet<gix::ObjectId>)> {
let (from, to) = input.split_once("..").unwrap_or(("", input));
let tip = branches::resolve(repo, if to.is_empty() { "HEAD" } else { to })?;
let excluded = if input.contains("..") {
ancestors(
repo,
branches::resolve(repo, if from.is_empty() { "HEAD" } else { from })?,
)?
} else {
BTreeSet::new()
};
Ok((tip, excluded))
}
pub(crate) fn merge_base(
repo: &gix::Repository,
a: &str,
b: &str,
is_ancestor: bool,
) -> Result<i32> {
let a = branches::resolve(repo, a)?;
let b = branches::resolve(repo, b)?;
let base = repo.merge_base(a, b).map_err(|e| error("merge-base", e))?;
if is_ancestor {
return Ok(if base.is_some_and(|id| id.detach() == a) {
0
} else {
1
});
}
if let Some(base) = base {
println!("{base}");
Ok(0)
} else {
Ok(1)
}
}
pub(crate) fn update_ref(
repo: &gix::Repository,
name: &str,
new: &str,
old: Option<&str>,
) -> Result<()> {
use gix::refs::{
Target,
transaction::{Change, PreviousValue, RefEdit},
};
let expected = match old {
Some(value) if value == "0".repeat(40) => PreviousValue::MustNotExist,
Some(value) => PreviousValue::MustExistAndMatch(Target::Object(
repo.rev_parse_single(value)
.map_err(|e| error("old revision", e))?
.detach(),
)),
None => PreviousValue::Any,
};
branches::edit_refs(
repo,
vec![RefEdit {
name: name.try_into().map_err(|e| error("reference", e))?,
deref: true,
change: Change::Update {
log: Default::default(),
expected,
new: Target::Object(
repo.rev_parse_single(new)
.map_err(|e| error("new revision", e))?
.detach(),
),
},
}],
)?;
Ok(())
}
pub(crate) fn trailer(message: &str) -> String {
if let Ok(value) = std::env::var("SAUSAGE_CONVERSATION") {
let message = message.trim_end();
let line = format!("Sausage-Conversation: {value}");
if message.lines().any(|l| l == line) {
return message.into();
}
let last = message.rsplit("\n\n").next().unwrap_or(message);
let has_trailers = message.contains("\n\n")
&& last
.lines()
.all(|l| l.split_once(':').is_some_and(|(k, _)| !k.contains(' ')));
format!(
"{message}{}{line}",
if has_trailers { "\n" } else { "\n\n" }
)
} else {
message.trim_end().into()
}
}
pub(crate) fn format(template: &str, message: &str) -> String {
let trailer = "%(trailers:key=Sausage-Conversation,valueonly)";
let values = message
.rsplit_once("\n\n")
.map(|(_, block)| {
block
.lines()
.filter_map(|line| {
line.split_once(':')
.filter(|(key, _)| key.eq_ignore_ascii_case("Sausage-Conversation"))
.map(|(_, value)| format!("{}\n", value.trim()))
})
.collect::<String>()
})
.unwrap_or_default();
let text = template.replace(trailer, &values);
let mut output = String::new();
let mut rest = text.as_str();
while let Some(i) = rest.find("%x") {
output.push_str(&rest[..i]);
rest = &rest[i..];
if let Some(hex) = rest.get(2..4).and_then(|s| u8::from_str_radix(s, 16).ok()) {
output.push(hex as char);
rest = &rest[4..];
} else {
output.push_str("%x");
rest = &rest[2..];
}
}
output.push_str(rest);
output
}
pub(crate) fn check_ignore(
repo: &gix::Repository,
paths: &[String],
no_index: bool,
) -> Result<i32> {
let index = repo.index_or_empty().map_err(|e| error("index", e))?;
let mut excludes = repo
.excludes(
&index,
None,
gix::worktree::stack::state::ignore::Source::WorktreeThenIdMappingIfNotSkipped,
)
.map_err(|e| error("excludes", e))?;
let mut found = false;
for path in paths {
crate::repo::validate_path(path)?;
if !no_index
&& index
.entries()
.iter()
.any(|e| e.path(&index) == path.as_bytes().as_bstr())
{
continue;
}
if excludes
.at_entry(path.as_str(), None)
.map_err(|e| error("ignore", e))?
.is_excluded()
{
println!("{path}");
found = true;
}
}
Ok(if found { 0 } else { 1 })
}
pub(crate) fn clean(repo: &gix::Repository, directories: bool, quiet: bool) -> Result<()> {
let root = crate::repo::worktree(repo)?;
let index = repo.index_or_empty().map_err(|e| error("index", e))?;
let mut excludes = repo
.excludes(
&index,
None,
gix::worktree::stack::state::ignore::Source::WorktreeThenIdMappingIfNotSkipped,
)
.map_err(|e| error("excludes", e))?;
fn walk(
root: &std::path::Path,
relative: &std::path::Path,
index: &gix::index::State,
excludes: &mut gix::AttributeStack<'_>,
directories: bool,
quiet: bool,
) -> Result<()> {
for entry in
std::fs::read_dir(root.join(relative)).map_err(|e| error("clean directory", e))?
{
let entry = entry.map_err(|e| error("clean entry", e))?;
if entry.file_name() == ".git" {
continue;
}
let path = relative.join(entry.file_name());
let name = path.to_string_lossy();
let dir = entry
.file_type()
.map_err(|e| error("clean type", e))?
.is_dir();
let tracked = index.entries().iter().any(|e| {
let p = e.path(index);
p == name.as_bytes().as_bstr()
|| (dir && p.starts_with(format!("{name}/").as_bytes()))
});
if excludes
.at_entry(
name.as_ref(),
if dir {
Some(gix::index::entry::Mode::DIR)
} else {
None
},
)
.map_err(|e| error("clean ignore", e))?
.is_excluded()
{
continue;
}
if dir {
if entry.path().join(".git").exists() || (!directories && !tracked) {
continue;
}
walk(root, &path, index, excludes, directories, quiet)?;
if directories
&& !tracked
&& std::fs::read_dir(entry.path())
.map_err(|e| error("clean directory", e))?
.next()
.is_none()
{
std::fs::remove_dir(entry.path()).map_err(|e| error("clean directory", e))?;
}
} else if !tracked {
std::fs::remove_file(entry.path()).map_err(|e| error("clean file", e))?;
if !quiet {
println!("Removing {name}");
}
}
}
Ok(())
}
walk(
root,
std::path::Path::new(""),
&index,
&mut excludes,
directories,
quiet,
)
}
+35 -8
View File
@@ -14,6 +14,7 @@ use std::{
/// Attach a linked worktree, or adopt existing bytes onto an unborn branch.
#[derive(serde::Serialize, serde::Deserialize)]
pub(crate) struct AddParams {
pub detached: Option<bool>,
/// Repository name or absolute worktree/bare path under /repos/ or /plugins/.
pub repo: String,
/// Absolute worktree path under /plugins/ or /repos/, without dot segments.
@@ -150,10 +151,28 @@ pub(crate) struct AttachParams {
/// Existing short local branch name.
pub branch: String,
}
pub(crate) fn prune(repo: &gix::Repository) -> Result<()> {
let base = repo.common_dir().join("worktrees");
if !base.exists() {
return Ok(());
}
for entry in fs::read_dir(base).map_err(|e| error("worktree prune", e))? {
let dir = entry.map_err(|e| error("worktree prune", e))?.path();
if dir.join("locked").exists() {
continue;
}
let gitdir = read(&dir.join("gitdir"))?;
if !Path::new(gitdir.trim()).exists() {
fs::remove_dir_all(dir).map_err(|e| error("worktree prune", e))?;
}
}
Ok(())
}
pub(crate) fn attach(repo: &gix::Repository, p: AttachParams) -> Result<WorktreeInfo> {
add_with(
repo,
AddParams {
detached: None,
repo: p.repo,
path: p.path,
branch: p.branch,
@@ -170,20 +189,24 @@ fn add_with(repo: &gix::Repository, p: AddParams, keep_files: bool) -> Result<Wo
let path = absolute(&p.path)?;
let common = canonical(repo.common_dir())?;
separate(&path, &common)?;
let detached = p.detached.unwrap_or(false);
let branch = branches::branch_name(&p.branch)?;
ensure_available(repo, &branch)?;
if !detached {
ensure_available(repo, &branch)?;
}
if let Some(current_path) = repo.workdir() {
if crate::head_info(repo)?.branch.as_deref() == Some(&branch) {
if !detached && crate::head_info(repo)?.branch.as_deref() == Some(&branch) {
return Err(error(
"branch checked out",
format!("{branch} is checked out at {}", current_path.display()),
));
}
}
let exists = repo
.try_find_reference(branch.as_str())
.map_err(|e| error("worktree branch", e))?
.is_some();
let exists = !detached
&& repo
.try_find_reference(branch.as_str())
.map_err(|e| error("worktree branch", e))?
.is_some();
let adopt = p.adopt.unwrap_or(false);
if adopt && (exists || p.from.is_some()) {
return Err(error(
@@ -296,8 +319,12 @@ fn add_with(repo: &gix::Repository, p: AddParams, keep_files: bool) -> Result<Wo
&linked,
branches::CheckoutParams {
repo: p.path.clone(),
target: p.branch.clone(),
create: Some(!exists),
target: if detached {
id.to_string()
} else {
p.branch.clone()
},
create: Some(!exists && !detached),
force: None,
},
)?
+131 -7
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env nu
# Compare a deterministic agent workflow with native Git. Fail on every difference.
def main [component: path] {
def main [component: path, server: path = '/tmp/p1-git/server-target/debug/git-test-http-server'] {
let component = ($component | path expand)
let wit = (^wasm-tools component wit $component | complete)
if $wit.exit_code != 0 {error make {msg: $wit.stderr}}
@@ -11,6 +11,14 @@ def main [component: path] {
print ($imports | str join (char newline))
let root = (mktemp -d -p /tmp/p1-git workflow.XXXXXX)
print $'Fixture: ($root)'
let server_root = ($root | path join server)
mkdir $server_root
let ready = ($root | path join port)
let server_job = (job spawn { ^$server $server_root $ready })
let deadline = ((date now) + 10sec)
while not ($ready | path exists) { if (date now) > $deadline { job kill $server_job; error make {msg: 'HTTP fixture did not start'} } }
let url = $'http://127.0.0.1:(open --raw $ready | str trim)/coord.git'
try {
let envs = {GIT_AUTHOR_NAME: 'Agent', GIT_AUTHOR_EMAIL: 'agent@example.test', GIT_COMMITTER_NAME: 'Agent', GIT_COMMITTER_EMAIL: 'agent@example.test', GIT_AUTHOR_DATE: '1700000000 +0000', GIT_COMMITTER_DATE: '1700000000 +0000', GIT_CONFIG_NOSYSTEM: '1', GIT_EDITOR: 'true', HOME: '/tmp/p1-git/empty-home', LC_ALL: 'C', TZ: 'UTC'}
mkdir $envs.HOME
with-env $envs {
@@ -29,6 +37,22 @@ def main [component: path] {
{args: [status --porcelain=v1], compare: true}
{args: [status], compare: true}
{args: [ls-files], compare: true}
{publish: true}
{args: [remote add fixture $url], compare: true, state: true}
{args: [fetch fixture main], compare: true, state: true}
{args: [rev-parse FETCH_HEAD], compare: true}
{args: [rev-parse --verify --quiet HEAD], compare: true}
{args: [rev-parse --verify --quiet absent], compare: true, exit: 1}
{args: [rev-parse --abbrev-ref HEAD], compare: true}
{args: [symbolic-ref HEAD], compare: true}
{args: [symbolic-ref --quiet --short HEAD], compare: true}
{args: [branch --list main], compare: true}
{args: [branch --list absent], compare: true}
{args: [worktree add --detach ../detached HEAD], compare: true, state: true}
{args: [worktree list --porcelain], compare: true, normalize_paths: true}
{remove: '../detached'}
{args: [worktree prune], compare: true, state: true}
{args: [worktree list --porcelain], compare: true, normalize_paths: true}
{args: [branch --show-current], compare: true}
{args: [branch -a], compare: true}
{args: [log -n 1], compare: true}
@@ -49,6 +73,21 @@ def main [component: path] {
{args: [rev-parse HEAD HEAD~1], compare: true}
{args: [diff HEAD~1], compare: true}
{args: [diff HEAD~1..HEAD], compare: true}
{args: [merge-base HEAD~1 HEAD], compare: true}
{args: [merge-base --is-ancestor HEAD~1 HEAD], compare: true}
{args: [merge-base --is-ancestor HEAD HEAD~1], compare: true, exit: 1}
{args: [rev-list --count HEAD~1..HEAD], compare: true}
{args: [rev-list --count HEAD..HEAD], compare: true}
{args: [log --oneline HEAD~1..HEAD], compare: true}
{args: [log --name-only --no-renames --diff-merges=first-parent '--format=%x1e%H%x00%s%x00%(trailers:key=Sausage-Conversation,valueonly)%x00' HEAD~1..HEAD], compare: true}
{args: [diff --name-only HEAD~1 HEAD], compare: true}
{args: [diff --name-only -z HEAD~1 HEAD], compare: true}
{args: [diff --numstat --no-renames HEAD~1 HEAD], compare: true}
{args: [diff --numstat --no-renames -z HEAD~1 HEAD], compare: true}
{args: [update-ref refs/heads/transaction HEAD], compare: true, state: true}
{args: [update-ref refs/heads/transaction HEAD~1 HEAD], compare: true, state: true}
{args: [update-ref refs/heads/transaction HEAD HEAD], compare: true, exit: 128, state: true}
{args: [branch -D transaction], compare: true, state: true}
{args: [diff --name-only HEAD~1..HEAD], compare: true}
{args: [diff --name-status HEAD~1..HEAD], compare: true}
{args: [diff HEAD~1 -- topic.txt], compare: true}
@@ -59,7 +98,7 @@ def main [component: path] {
{args: [branch disposable], compare: true}
{args: [branch -q -d disposable], compare: true}
{args: [switch main], compare: true}
{args: [merge topic], compare: true}
{args: [merge --ff-only topic], compare: true, state: true}
{write: 'changed\n', file: 'file.txt'}
{args: [status --short], compare: true}
{args: [diff], compare: true}
@@ -72,6 +111,8 @@ def main [component: path] {
{args: [commit -am conflict], compare: true}
{args: [switch main], compare: true}
{args: [merge topic], compare: true, exit: 1}
{args: [diff --diff-filter=U --name-only], compare: true}
{args: [diff --diff-filter=U --name-only -z], compare: true}
{args: [status --short], compare: true}
{args: [merge --abort], compare: true}
{args: [merge -q topic], compare: true, exit: 1}
@@ -113,7 +154,9 @@ def main [component: path] {
{args: [add agent.txt], compare: true}
{args: [commit -q -m agent], compare: true}
{args: [switch main], compare: true}
{args: [merge --no-ff -q agent-switch], compare: true}
{args: [merge --no-ff -q --no-edit -m landed agent-switch], compare: true, state: true}
{args: [log --name-only --no-renames --diff-merges=first-parent '--format=%x1e%H%x00%s%x00%(trailers:key=Sausage-Conversation,valueonly)%x00' HEAD~1..HEAD], compare: true}
{args: [merge --ff-only agent-switch], compare: true, state: true}
{args: [show --stat], compare: true}
{args: [commit --quiet --allow-empty -m quiet], compare: true}
{args: [init], compare: true, normalize_path: true, state: true}
@@ -217,6 +260,8 @@ def main [component: path] {
{args: [switch rework], compare: true}
{write: 'ours\n', file: 'agent.txt'}
{args: [commit -am ours], compare: true}
{args: [merge-base HEAD picks], compare: true}
{args: [merge --ff-only picks], compare: true, exit: 128, state: true}
{args: [cherry-pick picks], compare: true, exit: 1, state: true}
{args: [status --short], compare: true}
{args: [cherry-pick --abort], compare: true, state: true}
@@ -227,20 +272,73 @@ def main [component: path] {
{args: [log --oneline -n 3], compare: true}
{args: [status --short], compare: true}
{args: [cherry-pick --abort], compare: true, exit: 128, state: true}
{binary: true, file: 'binary.dat'}
{write: '+++ plus\n--- minus\n', file: 'counts.txt'}
{args: [add binary.dat counts.txt], compare: true}
{args: [diff --cached --numstat --no-renames], compare: true}
{args: [diff --cached --numstat --no-renames -z], compare: true}
{args: [reset --hard HEAD], compare: true, state: true}
{move: 'agent.txt', to: 'renamed.txt'}
{args: [add -A], compare: true}
{args: [status --porcelain -z], compare: true}
{args: [diff --cached --numstat --no-renames], compare: true}
{args: [diff --cached --numstat --no-renames -z], compare: true}
{args: [reset --hard HEAD], compare: true, state: true}
{write: 'ignored/\n*.tmp\n', file: '.gitignore'}
{args: [add .gitignore], compare: true}
{args: [commit -m ignores], compare: true}
{write: 'keep\n', file: 'ignored/keep.tmp'}
{write: 'loose\n', file: 'loose/file.txt'}
{write: 'loose\n', file: 'loose.txt'}
{args: [check-ignore --no-index ignored/keep.tmp], compare: true}
{args: [check-ignore --no-index file.txt], compare: true, exit: 1}
{args: [status --porcelain --untracked-files=all], compare: true}
{args: [status --porcelain --untracked-files=all -z], compare: true}
{args: [ls-files --others --exclude-standard], compare: true}
{args: [ls-files --others --exclude-standard -z], compare: true}
{write: 'raw path\n', file: "tab\tand\nnewline.txt"}
{args: [status --porcelain --untracked-files=all -z], compare: true}
{args: [ls-files --others --exclude-standard -z], compare: true}
{args: [clean -fdq], compare: true, state: true}
{args: [check-ignore --no-index ignored/keep.tmp], compare: true}
{write: 'dirty\n', file: 'agent.txt'}
{args: [checkout --detach --force HEAD~1], compare: true, state: true}
{args: [symbolic-ref --quiet HEAD], compare: true, exit: 1}
{args: [symbolic-ref HEAD], compare: true, exit: 128}
{args: [rev-parse --abbrev-ref HEAD], compare: true}
{args: [switch rework], compare: true}
{args: [commit --allow-empty -m trailer], native_args: [commit --allow-empty -m 'trailer\n\nSausage-Conversation: conversation-34'], conversation: conversation-34, compare: true, state: true}
{args: [log -n 1 --name-only --no-renames --diff-merges=first-parent '--format=%x1e%H%x00%s%x00%(trailers:key=Sausage-Conversation,valueonly)%x00'], compare: true}
{args: [commit --allow-empty -m no-trailer], compare: true, state: true}
{args: [log --name-only --no-renames --diff-merges=first-parent '--format=%x1e%H%x00%s%x00%(trailers:key=Sausage-Conversation,valueonly)%x00' HEAD~3..HEAD], compare: true}
{args: [status --bad-option], compare: true, exit: 129}
{args: [not-a-command], compare: true, exit: 1}
]
mut failures = 0
for step in $steps {
if 'write' in $step {
for side in [native wasi] { $step.write | str replace --all '\n' (char newline) | save --force ($root | path join $side repo $step.file) }
for side in [native wasi] {
let file = ($root | path join $side repo $step.file)
mkdir ($file | path dirname)
$step.write | str replace --all '\n' (char newline) | save --force $file
}
continue
}
if 'binary' in $step { for side in [native wasi] { 0x[610062] | save --force ($root | path join $side repo $step.file) }; continue }
if 'move' in $step { for side in [native wasi] { mv ($root | path join $side repo $step.move) ($root | path join $side repo $step.to) }; continue }
if 'publish' in $step {
let result = (^git clone --bare ($root | path join native repo) ($server_root | path join coord.git) | complete)
if $result.exit_code != 0 { error make {msg: $result.stderr} }
continue
}
if 'remove' in $step { for side in [native wasi] { rm -rf ($root | path join $side repo $step.remove) }; continue }
let args = $step.args
let native = (^git -C ($root | path join native repo) ...$args | complete)
let wasi = (^wasmtime run -S http=y -S cli-exit-with-code=y --dir $'($root)/wasi::/' --env GIT_AUTHOR_NAME --env GIT_AUTHOR_EMAIL --env GIT_COMMITTER_NAME --env GIT_COMMITTER_EMAIL --env GIT_AUTHOR_DATE --env GIT_COMMITTER_DATE $component -C /repo ...$args | complete)
let native_args = ($step.native_args? | default $args | each {|arg| $arg | into string | str replace --all '\n' (char newline)})
let native = (^git -C ($root | path join native repo) ...$native_args | complete)
let extra_env = if 'conversation' in $step { [--env $'SAUSAGE_CONVERSATION=($step.conversation)'] } else { [] }
let wasi = (^wasmtime run -S http=y -S cli-exit-with-code=y --dir $'($root)/wasi::/' --env GIT_AUTHOR_NAME --env GIT_AUTHOR_EMAIL --env GIT_COMMITTER_NAME --env GIT_COMMITTER_EMAIL --env GIT_AUTHOR_DATE --env GIT_COMMITTER_DATE ...$extra_env $component -C /repo ...$args | complete)
let expected = ($step.exit? | default 0)
let native_stdout = if ($step.normalize_path? | default false) { $native.stdout | str replace --all ($root | path join native repo) '/repo' } else { $native.stdout }
let native_stdout = if ($step.normalize_paths? | default false) { $native.stdout | str replace --all ($root | path join native) '' } else if ($step.normalize_path? | default false) { $native.stdout | str replace --all ($root | path join native repo) '/repo' } else { $native.stdout }
let equal = ($native.exit_code == $expected and $wasi.exit_code == $expected and (not $step.compare or $native_stdout == $wasi.stdout))
print $'($args | str join " "): (if $equal {"PASS"} else {"FAIL"}) native=($native.exit_code) wasi=($wasi.exit_code)'
if not $equal {
@@ -286,6 +384,30 @@ def main [component: path] {
$executable_ok = ($executable_ok and $indexed.exit_code == 0 and ($indexed.stdout | str starts-with '100755 '))
print $'executable checkout stays clean: (if $executable_ok {"PASS"} else {"FAIL"})'
if not $executable_ok { $failures += 1 }
# Status refreshes stale stats once, then detects a same-size content change.
let cached = ($root | path join cached)
mkdir $cached
^git -C $cached init -q -b main
'base' | save ($cached | path join file.txt)
^git -C $cached add file.txt
^git -C $cached commit --no-gpg-sign -q -m cached
^touch -m -d @1700000000 ($cached | path join file.txt)
let index = ($cached | path join .git/index)
let before = (open --raw $index | hash sha256)
mut cached_ok = true
mut refreshed = ''
for pass in [1 2] {
let result = (^wasmtime run -S http=y -S cli-exit-with-code=y --dir $'($cached)::/cached' $component -C /cached status --short | complete)
let current = (open --raw $index | hash sha256)
$cached_ok = ($cached_ok and $result.exit_code == 0 and $result.stdout == '')
if $pass == 1 { $cached_ok = ($cached_ok and $current != $before); $refreshed = $current } else { $cached_ok = ($cached_ok and $current == $refreshed) }
}
'edit' | save --force ($cached | path join file.txt)
let changed = (^wasmtime run -S http=y -S cli-exit-with-code=y --dir $'($cached)::/cached' $component -C /cached status --short | complete)
let native_changed = (^git -C $cached status --short | complete)
$cached_ok = ($cached_ok and $changed.exit_code == 0 and $changed.stdout == $native_changed.stdout and $changed.stdout == " M file.txt\n")
print $'status caches stats and detects edits: (if $cached_ok {"PASS"} else {"FAIL"})'
if not $cached_ok { $failures += 1 }
# A merge tree that differs from both parents requires a combined diff.
'combined resolution\n' | save --force ($root | path join native/repo/file.txt)
let native = ($root | path join native/repo)
@@ -299,4 +421,6 @@ def main [component: path] {
if not $equal { $failures += 1; print ($result | to json) }
if $failures > 0 {error make {msg: $'($failures) workflow comparisons failed'}}
}
} catch {|err| job kill $server_job; error make {msg: $err.msg} }
job kill $server_job
}