Files

T

surma 4b27e9afd5 userland/git: git-compatible CLI on v2's gix code (T10)

Local workflow, smart-HTTP clone/fetch/push and public HTTPS clone/fetch match
native git in stdout and exit codes. Only WASI imports (wasi:http, exit-with-code).

2026-11-06 16:07:57 +00:00

8.3 KiB

Raw Blame History

WASI patches for Git foundation (SAU-21)

All crates here are complete crates.io source archives, with their original licenses. Cargo.toml uses local [patch.crates-io] paths. No upstream submission has been made. Changes apply only to target_os = "wasi"; native paths retain upstream behavior. No parallel, signal/interrupt, or worktree-mutation features are enabled. SAU-24 enables blocking-network-client and a custom wasi:http backend (no curl/reqwest).

gix-tempfile 26.0.0: no OS PID on WASI

Exact panic: src/forksafe.rs, ForksafeTempfile::new, field owning_process_id: std::process::id(). Ref transactions acquire gix-lock files, which register these tempfiles. This is registry fork ownership, not filename generation. std::process::id() panics with no pids on this platform on wasm32-wasip2.

Patch: a private process_id() returns 0 on WASI (no fork, registry is per component instance), and the native OS PID elsewhere. Use it in creation and both registry cleanup methods. Random tempfile naming and exclusive file creation are unchanged. Verified the unpatched ref-updating commit panics; patched commits and index writes pass.

gix-pack 0.77.0: owned pack/index/multi-index bytes

src/lib.rs: WASI's MMap is Vec<u8> and the shared mmap::read_only() helper uses std::fs::read. Other targets retain memmap2. This covers the pack data, pack index, and multi-pack-index constructors that all use this helper. memmap2 0.9.11 selects its unsupported stub on WASI; the original map call returns ErrorKind::Unsupported rather than reading a pack.

Tests generate a tiny pack via gix-pack's data::output::bytes::FromEntriesIter; the simple v2 index is encoded in the test (fanout, sorted IDs, CRCs, offsets, pack checksum, index checksum). This avoids enabling the broken streaming-input feature just for the test. A second test packs all commit/tree/blob objects, removes every loose object copy and stores the branch only in packed-refs, reopens the repository and reads history, paths, branches, blob contents, and creates/reads its index. No shell or Git executable is used.

gix-odb 0.87.0: owned loose-object bytes

Additional breakage found by the full tests: src/store_impls/loose/find.rs uses memmap2 for every loose object read and header lookup, not just packs. Without this patch logs and tree/index construction return unsupported. Use the same platform-dependent gix_pack::MMap backing and std file reads on WASI. Decompression, missing-object handling, and validation remain upstream code.

gix-index 0.58.0: read index without mmap

src/file/init.rs: WASI reads into a Vec from the already opened file and rewinds that same handle before checksum streaming. This preserves a single inode snapshot across concurrent atomic index replacements. Native mmap code is unchanged. Nonempty v2 index round trip and checksum-corruption rejection pass under WASI.

filetime 0.2.29: metadata timestamp readers

Additional runtime panic found after the mmap fixes: filetime routes WASI to its generic wasm backend, whose metadata timestamp readers call unimplemented!(). src/lib.rs now selects src/wasi.rs before that backend. It obtains modified, accessed, and optional creation times through std Metadata; inaccessible modified/ accessed times fall back to zero, because the upstream API cannot return an error. Symlink time mutation remains an explicit Unsupported error; ordinary file-time setters retain the upstream std File::set_times implementation. Metadata reads are exercised by index tests; timestamp setters and symlink mutation are not exercised.

gix-ref 0.69.0: disable packed-refs mmap through existing API

No additional vendored crate is needed. On WASI, src/repo.rs::open calls repo.refs.set_packed_buffer_mmap_threshold(u64::MAX) before any reference lookup. Upstream store/packed/buffer.rs::Buffer::open uses owned bytes for files at or below that threshold; otherwise it uses unsupported memmap2. The default threshold is 32 KiB. Native repository handles retain the upstream default.

Regression large_packed_refs_head_branches_and_log writes 65,638 bytes of sorted packed refs (1,025 branches), removes the loose HEAD branch and reopens through our helper. It checks symbolic HEAD, all branch tips/current flag, HEAD log order, and log by a non-HEAD packed branch. Before the fix this fails on WASI with HEAD: Could not open packed refs; with the fix it passes. Packed refs also consume owned linear memory; this is not a repository size limit.

Intentionally disabled / work for later cards

worktree-mutation: gix-worktree-state depends on io-close 0.3.7; it lacks a WASI File close implementation (no method close for File). Add a WASI backend or write the checkout engine using std file writes. Do not silently ignore writes.
blocking-network-client is now enabled by SAU-24: see the streaming-input patch below.
gix-commitgraph remains unpatched: src/file/init.rs maps commit-graph files with memmap2, unsupported on WASI. Current log explicitly uses use_commit_graph(false) and does not need this cache. This is NOT disabled centrally by repo::open. Later merge-base/ahead-behind/revision operations must disable or avoid commit-graph loading (including direct graph APIs), or patch that crate to read owned bytes before using it. Some high-level gix walks fall back to the object database on graph-load errors; direct graph loading can fail. Existing native Git objects/info/commit-graph files are not fixture-tested here.
No signals or threads are provided; keep interrupt and parallel off.
No large-repo memory benchmark was run. Pack + pack-index bytes are held in 32-bit linear memory, in addition to decoded objects; whole-pack reads have a materially higher resident-memory cost than demand-paged native mmap. The tiny fixture is 53 pack bytes for one blob. No peak RSS/linear-memory measurement.
Multi-pack-index reading uses the patched helper but is not fixture-tested here. Delta packs, split index, and external Git-generated fixtures are not tested.

SAU-24: gix-pack 0.77.0 streaming-input on WASI

Existing vendored crate extended (no additional crate patch). In Cargo.toml and Cargo.toml.orig, allow gix-tempfile under cfg(any(not(target_arch = "wasm32"), target_os = "wasi")). Browser wasm is still excluded; gix-tempfile remains default-features = false, so no signals are added. The foundation's PID patch applies unchanged.

Additional compile error exposed by enabling the feature: bundle writer's new_pack_file_resolver and resolve_entry explicitly used memmap2::Mmap. Change those three type annotations in src/bundle/write/mod.rs to crate::MMap, reusing the existing owned-byte WASI helper and keeping native mmap behavior. No pack/index format change. No upstream submission made.

Verified by local fake Git HTTP servers through wasi:http in cargo.build: V1 clone writes a received 227-byte / 3-object pack and real index, updates tracking refs and checks out exact bytes; V2 shallow clone writes the shallow boundary. This tests the bundle writer, not merely hand-encoded pack indexes. Pack input is streamed to a tempfile; indexing still reads a whole-pack Vec on WASI and therefore needs 32-bit memory proportional to pack size. Use clone depth to reduce history. No large-repo memory measurement.

8.3 KiB Raw Blame History