All seven layers have Go implementations with explicit resource and backend limits. Objects, lineage and assertions sit over artifacts; artifacts builds on CAS, which builds on storage. Optional adapters add wiring, run coordination, typed access, tree I/O, GC, journal watching and read-only SQL. See the layer guides for supported subsets and verification evidence for qualification limits.
Features
- Immutable content: stream blobs atomically and deduplicate by SHA-256, with streamed read integrity established at EOF.
- Discoverable snapshots: publish files and collections, filter catalog metadata, and page through fixed query snapshots with explicit limits.
- Typed data and bounded queries: define table/JSON schemas, pin immutable schema versions, and query with a supported SQL subset or JSON Pointer; capabilities and provenance coverage are explicit.
- Independent histories: record generic provenance and revisions, including explicit row/column mappings and bounded traversal across derivations.
- Assertions with authority: record typed statements about stored things, with derived current views, keyed uniqueness and capability-restricted writers.
- Crash recovery by reopening: commits are flushed before success is reported, a dead owner cannot block a root, and composite operations converge when rerun.
- Shared observation: carry context across layers, report semantic progress and counters, and keep internal work out of default reporting while retaining diagnostic access.
Layer architecture
Solid arrows point to domain dependencies; the shared context band spans all layers. Higher-level coordination composes Objects publication and lineage recording, without making either peer depend on the other for storage. Objects owns schema and selector semantics; lineage carries generic references and mapping evidence. Catalog discovery finds snapshots; Objects adapters query their content.
| Layer | Responsibilities | Guide |
|---|---|---|
| Storage | Immutable keyed blobs, streaming I/O and atomic publication. | Storage |
| Observability | Context scopes, progress, semantic counters and internal-work filtering. | Observability |
| CAS | Content identity, deduplication and verified streaming reads. | CAS |
| Artifacts | Immutable manifests, file access and catalog discovery. | Artifacts |
| Objects | Runtime schemas, typed validation and content-query adapters. | Objects |
| Lineage | Generic activities, independent revisions and row/column provenance. | Lineage |
| Assertions | Typed statements about stored things, current views and writer authority. | Assertions |
Start with the documentation index, the runnable storage/observation example, or the runnable end-to-end walkthrough.
For coding-agent integration and repository development guidance, see AGENTS.md.
Development
Requires Go 1.27 or newer. The core has no external dependencies; adapters/sqlindex is a
separate module with its own.
go test ./...
go test -race ./...
go vet ./...
go build ./...See verification and measurements for crash qualification, Windows failure coverage, the opt-in large trials, and their limitations.