Open Abstractions
Interfaces for capabilities that applications keep rebuilding privately — durable work, downloads,
storage, logging, local caller identity — each written so more than one implementation can satisfy it, and a conformance
suite that tests whether one does. The first: a job is work with an identity, a state, and rules
about who may act on it right now. Go, Python and C++ agree on those rules to the byte: Go and Python finish each other's downloads in both
directions (XLANG-DOWNLOAD),
and a NAS finishes a PC's download after every process on the PC is killed
(NAS1).
Two programs are built on it, the downloader dl and the supervisor jobd, and they talk over a
local connection the kernel identifies: a caller it cannot name is refused before a request is read
(measured, Windows; in no released version yet).
Built first for desktop applications that fetch multi-gigabyte model files — image generators, local language-model
runners, and the tools that manage their models. One integration exists:
adopter-comfyui puts ComfyUI-Manager's
model downloads behind a job record without forking ComfyUI; a download it asked for kept arriving after the asking
process was killed, and the digest matched (measured, Windows). Nothing in the record mentions
a model; those applications are the first users, not the subject
(METHOD.md §14).
Real clock, 11.6 seconds. The supervisor jobd is running; the request is a text file
dropped into its wanted/ folder, the drop folder it answers; dl watch is killed at 62% and the file finishes, because the
work was never inside the process that asked. Without the supervisor the partial waits on disk and the next
dl with the same line resumes it (Adopt, step 2).
Three cases
The three places an application gains something the platform does not offer it. Each is argued, with its transcripts
and with what about it is unproven, on Cases.
The same page collects, in one list, what is missing from all three.
The conformance suite runs against your implementation rather than ours: a POSIX shell,
no Go, nothing of ours built.
Getting the programs
The redistributable packages jobd, dl, jobctl and the
openabstractions service host. Windows also includes the windowless jobdw supervisor and
Abstraction Panel: six Windows executables, four Linux/macOS programs, built from published module versions.
Find available downloads and their release-specific verification on the
redist releases page. Windows uses an MSI with a default per-user installation or an elevated Everyone installation;
Add to PATH is optional. Linux uses a per-user tarball installer. macOS packaging produces a
universal .pkg, attached only when the release's signing requirements are met.
Read each release's notes for its actual assets, signatures, notarisation and build verification.
A built package is not proof that it was installed on every supported architecture.
Installation notes explain the choices
and limitations. Checksums compare downloaded bytes with the published checksum file; they do not independently
establish origin.
For building programs directly from their public modules, see Adopt.
What is here
A layer is one contract — a page of tagged rules — with an implementation per language, in one
repository. Go, Python and C++, with nothing from a package registry: Python is standard library only; Go adds the Go
team's own golang.org/x/sys; C++ takes nothing, including a JSON library. Apache-2.0. Each repository
lists its released versions; the install lines on Adopt name one each.
| layer | what it is | languages | evidence |
|---|
job |
the record and the rules: lease, epoch, checkpoint, intent. CONTRACT.md, where every rule carries a tag a scenario can cite; job.thrift, the record's fields and types in no language's syntax, written in a profile of Apache Thrift's IDL |
Go, Python, C++ |
three implementations replay one scenario corpus and their transcripts are diffed byte for byte: BEHAVIOUR1, CONFORM1 |
download |
a job whose work is bytes: artifact, sources, sink; resume from the proven prefix; verify; deliver. dl is its command line |
Go; Python fetches and adopts orphans; C++ fetches, and never sweeps for them |
XLANG-DOWNLOAD Go and Python finish each other's downloads; RESUME1; four downloaders, interrupted; a server that lies |
jobd |
the supervisor: finishes work nobody is watching, adopts orphans, verifies and delivers. Packaged for a Synology NAS (addon-synology) and as a container (docker-jobd) |
Go, untagged |
SUPERVISOR1; NAS1 a NAS finishes a PC's download after every process on the PC is killed |
facade |
Discover() once; then Download(), Jobs(), Storage(), Log(). The application names no implementation |
Go |
one program, three providers |
Earlier, each one repository, Go unless said:
cas compare-and-set over a file, Go, Python and C++ — six writers in three languages, 600 changes, none lost (CAS-MIXED1) ·
watch change without polling, Go, Python and C++ ·
storage bytes at rest, named by digest, Go and C++ ·
model hf://org/repo#Q4_K_M to a digest before a byte moves, Go and Python ·
config how a machine answers "which store" ·
logging the part of a log line that leaves the process ·
asks a question a machine may put to a person ·
identity which program is on the other end of a connection — user, path and code signature, each on a nine-rung ladder; contract tagged in the tree, not yet in its tag ·
rights who may hold the machine awake (AWAKE1, Windows) ·
adopter-comfyui ComfyUI-Manager's downloads behind a job record, unreleased ·
polite-monitor a Windows window listing what the machine is doing ·
research the prior art read before drawing any of it ·
abstractions how an interface here is drawn, and which gaps are worth drawing one for (METHOD.md §14).
Three things that must not blur
A provider is whatever actually does the work, chosen by the machine
rather than named by the application: our own code, dlc over curl, or Windows'
Background Intelligent Transfer Service (BITS), the downloader every Windows machine already runs. An application is
written against the contract and names none of them
(one binary, three providers); a provider works with no service installed (that
table, first row); correct means the scenario corpus every implementation is replayed through
(BEHAVIOUR1),
never what the supervisor does.
Four defects, and the condition each needed to appear
A program that fetches one file, on one machine, in one language, from a source that publishes a digest, does not need
any of this. Each defect below needed a second something to appear — a second implementation, a second writer, a second
reader, or a server that lies — and each is recorded where it was found.
| the condition that failed | what was measured here |
|---|
| A server that lies, and no digest to catch it. The partial file's length is the resume point, and bytes on disk are taken as evidence of what was received |
Against a server that answers a resume with bytes from the wrong offset and a Content-Range that lies to match, curl -C - writes a complete, corrupt file and exits 0. So does dl without --digest. The checkpoint is the bytes proven, not the bytes written; a successor resumes from what was hashed. The verdicts. |
| A second implementation. The retry table — which HTTP statuses mean never and which mean later — written once, with nothing to disagree with it |
The Go and Python implementations disagreed on four of eighteen statuses. A downloader written over curl, sharing no transfer code, was then measured against dl: all eighteen agreed, and 50 of 54 wire cases ended with the same verdict and bytes; the four that differed were the same case, a partial nothing could vouch for. The matrix. |
| A second writer. The application on launch and the service unit both resume the same partial |
The lease and epoch exist for this and still missed two things: the same owner may re-claim its own live lease and the epoch rises, and update on the store — the directory of job records on disk — had no compare-and-set behind it. One process raced itself and reported a delivered download as a failed install, one run in eight. The compare-and-set is fixed; the re-claim is undecided. |
| A second reader. A state file with one reader; its format is whatever the writer does |
Three implementations passed sixty-one spec checks while escaping JSON three different ways, because no fixture held an ampersand or an accent. One ampersand in a URL produced three different records. One escaping rule now, with a fixture that carries the ampersand. The rule. |
Where none of those four is present, a Range loop and a sha256 at the end are
correct, and this site does not argue with it.
What an application pays. Get(url, path) and Wait, and no word from the
reference; the vocabulary is for whoever writes the thing that answers. The commands are on Adopt.
Ancestry
Every word on the reference names its ancestor or says it has none;
the envelope rows were re-read in the primary text on 2026-09-09.
| term | meaning | ancestor | |
|---|
| job | work with an identity, a state, and rules about who may act on it now; spec is opaque to the layer and kind says who may read it | google.longrunning.Operation, AIP-151: metadata is an Any whose type is declared beside the method | match |
| lease | the time-limited right to work on a job; lapses when the holder stops renewing | Chubby | match |
| epoch | rises by one on every claim; a write carrying an old one is refused | fencing token (Kleppmann, ch. 8); If-Match | match |
| checkpoint | what a successor inherits — the bytes proven, not the bytes written | Temporal's activity heartbeat | match |
| envelope.schema | which schema the opaque halves follow: a name, and nothing dereferences it | google.protobuf.Any's type_url, whose own doc comment says not to contact type URLs; CloudEvents dataschema, optional, a URI naming the schema of data | match on the pattern; divergence: both ancestors are URL-shaped, ours is a grammar that cannot spell a URL |
| envelope.actions | what may be asked of a job of this kind, declared on the record and matched by a supervisor that never built it | Kubernetes APIResource.verbs, published per resource at /apis/<group>/<version>; the Windows Service Control Manager's dwControlsAccepted, which sends a service only the controls it declared; link relations, RFC 8288 §2.1 | match; divergence from AIP-151, whose Operations service fixes five verbs where ours is open per kind; D-Bus Introspect is a call at runtime, ours a value in the record |
| the three answers | unavailable · forbidden · unknown: nothing can do this now, something said no, no answer arrived; a provider must never collapse them | SMTP 4yz/5yz for the first two | the third has none |
Measured
Each with its transcript on Evidence.
dl is the only one of four downloaders that resumes without being told to, and it is not the one that resumes best. 8 GiB killed at 3 GiB and restarted with the same command: dl re-fetched 11.6 MB, curl, wget and aria2c the whole 8 GiB. Told to resume, curl -C - and wget -c waste less than we do. The table.- No throughput claim is made. Two of our own loopback benchmarks on one machine disagree by 2.7×; the tables are withdrawn, and nothing on this site rests on a throughput number. What was withdrawn and why.
Stage
0.x. Every module is versioned 0.y.z, which under
Semantic Versioning §4 means the public API may change between releases.
- Go: every layer carries a released version; the supervisor
jobd carries none, and nothing marked in the tree above has been released yet.
- Python: source only, on no index. C++: source only, no tagged release.
- Integrations: ComfyUI-Manager and a Synology package, both built here. Neither has a published release file yet.
- Signing: nothing is signed.
sum.golang.org proves the bytes are the ones everyone else fetched, not who wrote them.
- Governance: one maintainer today, two organisation owners; the succession plan is written and needs nobody's permission (
GOVERNANCE.md). Faults go through SECURITY.md.