Open Abstractions
Inference
Build assistants, semantic search, voice features, and image and video generation through OA. Each task connects to a compatible model provider. Applications specify where execution is allowed; the service manages provider connections and credentials.
Current status. Current source implements all six modality profiles and the operator profile with controlled integration evidence. The generated schemas linked below and the standalone inference repository are the public technical surface. The loopback gateway and Realtime vendor connector now live in the Go adapter module in the integration source; the core module owns policy and service behavior. Provider interoperability and release qualification vary. Use Coverage for measured language and platform results; profile presence on a host is explicit and may cover only part of this family.
Choose by result
Six service profiles cover chat, embeddings, transcription, speech, live voice and images. Video generation uses a durable inference job: submit a prompt and duration, follow progress, then retrieve the generated clip. A configured provider must support the requested task. See how video jobs survive interruptions.
abstraction.inference/chat@1Stream an assistant answer, tool call, or structured response. Messages can contain text, tool results, and authorized image digests for a vision-capable host.
You Summarize the error in this screenshot.
Model The build stopped because… streaming
Use it for assistants, coding help, classification, and tool-driven workflows.
abstraction.inference/embed@1Convert up to a bounded batch of texts into numeric vectors in one call. Inputs and vectors remain absent from the inference audit.
Use it for semantic search, retrieval, clustering, and similarity.
abstraction.inference/transcription@1Turn stored audio into ordered text segments or words with millisecond timestamps. The request names an authorized content digest.
“Open the project settings.”
Use it for meeting notes, subtitles, searchable recordings, and voice commands.
abstraction.inference/speech@1Stream synthesized audio from bounded text, then receive a digest-addressed final delivery whose bytes the client can verify.
Use it for spoken notifications, narration, accessibility, and voice responses.
abstraction.inference/live@1Send ordered PCM audio frames, commit the input turn, and observe transcript and audio output for one final response.
Use it for push-to-talk commands and one-turn voice interactions. A conversation creates a new session for each response. A program that speaks OpenAI's Realtime API reaches the same session through GET /v1/realtime on the gateway window.
abstraction.inference/image@1 · image@2Generate an image from a prompt or edit an authorized image and mask. Image@1 names an explicit model. Image@2 also accepts a saved selection policy. Progress and ordered results lead to typed content deliveries.
Use it for concept art, variations, masked edits, and generated assets.
A policy is managed through selection-operator@2 for a program in the bound account. A candidate may pin a current logical provider declaration; managing another program requires a separate grant.
Three cooperating abstractions
What is this model?
Resolves a portable registry and repository reference into a manifest. The manifest can describe files, revisions, and how content is obtained.
abstraction.model/resolver@1Model catalogue entryWhere can it run?
Lists eligible models and hosts, then picks a host that matches the model, modality, guarantees, and placement.
abstraction.router/router@1Router schemaHow does this call run?
Checks admission, rights, content, capacity, credentials, and budget. It owns the upstream request and returns the modality result.
abstraction.inference/*@1Inference schemaAn application can use Model without running inference, and it can request inference with a model name already known to its workflow. The runtime composes the three when a call needs all of them.
Execution boundary
A model engine on the receiving machine performs the call. local-only@1 constrains selection to this placement.
The local OA service relays to a declared, mutually trusted runtime. The selected binding records remote execution.
hosted-allowed@1 lets the router consider a configured hosted provider after eligible local hosts.
Vendor keys, base URLs, and request headers remain inside the service. Hosted credentials are named in requests and applied inside the receiving service. A remote runtime resolves digest references from its own authorized content store. Digest transfer between stores remains future work.
Pick the lifetime
Start returns an operation. Observe reads retained deltas from a cursor. Cancel explicitly stops the operation.
gap.The application encodes the generated inference job request and submits it through abstraction.job/acceptance@1.
Operator control
An authorized operator can list, add, and remove hosts, inspect profile readiness and current credential spend, issue local gateway keys, and read the inference audit.
Removing a host leaves already admitted operations running to their defined end. Provider declaration and generation pinning prevent an existing binding from silently becoming a different provider.
Each host advertises specific profiles. A chat-only provider remains chat-only.
Developer API
machine := facade.Discover()
chat, err := machine.ResolveInference(ctx, facade.Requirements{
Scope: facade.ScopeLocal,
})
// Complete is Stream's non-streaming twin, observing a call to its
// end and folding the deltas into one reply.
reply, err := chat.Complete(ctx, wire.Request{
Model: model,
Messages: messages,
Guarantees: []wire.RequestGuarantee{wire.RequestGuaranteeLocalOnly},
})The facade currently exposes direct resolvers for chat, embeddings, and inference operator control. Generated modality clients bind transcription, speech, live, and image service endpoints.
Start · Observe · CancelGo helpers: Stream, CompleteEmbedGo helper: VectorsStart · Observe · CancelGo helper: TranscribeStart · Observe · CancelGo helper: SynthesizeStart · Append · Commit · Observe · CancelOne committed turn per sessionStart · Observe · CancelGo helper: GenerateEvery streaming profile uses bounded pages, opaque operation identifiers, caller-scoped visibility, explicit cancellation, and typed terminal outcomes. The developer reference lists the profiles beside their entry points.
The loopback window
A program that only knows a base URL and an API key reaches the same rights decision and audit as the native route. The window listens on loopback only and is closed by default.
| Route | Carries | Contract rule |
|---|---|---|
POST /v1/chat/completions | OpenAI chat, including tool calls and streaming | INF-W1 – W7 |
POST /v1/messages | Anthropic messages | INF-W1 – W7 |
GET /v1/models | The bound key's eligible models | INF-W1 – W5 |
POST /v1/embeddings | Embedding vectors | INF-W1 – W7 |
POST /v1/audio/transcriptions | Audio to text | INF-W1 – W7 |
POST /v1/audio/speech | Text to audio | INF-W1 – W7 |
POST /v1/images/generations | Image generation | INF-W1 – W7 |
POST /v1/images/edits | Authorized image edits | INF-W1 – W7 |
GET /v1/realtime | One live@1 session per WebSocket connection, PCM16 at 24000 Hz | INF-W8 |
On the Realtime route, abstraction.inference/complete on the host the named model routes to is decided for the bound program before the WebSocket upgrade completes; a refusal is answered with the same status the other eight routes use, on the unupgraded connection. Every refusal after the upgrade is one error event, then the close.
A declared native abstraction.inference/chat@1 provider with an explicit model allowlist — the model host is one — becomes a mediated local resolver candidate after the runtime's own, reached through the same generated client as any other host. Resolve itself never blocks: the first mediated call that needs an on-demand provider waits for its readiness inside the caller's own deadline, and never longer than the runtime's launch budget of 30 seconds. A caller whose deadline passes first reads unavailable with reason activating:<name>; the launch continues, and the next call finds the provider ready or refused with its reason.
Adoption choice
| Your application needs | Start with | Plan for |
|---|---|---|
| A complete text answer | ResolveInference + Complete | Typed refusals, host permission, model eligibility |
| Incremental text or tool calls | Stream or Start/Observe | Cursor gaps and cancellation when iteration ends |
| Semantic vectors | ResolveEmbeddings + Embed | Input count, dimensions, and provider support |
| Audio or image content | Storage content writer/reader plus the modality client | Digest authorization, media bounds, output delivery |
| One push-to-talk response | Live.Start/Append/Commit/Observe | PCM format, frame sequence, one response session |
| Work that must survive process exit | Generated inference job request through Jobs | Persist request identity and receipt before waiting |
| An OpenAI- or Anthropic-compatible program | Operator-managed loopback gateway window | Program-bound local key, explicit host rights, supported route subset |
Generated records establish the API shape. Current platform, language, provider, and packaging evidence remains in Coverage. The catalogue entry records the inference layer alongside its dependencies.