Your Own Software

How to integrate Locai:Link into your host application as an on-device inference sidecar. This guide is for engineers building applications that delegate model inference to Link running on the same machine. Link's public surface is OpenAI-compatible, so the integration is shallow.

If you are an operator deploying Link to a fleet of devices, see Deployment and lifecycle below.

What Link is

Link is a long-running on-device inference agent. It is a plugin-based platform capable of running a range of models on the user's machine. Today it ships plugins for language models, audio transcription, and image and audio classification, and new modalities can be added via the plugin surface.

This guide focuses on the language-model integration path, the most common host-app use case, where Link exposes an OpenAI-compatible HTTP endpoint that your application talks to exactly like chat.completions, except the call never leaves the device. Models, inference, and any user data stay on-host unless your application explicitly forwards them.

Two surfaces matter to integrators:

• HTTP API. A fixed loopback discovery endpoint on 127.0.0.1:20505 (/healthz, /models) plus OpenAI-compatible inference on a per-model port that Control chooses at deploy time. 8100 is the factory default and what most docs show, but the real port comes back from /models, so do not hardcode it.
• Lifecycle. How Link gets onto the device, how it starts up, and how it self-updates without your application needing to participate.

The rest of this guide covers both.

Quick start

A single end-to-end example. This assumes Link is already installed and serving a model (see Deployment and lifecycle for the install side).

1. Confirm the agent is up

curl http://127.0.0.1:20505/healthz

{
  "version": "1.0.17",
  "uptime_seconds": 142,
  "currently_serving": true,
  "model_id": "llama-3.1-8b"
}

If the agent is not reachable, or currently_serving is false, your application should surface a useful state to the user rather than attempt a chat completion. See Detecting the agent.

2. Send a chat completion

curl http://127.0.0.1:8100/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "llama-3.1-8b",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

# Python: use any OpenAI-compatible client.
from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8100/v1",
    api_key="locai-local",  # any non-empty value; no auth on loopback
)

response = client.chat.completions.create(
    model="llama-3.1-8b",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

// JavaScript / TypeScript
const response = await fetch("http://127.0.0.1:8100/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "llama-3.1-8b",
    messages: [{ role: "user", content: "Hello!" }],
  }),
});
const data = await response.json();
console.log(data.choices[0].message.content);

That is it. The rest of this guide is detail on each surface.

HTTP API

Endpoints

Endpoint	Port	Method	Purpose
/healthz	20505	GET	Agent state: version, uptime, current model
/models	20505	GET	Per-pipeline model list and serving state
/v1/models	8100	GET	List served models (OpenAI-compatible)
/v1/chat/completions	8100	POST	OpenAI-compatible inference
/health	8100	GET	Liveness probe of the inference proxy

The split (20505 for agent state, 8100 for inference) is deliberate. The agent answers /healthz even when no model is loaded, so your host app can distinguish "Link is offline" from "Link is running but no model is ready."

The agent-state endpoints on 20505 bind loopback-only and reject requests whose Host or Origin header is not a loopback name (403). The inference port (8100) binds to whatever interface the served model's config specifies, typically loopback, but it can be a LAN interface for deployments that intentionally expose inference off-host. Treat the inference port as the public port your host app is told about at registration time, not as a constant: it is per-model-configurable.

Port 20505 is reserved

The health server binds it first, so a model configured to serve on 20505 will start llama-swap fine but its front-door proxy will fail to bind and /v1/chat/completions will never be reachable. Pick a different port when configuring a served model in Control.

/healthz response

{
  "version": "string",            // SemVer, e.g. "1.0.17"
  "uptime_seconds": 142,          // seconds since the agent process started
  "currently_serving": true,      // legacy, see note below
  "model_id": "llama-3.1-8b"      // legacy, nullable, see note below
}

model_id is null when nothing has been started via START_SERVING yet.

This is a stable contract. The shape is mirrored in the Rust shared/ crate so every consumer reads the same fields.

currently_serving and model_id are legacy

These fields are a single-model legacy view. They only reflect the last pipeline that was started via an explicit START_SERVING command, and cannot represent Link's multi-model serving via llama-swap. For accurate per-pipeline serving state, use GET /models below. The fields stay in place for backwards compatibility with existing integrators.

/models response

{
  "models": [
    {
      "id": "llm_server",             // pipeline id (stable handle)
      "alias": "smollm-135m",         // human-readable label
      "port": 8123,                   // llama-swap serving port (nullable)
      "host": "127.0.0.1",            // bind interface
      "is_serving": true              // currently accepting traffic
    }
  ]
}

One entry per configured servable-model pipeline (pipelines whose source args include a model_path). Snapshotted lazily on each request so the payload is always current, with no push updates required. port is null when the pipeline's args do not set one, since a pipeline can be servable-capable without a currently assigned port.

is_serving is true only when the pipeline is running and its source has mode=serve (running in inference does not count). When multiple pipelines are configured on the same port, llama-swap multiplexes them and rotates the loaded model on demand; all such pipelines report is_serving=true because they are registered with the swap manager.

Inference (/v1/chat/completions)

Exactly the OpenAI shape. Refer to OpenAI's API reference for the full request and response schema. Notable behaviours specific to Link:

• Streaming via stream: true returns SSE (text/event-stream). Each event is a chat.completion.chunk object with delta content.
• usage block. When you set stream_options.include_usage: true, the final SSE chunk carries usage. Link relies on this for its own telemetry. If your client never asks for include_usage, Link falls back to counting delta.content events to estimate token throughput.
• Disconnects. If your client disconnects mid-stream, the proxy catches the socket error, logs the event, and stops emitting.
• model field in responses. Link returns the model file path (for example models/SmolLM-135M.Q4_K_M.gguf) rather than the alias you passed in. If you match on the response model, use a prefix or substring rule rather than an exact-string compare.

Listing available models (/v1/models)

{
  "object": "list",
  "data": [
    {
      "id": "smollm-135m",
      "object": "model",
      "owned_by": "llama-swap",
      "created": 1783100713
    }
  ]
}

id is the alias registered with llama-swap (the same value that appears in alias on GET /models). owned_by is always "llama-swap", since this endpoint is llama-swap's own list, proxied through unchanged. Only models currently loaded by the agent appear here.

Detecting the agent

Your host app cannot assume Link is running. The user may not have installed it, may have uninstalled it, may have paused it, or it may have crashed mid-session.

Recommended pattern:

async function probeLink(timeoutMs = 1500) {
  try {
    const ctrl = new AbortController();
    const t = setTimeout(() => ctrl.abort(), timeoutMs);
    const r = await fetch("http://127.0.0.1:20505/models", { signal: ctrl.signal });
    clearTimeout(t);
    if (!r.ok) return { available: false };
    const { models } = await r.json();
    const serving = models.filter((m) => m.is_serving);
    return { available: true, serving };
  } catch {
    return { available: false };
  }
}

• No response within ~1.5s. Assume Link is not running. Direct the user to install or start it (link to your help page; do not pretend the problem is elsewhere).
• Reachable, serving is empty. Link is up but no model is currently serving. Direct the user to enable a model via Control.
• Reachable, serving has entries. Safe to proceed with inference. Use the entries' port and alias to target the right llama-swap instance.

Probe on app launch and again before each inference batch. Do not poll every second from a web client; once per user interaction is enough.

CORS

Link's inference endpoint (8100) sits behind a reverse proxy (ServingProxy) that handles CORS for browser-based host apps.

Allowlist management. Allowed origins are a per-model deploy-time setting, the cors_allowed_origins arg on the served model, delivered as part of that model's pipeline config when Control deploys it. To add your host app's production and (if relevant) staging origins, open a request with the URLs and the Locai team will roll the change into the model config on your fleet. See the reference section for how to reach the team.

Once on the allowlist, your browser-based app can call POST http://127.0.0.1:8100/v1/chat/completions directly. Preflight OPTIONS is handled, the Access-Control-Allow-Origin header is echoed for the allowed origins, and SSE streaming works.

Origins not on the allowlist fail the browser's CORS check and the request never reaches Link. Native (non-browser) clients on the same machine ignore CORS and can talk to Link freely.

Deployment and lifecycle

Link is a long-running background agent that your host app talks to over the loopback HTTP surface described above. How Link gets onto the device and started is coordinated with the Locai team per integration. See the sections below for what is shipped today.

Install path

Link is distributed today as a signed bundle archive from the public releases page:

https://github.com/locai-co-uk/locai-link/releases/latest

Assets are per-platform (macOS arm64, Linux x86_64, Windows x86_64) with an accompanying .sha256 for each. The typical integration pattern is that your installer bundles Link alongside your own app, extracts the archive to a known location, and starts the launcher before your app's first probe against /healthz. Your host app should not re-fetch or re-install Link at runtime.

Standalone installer now available on macOS and Linux

A user-facing standalone installer (a macOS .pkg and a Linux tarball installer, plus a GUI Setup Assistant that handles device registration) is now available; see Using the Link App. A Windows GUI installer is not shipping yet. The integration surface described here (localhost HTTP, /healthz, CORS allowlist) is unchanged.

Starting Link

The exact bootstrap flow (device registration, config provisioning, and first launch) depends on your integration path and is coordinated with the Locai team when you onboard. Once Link is running, your host app can treat /healthz on 127.0.0.1:20505 as the ready signal regardless of how it was started.

The launcher binary at the root of the extracted archive (<install-root>/locai-link) is the stable entry point for the agent lifecycle. It resolves the current version pointer and manages OTA swaps, but host apps generally should not invoke it directly. Reach out to the Locai team for the shape that fits your host app.

OTA updates

Link self-updates transparently:

• The launcher binary the OS invokes is stable, a small entry point that dispatches to a versioned runtime living alongside it.
• When a newer version is published, the launcher downloads it, verifies its SHA-256 against the sibling .sha256 checksum file published alongside each release asset (and, on macOS, additionally runs codesign --verify over the extracted bundle), swaps the active version, and restarts the runtime, all while your host app is using it.
• The version field in /healthz reflects the currently running runtime. If your host app cares about a specific Link version, that is the field to check.

You do not need to participate in updates. If a request in flight straddles a swap, your host app sees a brief connection drop; a normal retry against /healthz confirms the agent is back up, usually within 2 to 3 seconds.

Detecting whether Link is installed (without trying it)

The /healthz probe is the most reliable indicator that Link is running and reachable. Since a host app that bundles Link knows where it dropped Link on disk, filesystem detection is not usually required. The probe is enough for almost every use case.

If you still need to distinguish "installed but not running" from "never installed", check the path your own installer chose to extract Link into. There is not a system-wide install-detection convention yet; the shipped surface today is your installer's choice of layout.

Identity and privacy

Device registration

Each device running Link is registered with the Locai control plane on first install. The user authenticates (password or SSO device flow) and the agent receives an API key tied to the device. Your host app does not see this credential and does not participate in registration.

What stays on-device

By default:

• All inference requests and responses
• All model weights
• All user data passing through inference

Nothing in the list above reaches the control plane. Inference input and output stay on-host end-to-end.

What gets reported to Control

• Operational metadata: device id, OS and architecture, hostname, Link version
• System metrics: CPU, RAM, temperature, storage
• Lifecycle and command status: agent online or offline, model deployment progress, command success or failure

This is the same telemetry shape regardless of which host app is using Link. Nothing your host app sends through /v1/chat/completions is included.

Error handling

Symptom	Likely cause	What your host app should do
Connection refused on /healthz	Link not running	Direct the user to install or start Link
/healthz reachable, currently_serving: false	Link up, no model loaded	Direct the user to load a model
/v1/chat/completions returns 502	Inference backend (llama-swap) crashed	Retry once after 2s; if still failing, surface the error
Stream disconnects mid-response	Network blip or upstream crash	Retrying the same request is safe; the cancelled response is discarded
Version mismatch (older version than expected)	OTA update has not run yet	Continue if your client is compatible; warn the user if not

Retry strategy

• Idempotent operations (/v1/models, /healthz): retry freely.
• Inference (/v1/chat/completions): retry on 502 or connection reset only when you can re-issue the same prompt. For streaming responses already partially delivered, treat the partial as the final output and do not retry mid-stream.
• Exponential backoff with 1s, then 2s, then 4s, capped at 3 attempts, is the recommended default.

Versioning compatibility

Link follows SemVer:

• Patch (1.0.17 to 1.0.18): bug fixes, no behaviour change. Safe to pick up automatically.
• Minor (1.0.x to 1.1.0): additive features. Existing endpoints and shapes are preserved; new fields may appear.
• Major (1.x.x to 2.0.0): breaking changes. A migration guide is published alongside the release.

Your host app should treat anything ^1.0 as compatible until major release notes say otherwise.

Reference

• Endpoint summary
• CORS allowlist process: request a new origin via the Locai team
• Releases page: where the current signed bundle is published
• Privacy posture: what to communicate to your end users

Getting help

• Integration questions: support@locai.co.uk
• Issue tracker: github.com/locai-co-uk/locai-link/issues