Platform Architecture

Understand how Loc.ai's components work together to provide secure, scalable infrastructure for deploying AI models at the edge.

The architecture is designed around two powerful layers: Loc.ai:Link running on edge devices and Loc.ai:Control managing operations centrally. Devices process data locally while securely syncing commands, telemetry, and results. With cloud orchestration, secure storage, and local inference, the platform delivers enterprise-grade performance, privacy, and control.

System Components

Loc.ai is divided into two connected systems:

• Loc.ai:Link: A lightweight Python runtime installed on each registered device: Windows, macOS, Linux PCs, local servers, or edge hardware such as NVIDIA Jetson and Intel NUC. Responsible for local model execution, system metric collection, and reporting state back to the control plane. Link supports multiple inference engines through a plugin system including large language models (via llama-cpp/GGUF), audio transcription, and image and audio classification (via TensorFlow Lite).

  It also interfaces with hardware sensors for telemetry collection and listens for commands via a pub/sub architecture. Loc.ai:Link is designed to operate in low-bandwidth and intermittently connected environments; it does not require a persistent connection to function.

• Loc.ai:Control: The centralised management platform, composed of two layers:

  • Backend: A FastAPI service that acts as the central coordination layer. Manages device lifecycles, orchestrates model deployments, queues commands for connected devices, and processes incoming telemetry and inference results.
  • Frontend: A React web dashboard for device monitoring, model deployment, fleet management, and real-time inference result visualisation, including filtering and CSV export.

Architectural Layers

These layers describe how local model execution, secure communication, and centralised services work together to deliver reliable AI inference and data management.

• Remote device layer: Where data is processed. Models run entirely on-device using TensorFlow Lite or GGUF/llama-cpp, keeping raw sensor and input data local. No inference payload transits the network.

• Messaging layer: A Zenoh pub/sub router carries low-latency, bidirectional traffic between agents and the control plane, with REST as a fallback for environments where Zenoh is unavailable.

• Secure communication layer: TLS encryption on every channel between the agent, the control plane, and the browser. All traffic uses TLS 1.2 or higher, with every request carrying a device-scoped API key in the Authorization: Bearer header.

• Orchestration & service layer: Cloud-hosted logic that manages device state, dispatches commands triggered from the dashboard, and streams events from devices to the dashboard. In poll-based environments, instructions are queued server-side and picked up on the device's next poll cycle.

• Data & persistence layer: Distributed storage for system state, telemetry history, and model artifacts. Model files are stored in cloud storage (GCS) and delivered to devices via signed, time-limited URLs. Telemetry and results are persisted in Firestore.

Data Flow

Interaction between Loc.ai:Control and Loc.ai:Link runs over a Zenoh pub/sub messaging layer where available, with channels chosen for the kind of traffic they carry. The design is asynchronous and event-driven, ensuring reliability in environments where connectivity is intermittent or bandwidth-constrained.

Upstream (Device → Platform)

Data type	Frequency	Contents
Status & telemetry	Every 5 seconds (default, configurable via agent configuration)	CPU, RAM, storage, temperature, GPU utilisation, inference stats
Inference results	Per inference	Classification label, confidence score, timestamp, model version
Logs	On event	Categorised log events for debugging and remote monitoring
Deployment progress	During deployment	Live progress updates while a model artifact is being downloaded and installed

Downstream (Platform → Device)

• Commands: Dashboard actions (deploy a model, start or stop inference, update the agent, change configuration) are published to the device's command topic and acted on immediately.
• Secure artifact delivery: Signed URLs for model files are delivered alongside deployment commands. The device fetches and verifies the artifact directly from cloud storage, then confirms receipt to the platform.

Real-Time Updates to the Dashboard

Events received from devices over Zenoh are forwarded to connected browsers as a live event stream, so device status, metrics, deployment progress, and inference results appear in the dashboard within moments of being reported.

Security & Trust Model

• Device activation: A device can only join the network after being activated with a valid owner-generated registration key. This key is single-use and ties the device to the issuing account.
• Per-device API key identity: After activation, each device is issued a unique, persistent API key included in the Authorization: Bearer header on every request. It authenticates and scopes all platform interactions to that specific device, so a device can only access its own topics and endpoints.
• Traffic security: All communication between Loc.ai:Link and Loc.ai:Control uses TLS 1.2 or higher. Data in transit is encrypted end-to-end.
• Infrastructure resilience: Rate limiting is applied at the API layer to protect against abuse. Inference tasks run in isolated execution contexts to prevent cross-device interference. Best-effort delivery on event channels ensures a slow consumer never blocks a producer.
• User authentication: Dashboard and API access uses OAuth2/OIDC with JWT tokens. Session management and token expiry are handled by the authentication provider.

Data Privacy

Telemetry payloads contain only health and performance metadata CPU load, memory usage, inference latency, and result summaries. Raw input data is excluded from all telemetry.

Your data stays local

Inference is processed on your edge devices. Prompts, inputs, responses, and any data processed by your models remain on your infrastructure and are never transmitted to Loc.ai's servers.

For organisations requiring complete data isolation, Loc.ai offers an Enterprise deployment option where both Loc.ai:Control and Loc.ai:Link are self-hosted, enabling a fully air-gapped, off-cloud system.