LMForge — Architecture

// architecture

Under the hood

A single Rust binary hosts an axum HTTP server, an EngineManager that supervises engine subprocesses (one per loaded model), and a ModelIndex with automatic capability detection over the on-disk store. Clients of every flavour speak HTTP to 127.0.0.1:11430; the daemon normalises and proxies their requests to the appropriate subprocess. The diagram-as-code source lives in the LMForge repo at docs/architecture/ARCHITECTURE.md.

LMForge system architecture — clients call the daemon over HTTP; middleware chain enforces auth, body limits, concurrency, and metrics; orchestration core spawns and supervises engine subprocesses (oMLX, SGLang, llama.cpp); state persists under ~/.lmforge.

HTTP Surface

Middleware (tower)

Orchestration Core

Engines (subprocesses)

axum Router

axum · tower

Single HTTP entrypoint on 127.0.0.1:11430. Three API namespaces — /v1/*, /api/*, /lf/* — plus /health, /metrics, /ui.

Thinking budget — two-call flow

Sequence diagram of LMForge's thinking-budget orchestration: the client posts a chat completion with think enabled; call 1 streams reasoning deltas live until the budget is exhausted; the daemon closes the think turn and issues call 2 with thinking disabled; the final answer streams back separately.

Reasoning tokens stream live during call 1 and stay separate from the final answer — clients get cloud-style thinking controls (on/off, budget, separate reasoning_content) against a fully local model.

// platform support

Where it runs

Platform	Hardware	Default engine	Desktop UI
macOS 13+	Apple Silicon (M1–M4)	oMLX — Metal/MLX	✓ DMG
Windows 10/11	NVIDIA GPU	llama.cpp CUDA (matched to driver)	✓ NSIS
Windows 10/11	AMD / Intel GPU, CPU	llama.cpp Vulkan / CPU	✓ NSIS
Ubuntu 22.04+	NVIDIA GPU	llama.cpp CUDA (cuda12 / cuda13) · opt-in SGLang	✓ AppImage
Ubuntu 22.04+	AMD / Intel GPU, CPU	llama.cpp Vulkan / CPU	✓ AppImage

// tech stack

Languages

RustTypeScriptSvelte 5

HTTP / async

axumtowertokioreqwesttracing

Engines

oMLX (Metal/MLX)llama.cpp (CUDA · Vulkan · CPU)SGLang (CUDA)opt-in: vLLM · ExLlamaV3

Desktop UI

Tauri 2SvelteKitVite

Infra

Docker (multi-stage)launchdsystemd --userWindows user autostart

Observability

PrometheusSSEtracing-subscriber