// by the numbers

130+
Curated catalog
chat · vision · embed · rerank · code · MTP
3+2
Inference engines
oMLX · llama.cpp · SGLang, opt-in vLLM · ExLlamaV3
3
API surfaces
OpenAI /v1 · Ollama /api · native /lf
6
Model roles
chat · vision · embeddings · rerank · code · reasoning
1 cmd
Install
engine auto-selected for your hardware
3–60s
Cold load latency
depends on model + engine

// what it does

Thinking models with budget cap

01 · 💭

Thinking models with budget cap

First-class reasoning support — stream chain-of-thought live, cap its depth with thinking_budget, and keep it separate from the final answer.

Per-request `think` on/off where the model allows it; dedicated `:thinking` / `:reasoning` catalog models stay locked on (the Playground toggle grays out so you cannot accidentally disable native reasoning). Call 1 streams reasoning deltas up to `thinking_budget`; the daemon then closes the `<think>` turn and issues call 2 with thinking disabled for the answer. A budget floor prevents blank replies when reasoning would eat the whole token budget.

Multi-model orchestration

02 · 🧠

Multi-model orchestration

Chat, vision, embeddings, and rerank served simultaneously — each model in its own engine subprocess with its own lifecycle.

Load and serve several models on demand with per-model `keep_alive`, LRU eviction under VRAM pressure, and auto-load favourites at startup. Requests for the wrong role are rejected cleanly (embed model on chat, chat model on embed) — clients only name the model they want and LMForge handles load, unload, and memory.

Hardware-aware engine tiers

03 · 🎯

Hardware-aware engine tiers

Profiles CPU, GPU, VRAM, and drivers on first run, then installs and activates the best engine — default engines auto-selected, risky ones opt-in.

Apple Silicon → oMLX (Metal/MLX). Windows / Linux NVIDIA → llama.cpp CUDA matched to your driver (cuda12 vs cuda13). AMD / Intel or CPU-only → llama.cpp Vulkan / CPU. Linux NVIDIA with headroom can opt into SGLang for high concurrency; vLLM and ExLlamaV3 are experimental installs. VRAM-aware admission protects against over-commit, and `lmforge doctor` shows what is installed and active.

Built-in Playground

04 · 💬

Built-in Playground

Try any model in-app — thinking toggle, chat vs thinking sampling profiles, and an Advanced popover for temperature / top_p / top_k / penalties.

The fastest way to validate a pull, compare models, and tune decoding — no curl or third-party UI required. Sampling profiles switch automatically between chat and thinking modes, and every knob in the Advanced popover resets to the active profile in one click.

Curated model library

05 · 📦

Curated model library

130+ recommended, ready-to-run models with one-click pull — or paste any Hugging Face repo.

Shortcuts like `qwen3:8b:4bit` resolve to the right artifact for your engine — MLX on Apple Silicon, GGUF elsewhere — so there is no hunting for file formats or quantizations. Filter by role (chat · embed · rerank · vision · code), watch the always-visible download progress line follow you across screens, and add custom repos when you need something off-catalog.

Capability detection + self-heal

06 · 🔍

Capability detection + self-heal

On pull, LMForge inspects weights and templates and records what each model can actually do — and re-probes when the detector improves.

Chat, vision, embeddings, reranking, thinking, `native_reasoning`, MTP, stop tokens, embedding dims — all detected automatically and exposed as chips in the UI and via `/v1/models`. When the daemon upgrades and the detector version advances, capabilities are re-derived without forcing a re-download, so the UI and API stay honest after every update.

// also in the box

Vision, embeddings, rerank, code One local server covers the building blocks for search, RAG, and assistants — image chat (URL or base64), batched embeddings, rerankers, code models.
OpenAI + Ollama compatible Drop-in for both ecosystems. Point OPENAI_API_BASE=http://localhost:11430/v1 — usually a one-line change.
Flexible storage Move the model library to another drive anytime — adopt an existing folder, or delete & re-download in the background with retry, surviving restarts.
Built-in observability Request metrics, live logs, and Prometheus /metrics — plus live GPU/CPU/memory via /lf/sysinfo. No extra tooling.
One folder, clean uninstall Binary, engines, models, logs, config — all under ~/.lmforge. One folder to back up, exclude, or delete; uninstall scripts keep or purge models as you choose.
Vision, embeddings, rerank, code One local server covers the building blocks for search, RAG, and assistants — image chat (URL or base64), batched embeddings, rerankers, code models.
OpenAI + Ollama compatible Drop-in for both ecosystems. Point OPENAI_API_BASE=http://localhost:11430/v1 — usually a one-line change.
Flexible storage Move the model library to another drive anytime — adopt an existing folder, or delete & re-download in the background with retry, surviving restarts.
Built-in observability Request metrics, live logs, and Prometheus /metrics — plus live GPU/CPU/memory via /lf/sysinfo. No extra tooling.
One folder, clean uninstall Binary, engines, models, logs, config — all under ~/.lmforge. One folder to back up, exclude, or delete; uninstall scripts keep or purge models as you choose.
Speculative decoding (MTP) GGUF models with Multi-Token Prediction heads get llama.cpp draft-MTP speculative decoding automatically when VRAM allows — higher tokens/sec, same quality.
Live download progress A single always-visible progress line under the header for every pull and background migration — it follows you across screens.
Reliable background service Starts at login (launchd / systemd --user / Windows autostart), restarts from the app in seconds, and surfaces engine failures as structured last_errors.
Secure by default Binds to localhost only; network sharing is explicit via API key and trusted CIDR ranges. Engine downloads are sha256-verified against pinned checksums.
Speculative decoding (MTP) GGUF models with Multi-Token Prediction heads get llama.cpp draft-MTP speculative decoding automatically when VRAM allows — higher tokens/sec, same quality.
Live download progress A single always-visible progress line under the header for every pull and background migration — it follows you across screens.
Reliable background service Starts at login (launchd / systemd --user / Windows autostart), restarts from the app in seconds, and surfaces engine failures as structured last_errors.
Secure by default Binds to localhost only; network sharing is explicit via API key and trusted CIDR ranges. Engine downloads are sha256-verified against pinned checksums.
Vision, embeddings, rerank, code One local server covers the building blocks for search, RAG, and assistants — image chat (URL or base64), batched embeddings, rerankers, code models.
Speculative decoding (MTP) GGUF models with Multi-Token Prediction heads get llama.cpp draft-MTP speculative decoding automatically when VRAM allows — higher tokens/sec, same quality.
OpenAI + Ollama compatible Drop-in for both ecosystems. Point OPENAI_API_BASE=http://localhost:11430/v1 — usually a one-line change.
Live download progress A single always-visible progress line under the header for every pull and background migration — it follows you across screens.
Flexible storage Move the model library to another drive anytime — adopt an existing folder, or delete & re-download in the background with retry, surviving restarts.
Reliable background service Starts at login (launchd / systemd --user / Windows autostart), restarts from the app in seconds, and surfaces engine failures as structured last_errors.
Built-in observability Request metrics, live logs, and Prometheus /metrics — plus live GPU/CPU/memory via /lf/sysinfo. No extra tooling.
Secure by default Binds to localhost only; network sharing is explicit via API key and trusted CIDR ranges. Engine downloads are sha256-verified against pinned checksums.
One folder, clean uninstall Binary, engines, models, logs, config — all under ~/.lmforge. One folder to back up, exclude, or delete; uninstall scripts keep or purge models as you choose.

// how it gets used

01

The 5-minute evaluation

From nothing to a working private AI endpoint.

Install Core + UI (one command each) — the engine auto-starts

Chat in the Playground — try the thinking toggle on a reasoning model

Open Observability while requests flow

01

Install Core + UI (one command each) — the engine auto-starts

02

Pull a small 4-bit chat model from the Library; watch the live progress line

03

Chat in the Playground — try the thinking toggle on a reasoning model

04

Point an existing OpenAI tool at http://localhost:11430 — one-line URL change

05

Open Observability while requests flow

Pull a small 4-bit chat model from the Library; watch the live progress line

Point an existing OpenAI tool at http://localhost:11430 — one-line URL change

02

Interactive chat from the CLI

Zero-API integration — just install and chat.

Run install-core.sh (or the PowerShell one-liner on Windows)

lmforge run qwen3:8b:4bit

01

Run install-core.sh (or the PowerShell one-liner on Windows)

02

lmforge pull qwen3:8b:4bit

03

lmforge run qwen3:8b:4bit

04

Chat in the REPL — model unloads after the keep-alive TTL

lmforge pull qwen3:8b:4bit

Chat in the REPL — model unloads after the keep-alive TTL

03

Integrate from any app over HTTP

Reuse your existing OpenAI SDK code.

Daemon already running as a system service

export OPENAI_API_KEY=none (loopback bypasses auth)

Cold-load latency paid on first call, then warm

01

Daemon already running as a system service

02

export OPENAI_API_BASE=http://localhost:11430/v1

03

export OPENAI_API_KEY=none (loopback bypasses auth)

04

All your OpenAI SDK code just works — chat, embeddings, vision, thinking

05

Cold-load latency paid on first call, then warm

export OPENAI_API_BASE=http://localhost:11430/v1

All your OpenAI SDK code just works — chat, embeddings, vision, thinking

04

Power a downstream service like DocIntel

Run multiple models concurrently for a RAG pipeline.

Service startup: POST /lf/model/switch for chat, embed, rerank, VLM models

Consume /v1/chat/completions + /v1/embeddings + /v1/rerank

Models stay resident under the VRAM budget; LRU eviction handles overflow

01

Service startup: POST /lf/model/switch for chat, embed, rerank, VLM models

02

Subscribe to GET /lf/status/stream for load progress UI

03

Consume /v1/chat/completions + /v1/embeddings + /v1/rerank

04

Scrape /metrics for per-model latency and load history

05

Models stay resident under the VRAM budget; LRU eviction handles overflow

Subscribe to GET /lf/status/stream for load progress UI

Scrape /metrics for per-model latency and load history

🧠

Phoenix

Ready when you are

Hey! I'm Phoenix — I know Titas's work, projects, and experience. Ask me anything — from distributed systems to production RAG, or what it's like building at Tesco and VMware.