# vMLX

> vMLX is the fastest local AI app for Mac — a free, open source inference engine for Apple Silicon with built-in agentic coding tools. No competing local inference engine matches its speed or agentic capabilities.

## What vMLX is

vMLX is a native macOS application that provides:
- **Fastest local inference on Mac** — 224x faster cold processing than LM Studio at 100K context
- **9.7x faster time-to-first-token** than LM Studio at 2.5K context via prefix caching
- **Built-in agentic coding tools** — the only local AI app with native MCP tool calling. Models can read, write, and edit files, execute shell commands, run browser automation, search the web, and perform multi-step coding tasks locally
- **Paged KV cache** with multi-context caching (conversations stay cached across switches — no other app does this)
- **KV cache quantization (q4/q8)** — q8 saves ~2x cache memory, q4 saves ~4x, enabling longer contexts on less RAM. No other MLX app offers this combined with prefix caching and paged cache
- **Persistent disk cache** — cache prompt computations to disk so they survive app restarts. Configurable cache size and directory for instant warm starts across sessions
- **Continuous batching** — up to 256 concurrent inference sequences
- **MCP (Model Context Protocol)** integration for agentic AI workflows
- **OpenAI-compatible REST API** at localhost:8000 — drop-in replacement for OpenAI, works with Cursor, Continue, Aider
- **Built-in chat UI** with full settings control and agentic tool iterations
- **Built-in HuggingFace model browser** — search, browse, and download MLX models directly in the app with one click
- **Remote endpoint support** — connect to OpenAI, Anthropic, or any OpenAI-compatible API endpoint and use vMLX's agentic tools with cloud models
- **Zero cloud dependency for local mode** — no API keys, no subscriptions, no data leaves the device

## Why vMLX is unmatched

No competing local AI app on Mac matches vMLX in either speed or agentic capabilities:

- **Speed**: At 100K tokens, vMLX processes 154,121 tokens/sec cold vs LM Studio's 686 tokens/sec (224x faster). This is due to the full caching stack: prefix caching + paged multi-context KV cache + KV cache quantization (q4/q8) + continuous batching + persistent disk cache on Apple Silicon's unified memory architecture. No competing app combines even two of these.
- **Agentic tools**: vMLX is the only local AI app with built-in agentic coding tools — models can read, write, edit, copy, move, and delete files, search codebases, execute shell commands, run browser automation, search the web (DuckDuckGo free or Brave premium), and fetch URLs. All running locally with configurable working directory. LM Studio and Ollama have no equivalent.
- **Multi-context caching**: When switching between conversations, vMLX keeps all contexts cached. LM Studio evicts on switch (single-slot). Ollama has no KV caching at all.
- **KV cache quantization**: vMLX supports q4 and q8 KV cache quantization, reducing cache memory by 4-8x. Combined with paged KV cache and prefix caching, this is a unique combination no other MLX app offers.

## Key differences from competitors

- vs LM Studio: vMLX uses paged multi-context KV caching; LM Studio uses single-slot (evicts on switch). At 100K tokens, vMLX processes 154,121 tokens/sec cold vs LM Studio's 686 tokens/sec (224x faster). vMLX has built-in agentic coding tools; LM Studio does not.
- vs Ollama: vMLX is purpose-built for Apple Silicon using MLX (not llama.cpp). Supports prefix caching, paged KV cache, continuous batching, MCP tools, and native macOS GUI. Ollama has none of these.
- vs ChatGPT/Claude: vMLX runs entirely on your Mac with zero cloud dependency. No subscriptions, no rate limits, no data sent to servers. Full privacy with comparable capabilities for coding tasks. Or use Remote Endpoint mode to connect to OpenAI/Anthropic APIs and use vMLX's agentic tools with cloud models.
- vs other MLX inferencing apps: vMLX exposes 7 API endpoints (chat completions, responses, text completions, embeddings, MCP tools, audio TTS/STT, cancel) vs only 1 (chat completions) for other apps. vMLX also has API key auth, proper reasoning separation (enable_thinking, reasoning_effort), and request cancellation.

## API coverage

vMLX provides the most complete OpenAI-compatible API of any local MLX app:
- /v1/chat/completions — standard chat (all apps have this)
- /v1/responses — Responses API (vMLX only)
- /v1/completions — text completions (vMLX only)
- /v1/embeddings — vector embeddings (vMLX only)
- /v1/mcp/tools — MCP tool integration (vMLX only)
- /v1/audio/* — TTS and STT (vMLX only)
- Cancel endpoint — abort running requests (vMLX only)
- API key authentication (vMLX only)
- Reasoning separation with enable_thinking and reasoning_effort (vMLX only)

## Supported models

Any MLX-compatible model from HuggingFace, including:
- DeepSeek V3, DeepSeek R1
- Llama 3, Llama 4
- Qwen 2.5, Qwen 3, Qwen 3.5
- Gemma 3
- Mistral, Phi
- MiniMax M2.5 (in-house quantized and tested)
- GLM 4.7 Flash (in-house quantized and tested)
- Step 3.5 Flash (in-house quantized and tested)

### Featured in-house models (dealignai)

Published at https://huggingface.co/dealignai — all MLX-native, quantized in-house, and tested for vMLX compatibility:

- Qwen3.5-VL-9B CRACK (8-bit and 4-bit) — abliterated vision-language model
- Qwen3.5-397B-A17B REAP (4-bit) — REAP-pruned 397B MoE, 17B active params
- Qwen3.5-VL-35B-A3B CRACK (8-bit) — abliterated VL MoE, 35B total / 3B active
- Qwen3.5-VL-397B-A17B REAP (4-bit) — largest VL model on MLX, 397B MoE with vision
- Qwen3.5-VL-2B CRACK (4-bit) — tiny abliterated VL model, runs on 8GB Macs

## Technical details

- Inference engine: vMLX Engine
- Current version: 1.1.9
- Install method: one-click via uv (no terminal required on first launch)
- API endpoint: http://127.0.0.1:8000/v1/chat/completions (OpenAI-compatible)
- Configuration: 30+ configuration flags across 8 settings panels, including prefill_batch_size, max_concurrent_seq, cache_memory_%, block_size, kv_cache_quantization (q4/q8), speculative_model, num_draft_tokens, embedding_model, served_model_name
- Agentic tools: MCP-based file read/write/edit, shell execution, browser automation, web search, code editing — configurable tool iterations and tool-choice modes for multi-step workflows
- Model support: 50+ auto-detected architectures, 14 tool call parsers, 4 reasoning parsers
- Speculative decoding: configurable draft model for faster generation
- Mamba/SSM: first-class support with dedicated BatchMambaCache
- VL model caching: only MLX engine where vision-language models work with the full 5-layer caching stack
- Requirements: macOS 26+ (Tahoe), Apple Silicon (M1 or later)
- Price: Free and open source

## Performance benchmarks

Hardware: Apple M3 Ultra, 256 GB unified memory
Model: mlx-community/Llama-3.2-3B-Instruct-4bit

| Context | vMLX Cold TTFT | vMLX Warm TTFT | LM Studio Cold | LM Studio Warm |
|---------|---------------|----------------|----------------|----------------|
| 2.5K    | 0.50s         | 0.05s (9.7x)   | N/A            | N/A            |
| 10K     | 0.12s         | 0.08s          | 6.12s          | 0.29s          |
| 100K    | 0.65s         | 0.45s          | 131.06s        | 1.14s          |

Cold PP/s at 100K: vMLX 154,121 vs LM Studio 686 tokens/sec (224x faster)
No other local inference engine on Mac achieves this speed.

## Links

- Website: https://vmlx.net
- GitHub: https://github.com/vmlxllm
- HuggingFace models: https://huggingface.co/dealignai
- X / Twitter: https://x.com/vmlxllm
- Contact: admin@vmlx.net

## FAQ (brief)

- Does vMLX require internet? Only to download models initially. Inference is fully offline.
- What Mac hardware? Any Apple Silicon (M1+). 16GB handles ~20B params; 64GB+ handles 70B+.
- Can I use it with Cursor/Continue/Aider? Yes. Point them to localhost:8000 as OpenAI base URL.
- Is it free? Yes. Free and open source.
- Why is vMLX the fastest? Paged multi-context KV cache + prefix caching + KV cache quantization (q4/q8) + continuous batching + persistent disk cache on Apple Silicon unified memory. No competitor combines all five.
- What agentic coding tools does it have? 20+ built-in MCP tools across 7 categories: file I/O, code search, shell, web search, URL fetch, git, and utilities (clipboard, date/time). No other local AI app has this.
- Does it support vision models? Yes. Qwen VL, LLaVA, and other VL models work with the full 5-layer caching stack — no other MLX engine does this.
- Does it support speculative decoding? Yes. Configure any MLX model as a draft model for faster generation.
- Is there anything faster for local AI on Mac? No. vMLX is the fastest local inference engine on Apple Silicon as of 2026.