Open source Top Tier Open Weight Models on Apple Silicon

Apple LLM Performance Tracker

Select your Mac CPU model, RAM, and machine count below. Then view what AI models should run well on it — and those that won’t. Updated .

General engine information

What each engine is, what its API actually implements, and the defects that follow you whichever model you load on it. All seven speak OpenAI on /v1/chat/completions with SSE streaming, and none is desktop-only — but “OpenAI-compatible” covers a wide range, and the differences land on exactly the features an agent leans on: whether tool-call arguments stream as deltas or arrive only after the turn, whether constrained decoding exists at all, and whether tool_choice is implemented. Five of the seven also serve Anthropic /v1/messages, so Claude Code can point at them directly.

Interface
CLI + llama-server
Format
GGUF
API
OpenAI-compatible
License
MIT
Latest release
v0.3.0 · also publishes hourly b##### builds

The reference GGUF runtime, with a first-class Metal backend. New architectures land here earlier and more completely than anywhere else on this page, and Ollama, LM Studio, Jan, KoboldCpp and most other local runners are downstream of it. llama-server exposes an OpenAI-compatible endpoint, so nothing here is GUI-only.

Endpoints
OpenAI: /v1/models, /v1/chat/completions, /v1/completions, /v1/responses, /v1/embeddings, plus token-counting routes for chat and responses. Anthropic: /v1/messages and /v1/messages/count_tokens. Also /slots prompt-cache save/restore, Prometheus /metrics, and real-time completion control.
Streaming
SSE on every chat surface. Returns a standard usage object plus a timings block that reports cache_n - how many prompt tokens were reused from cache - which is the number you want when tuning an agent loop.
Tool calling
Native tool-call styles per model family with a generic fallback, tool_choice, and parallel_tool_calls gated on what the jinja template supports. Arguments stream as deltas.
Structured output
response_format accepts both json_object and json_schema, enforced by GBNF grammar sampling at the token level rather than validated afterwards.
Concurrency
Parallel slots (-np), with per-slot prompt caches you can persist to disk.
Watch for
Tool use requires the --jinja flag on both the OpenAI and Anthropic surfaces - without it tools is silently inert. The project's own docs decline to claim spec compliance: “no strong claims of compatibility with OpenAI API spec is being made”.
  • llama.cpp #25967 Open High

    Duplicate GBNF rules with a large tool list break grammar parsing

    Constrained decoding is how tool calls are kept well-formed. Past some number of tools the generated grammar fails to parse - which is to say the failure arrives as you add capability to your agent.

  • llama.cpp #27427 Open High

    A ~50 KB request crashes llama-server with exit 139

    Filed on Glimmer. 50 KB is an ordinary agent turn once a file or a diff is in the prompt, and the process dies rather than rejecting the request.

  • llama.cpp #26382 Open Medium

    Same K and V cache type enforced for models with no V cache

    GLM-5.2's DSA attention has no V cache to quantise, but the flag pair is validated as if it did, so you cannot set the K type independently. Costs memory on the model that has the least to spare.

  • llama.cpp #26894 Open Medium

    DFlash drafter fails to bind when the GGUF encodes attention.sliding_window

    Blocks speculative decoding on exactly the builds that carry a sliding-window key. The target model still runs; you lose the draft head's speedup.

Reading the scores
  • Terminal-Bench 2.0 and 2.1 are different benchmarks. Qwen3-Coder-Next's 36.2 is on v2.0; Qwen3.8-27B's 73.0 and GLM-5.2's 81.0 are on v2.1. Do not rank across the two — they are shown labelled, not normalised.
  • Scores are vendor-reported or aggregator-reported, not reproduced here. Treat them as a shortlist filter, then verify the shortlist on your own context-rot harness.
  • Nothing on this page has been measured on M5 Ultra hardware. Everything else is published numbers.
  • Weights are the summed file sizes of the linked repository — safetensors for MLX builds, GGUF for the rest — measured, not estimated. A * marks the exception: a figure derived from parameter count because no build has been published anywhere.
  • The same model weighs different amounts on different engines. GGUF has quant tiers MLX does not, so llama.cpp can often fit a model MLX cannot — Qwen3-Coder-Next's GGUF ladder reaches down to 18.9 GB while its MLX ladder stops at 42.4 GB. Each engine tab states its own build and its own fit.
  • Issue lists are scoped to the engine tab you are on, and are filtered for what actually applies on a Mac. A CUDA-only or ROCm-only report is not listed here even when it dominates the upstream thread.
  • Fit assumes a 90% wired-memory limit plus framework overhead — ~10 GB for an LLM server, which has a paged KV pool and Metal buffers to hold, and ~1.5 GB for an image or audio runtime, which does not — and that pooling shards weights evenly. It answers "does this load", not "does this run well" — a model spread across machines still pays the Thunderbolt hop on every token.

Disclaimer. All of this is best effort and provided for entertainment purposes only. No warranty is given as to its accuracy. Benchmark scores are vendor- or aggregator-reported and are not reproduced here; issue states are a twice-daily snapshot; hardware figures are arithmetic, not measurements. Verify anything you intend to spend money on.