On my Apple M1, Ollama and llama.cpp ran the same model at practically the same speed. I gave both runners the same 940.37 MB Qwen2.5 Q4_K_M GGUF file, chat prompt, seed, context, and output cap. Ollama's median output rate was 48.078 tokens per second and llama.cpp's was 47.17, with median wall times of 1,268.53 ms and 1,268.76 ms. llama.cpp used slightly less memory: 1,110.23 MB resident against Ollama's 1,164.42 MB.

So Ollama vs llama.cpp is mostly a choice about how you want to run local models, not a speed contest. Ollama gives you one-command model management and a simple local API; llama.cpp gives you direct control over the GGUF file and the server's runtime flags. The model file, quantization, chat template, hardware backend, prompt, and cache state can move a benchmark more than the runner's name, and this is one reproducible M1 comparison, not a universal ranking. It also says nothing about concurrent serving, which needs its own load test.

What Ollama and llama.cpp actually are

llama.cpp is an open-source C and C++ inference project that loads GGUF model files and runs them on hardware backends including Metal, CUDA, Vulkan, and CPU instruction sets. Its direct server gives you control over context size, GPU layers, batching, cache behavior, and the HTTP surface. GGUF stores model weights and metadata together, so the exact file can be identified and reused across compatible runners.

Ollama packages a local model service behind commands such as ollama run, ollama list, and ollama ps. Its API exposes generation and chat endpoints, and its model library handles names, manifests, templates, and storage. The Ollama model details endpoint reports the architecture, parameter count, context length, and quantization level, which lets you record the actual input to a comparison instead of comparing vague model names.

Whether Ollama uses llama.cpp under the hood depends on the platform and the Ollama version. The Ollama repository documents llama.cpp in parts of its backend history, and Ollama has also used Apple's MLX path on Apple Silicon. Check the current runtime and model details rather than assuming a new release uses the same backend as an older one.

Because both runners load the same GGUF files, the quantization choice carries over whichever you pick; the Q4_K_M vs Q5_K_M comparison covers which quant level to download.

The same-model benchmark

Both runners got the same chat message, seed 42, temperature 0, a 4,096-token context, and a 128-token output cap. Ollama received the request through /api/chat; llama.cpp received it through its OpenAI-compatible /v1/chat/completions endpoint. Using chat endpoints on both sides means each one applies the model's chat template, so I was not comparing two different raw prompt formats.

The servers ran one after the other, never at the same time, so one model process could not hold memory while the other was measured. Each runner served the request three times, and the table shows the medians.

RunnerMedian wall timeOutput speedRSSValid outputs
Ollama 0.32.131,268.53 ms48.078 tok/s1,164.42 MB3/3
llama.cpp build 11,268.76 ms47.170 tok/s1,110.23 MB3/3

Both runners returned 54 prompt tokens and 53 output tokens on every repetition, and the output hash was identical across both runners, so they were doing the same work.

What these numbers support is narrower than "llama.cpp is faster." The warm wall times are indistinguishable at the resolution of a small local benchmark, and Ollama's output rate was slightly higher. Once the model is loaded, the wrapper does not necessarily add a large per-token cost. A different model family, prompt length, context window, Metal build, or cache policy could produce a different result.

If speed is what decides it for you, run both on your own hardware with the same model file, quantization, prompt, context, thread count, and warmup, and record first-token latency, generation rate, peak resident memory, and failure behavior.

Setup friction and the first request

Setup has two parts: starting a service and loading a model. Ollama's server can be up before any model is loaded, and it loads the named model into its runner on the first request. A direct llama.cpp server takes a file path at process start and loads the model before it serves anything. Timing only the HTTP request hides that difference.

In my run, a fresh Ollama process answered /api/tags after 155.2 ms. Its first chat request then took 2,720.94 ms of wall time, of which Ollama reported 1,463.05 ms as model load. That cold request was the first of Ollama's three repetitions; the median in the table comes from a warm run. A fresh llama.cpp server answered /health after 768.05 ms, and that figure includes the model load, because the endpoint was not ready until loading finished. These are observations from one machine, not install-time promises for every operating system.

Ollama's advantage is the path from an installed application to a named model: ollama run qwen2.5:1.5b hides the model-file path, manifest details, and server launch flags. llama.cpp's advantage is the reverse. As Bswen's llama.cpp vs Ollama comparison for local coding points out, the file path and runtime flags are explicit, which is what you want when you need a specific community GGUF, a custom context size, a chosen GPU-layer count, or a server option the packaged runner does not expose.

Memory, cache, and failures

Memory use is not the same as model file size. The process also holds runtime code, GPU buffers, the context cache, and HTTP server state; the KV cache size guide works out how much the context cache adds for a given model and context length. A fair comparison reports the observed resident memory and context settings next to the quantization. ollama ps shows what Ollama keeps loaded, and a llama.cpp process can be inspected with the operating system's process tools and its own slot and metrics endpoints.

Median RSS in my run was 1,164.42 MB for Ollama and 1,110.23 MB for llama.cpp. All 6 requests succeeded with valid output, and none failed. That RSS gap describes this process configuration; it does not promise the same gap for other models or backends.

Failure behavior belongs next to speed. A runner can post a good tokens-per-second number while failing to load a model, applying a different template, or exhausting the context window. My test used a short prompt and a 4,096-token context, so it does not cover long-context pressure, concurrent queues, tool calls, or streaming backpressure.

Which runner should you pick?

Pick Ollama if you want a local model running quickly, a named model library, and a familiar REST API. It is the better default for a laptop prototype, a local coding assistant, or a one-user internal tool where model management matters more than exposing every inference knob. Before downloading a larger model, check what an 8GB M1 can run locally. If a graphical model browser and MLX speed on Apple Silicon matter more than CLI habits, LM Studio vs Ollama compares the two from that angle. If Ollama will back a coding agent, the next decision is the model, and the best Ollama model for coding guide lists which model fits each GPU and Mac memory tier.

Pick llama.cpp if the exact GGUF file is part of the requirement, if you need direct control of GPU offload or context settings, or if the deployment should have fewer packaged layers. The llama-server guide covers its flags, parallel slots, and API key.

Whichever you choose, check the model fit first with the LLM VRAM calculator; changing runners cannot fix a machine that lacks enough memory. For a multi-user service, compare both against a serving system designed for concurrency rather than extrapolating from this single-request test. vLLM vs Ollama covers that decision.

How I tested this

  • Machine: Apple M1 (arm64, Darwin 26.3), with the harness driven by Python 3.13.5.
  • Versions: Ollama 0.32.13 and llama.cpp llama-server build 1 (0b1bad14f).
  • Model: Ollama's installed qwen2.5:1.5b, loaded by both runners from the same 940.37 MB Q4_K_M GGUF blob.
  • Date: August 24, 2026.
  • Workload: one short chat prompt, seed 42, temperature 0, 4,096-token context, 128-token output cap, three requests per runner, with each server started fresh and run on its own.
  • Not tested: other models or quantizations, other hardware or GPU backends, long contexts, concurrent requests, tool calls, and streaming.
  • Raw record: every per-run metric, the model file hash, and the output hashes are in the public results file.

Related coverage

References