Ollama Qwen: Which Qwen Model to Run for Your Memory

I ran three ollama qwen tags on my 8GB MacBook Air M1, and the 9B I used to recommend for 8GB did not work there. qwen3.5:9b took 47.93 to 53.36 seconds to load, ran 58% on the CPU, pushed swap past 5.8 GiB, and generated 0.079 to 0.086 tokens per second. qwen3.5:4b loaded in about 5 seconds, ran 82% on the GPU, and generated 8.96 to 18.55 tokens per second across two sessions.

So my picks are qwen3.5:4b on an 8GB Mac or a small card, qwen3.5:9b from 16GB, qwen3.8:27b on 24GB, and qwen3.6:35b-a3b on 32GB or more. All of them are current Qwen releases in the Ollama library, each with tools, vision, and thinking and a 256K context window. The newer Qwen generations also cache far less memory per token than qwen3:8b, so where both fit, the newer model leaves more room for context.

The tag size is only part of the fit. The KV cache for your context window sits on top of the weights, and the Ollama context length docs say Ollama picks a default context from your VRAM: 4k below 24 GiB, 32k from 24 to 48 GiB, and 256k at 48 GiB or more. The tables below map each Qwen family to a memory tier, with the math. Checked with Ollama v0.35.0 (published September 28, 2026) and the library as of September 30, 2026; the test runs below used the newer 0.35.1.

Every Qwen model on Ollama

Ollama's library lists 18 Qwen entries, from the original qwen (Qwen 1.5) through qwen3.8, plus coder and vision variants. The table keeps the ones that matter. Tags, sizes, and context values come from each model's tags page on ollama.com, and the minimum Ollama version from each model's registry config. qwen3.8 is the newest family.

The qwen3.8, qwen3.6, and qwen3.5 downloads each include an Apache 2.0 license layer. None of their tags pages lists a cloud tag, so every row above runs on your own hardware. The one Qwen cloud option Ollama documents is qwen3-coder:480b-cloud, offered because the local 480B build needs at least 250GB of memory; cloud models run on Ollama's servers and need ollama signin.

If you searched for Qwen 4, the closest thing on Ollama is qwen3.8-flash-next, which its model page calls an experimental preview of the architecture that will underpin Qwen4: 125B total parameters, 6B active per token, and a smallest tag of 105GB.

Which Qwen to run for your memory

Pick the row that matches your GPU VRAM, or the memory left over on an Apple Silicon Mac after macOS and your apps. On my 8GB M1, Ollama reported 5.3 GiB of GPU memory, so an 8GB Mac behaves like a 5GB card. The first two rows are what I measured there with ollama ps; the rest add the tag size to the f16 KV cache at the suggested context, using the per-token figures in the next section.

I would leave 1 to 2GB above the last column for Ollama's compute buffers and the display. That headroom is why qwen3.6:35b-a3b sits in the 32GB row even though its 23GB file looks close to a 24GB card: it should still run at 24GB, but expect part of it on the CPU.

The 48GB row is also where Ollama's own default reaches 256K, so on that hardware qwen3.8:27b gets its full window with no setting. To run the same math for another model or quant, use the LLM VRAM calculator.

A 12GB card should hold qwen3.5:9b at 16K by the same math (about 7.1GB), but I have not run that, and after what the 9B did on 8GB I would not recommend it below 16GB without seeing your own ollama ps output first.

What happened on an 8GB MacBook Air

I ran the 9B, 4B, and 2B tags with Ollama 0.35.1 through /api/generate, with thinking off, temperature 0, and a fixed 35-token prompt. Load is the cold load_duration. Generation is the median of three warm runs of 128 tokens, except the 9B, which got two single runs of 10 tokens because that was all it could manage.

The 9B is why the 8GB row changed. Ollama put only 12 of its 34 layers on the GPU, and the Mac swapped hard. My first 9B request, with a 200-token limit, had not finished after 17 minutes 29 seconds; swap peaked at 10.3 GiB and free disk fell to 4.9 GiB as the swap files grew, so I cancelled it. The KV cache was not the problem: at 4K it was 128 MiB. The 6.2 GB the loaded model needed is more than the 5.3 GiB of GPU memory Ollama saw, so most layers went to the CPU and the rest of the system went to swap.

The 4B put 33 of 34 layers on the GPU and stayed usable. The 2B was the only tag that ran 100% on the GPU, and swap barely moved: 2.85 GiB before the runs and 2.91 GiB at most during them. I would start with the 4B because it is the larger model at a workable speed, and drop to the 2B when other apps already have the Mac swapping.

Session 1 (October 3) was about 2 to 4 times slower than session 2 (October 4) for the same commands, and the 2B swung the most, from 6.67 to 27.27 tokens per second. Ollama logged similar free memory at load in both sessions and I have not found the cause, so read these speeds as a range for a busy 8GB Mac, not a fixed number. The CPU/GPU splits and context sizes were identical in both sessions.

Why Qwen3.5 and newer need less KV cache

Qwen3.5, Qwen3.6, and Qwen3.8 use a hybrid attention design, visible in the Qwen3.8-27B config: three of every four layers are Gated DeltaNet linear attention, and only every fourth layer is full attention. The linear layers keep a fixed-size state that does not grow with the conversation, so only the full-attention layers add to the KV cache as context grows.

The per-token cost is 2 (keys and values) x full-attention layers x KV heads x head dimension x 2 bytes at f16. For the 27B models, the config lists 64 layers with a full-attention interval of 4, so 16 layers cache, each with 4 KV heads of 256 dimensions: 2 x 16 x 4 x 256 x 2 = 65,536 bytes, or 64 KiB per token. At 32,768 tokens that is about 2.1GB.

The server log on my M1 agreed with this math. Ollama allocated 128 MiB of f16 cache for 4096 tokens on both qwen3.5:9b and qwen3.5:4b, spread over 8 layers, which is 32 KiB per token, and 1,024 MiB when I set the 4B to 32768 tokens. qwen3.5:2b used 6 cache layers and 12 KiB per token.

The older rows come from the Qwen3-8B, Qwen3-VL-8B, and Qwen2.5-7B configs, and the 35B row from the Qwen3.6-35B-A3B config. So qwen3.5:9b holds 32K of context in 1.1GB while qwen3:8b needs 4.8GB, which makes the newer model the better pick once you have room for its larger file. The KV cache size guide covers the formula for other architectures.

How to pull a specific Qwen quant tag in Ollama

Default Qwen tags are 4-bit Q4_K_M builds: the registry configs for qwen3.8:27b, qwen3.6:35b-a3b, and qwen3.5:9b all report file_type Q4_K_M. For a different quant, name the full tag from the tags page:

ollama pull qwen3.5:9b-q8_0      # 11GB, 8-bit
ollama pull qwen3.8:27b-q8_0     # 30GB, 8-bit
ollama pull qwen3.8:27b-bf16     # 56GB, unquantized
ollama pull qwen3.8:27b-mlx      # 18GB, Apple Silicon MLX build

The tags page labels MLX builds, including the nvfp4 and mxfp8 variants; those target Apple Silicon. Tags with mtp in the name, such as qwen3.8:27b-mtp-q4_K_M, add multi-token prediction for speculative decoding, which has had its own regression reports.

The two small qwen3.5 sizes are the exception to the 4-bit default: qwen3.5:2b and qwen3.5:0.8b point at their q8_0 builds, which is why 2b is 2.7GB while 2b-q4_K_M is 1.9GB. Whether the step from 4-bit to 5-bit or 8-bit is worth the memory is covered in the Q4_K_M vs Q5_K_M comparison.

qwen3.8 needs Ollama 0.32.12 or newer and qwen3.6 needs 0.30.0, so update to v0.35.0 or newer before pulling either. After that, ollama pull downloads a tag and ollama run starts a chat with it.

How to set the context length for Qwen in Ollama

Qwen models advertise 256K, but Ollama loads them with far less unless you raise it. The documented default is 4k tokens below 24 GiB of VRAM, and Ollama recommends at least 64000 tokens for agents, web search, and coding tools. The Ollama FAQ still gives a flat 4096-token default without the VRAM tiers, so treat the context-length page as current. On my 8GB M1, the server log reported 5.3 GiB of GPU memory and picked 4096 tokens, and ollama ps showed 4096 in the CONTEXT column for every load where I did not set it.

You can set it in three places. On the server, which applies to every client:

OLLAMA_CONTEXT_LENGTH=32768 ollama serve

Inside an ollama run qwen3.5:4b session, type /set parameter num_ctx 32768. Through the API, pass "options": {"num_ctx": 32768} in the request body.

Then confirm it took. ollama ps shows a CONTEXT column and a PROCESSOR column; you want your number in the first and 100% GPU in the second. When I raised qwen3.5:4b to 32768 on the M1, ollama ps showed 32768 and a 38%/62% CPU/GPU split: the 1,024 MiB cache pushed Ollama from 33 to 22 of 34 layers on the GPU, and warm speed fell from 8.96 to 3.98 tokens per second in session 1 and from 18.55 to 14.37 in session 2. If the split shows CPU, lower the context or quantize the cache: the FAQ says OLLAMA_KV_CACHE_TYPE=q8_0 uses about half the memory of the default f16 and only works when Flash Attention is active. Parallel requests multiply the cache too, because RAM scales with OLLAMA_NUM_PARALLEL times the context length. The full list of environment variables is in the Ollama commands cheat sheet.

How to turn Qwen thinking on and off in Ollama

Thinking-capable Qwen models return their reasoning in a separate thinking field. Per the Ollama thinking docs, the think request field controls it: true asks for thinking, false asks for none, null uses the model default, and a string picks a named level. Ollama's v0.35.0 renderer setup starts qwen3.5 (which qwen3.6 also uses) and qwen3.8 with thinking on. That matched my runs: with think omitted, qwen3.5:4b returned a thinking field on both /api/chat and /api/generate.

Pick the method that matches how you run the model:

• Interactive chat: type /set nothink to turn thinking off and /set think to turn it back on, both listed in the session's help text in cmd/interactive.go. In my session, the answer after /set nothink printed with no Thinking block, and /set think brought it back.
• One-shot CLI: ollama run qwen3.5:4b --think=false "summarize this" skips it, and --hidethinking keeps the reasoning but prints only the answer, per Ollama's thinking announcement.
• API: send "think": false in the /api/chat or /api/generate body.

The difference is large on a small model. I asked qwen3.5:4b "What is 17 * 23? Reply with just the number." With thinking on, it generated 375 tokens and 1,022 characters of reasoning before answering 391. With "think": false it answered 391 in 4 tokens. From the CLI on the M1, the default run took 80 seconds, --think=false took 1 second, and --hidethinking still took 63 seconds, because it hides the reasoning rather than skipping it. Use --think=false when you want speed, and --hidethinking only when you want the model to reason but do not want to read it.

qwen3.8 differs from the older Qwen models, which take only true or false. In the v0.35.0 source, qwen3.8 accepts false, "low", "medium", and "xhigh", and uses "xhigh" when think is not set (qwen35.go at v0.35.0). Run curl http://localhost:11434/api/show -d '{"model":"qwen3.8"}' to see the values your install supports.

Stick to those values: an open issue reports that "high" and "max" were silently accepted on 0.34.x and ran the default, then medium (ollama#18632). On qwen3.8 I would still try "low" before switching thinking off, though I have not run qwen3.8 myself.

qwen3.5 has no such levels. When I sent "think": "low" to qwen3.5:4b, Ollama returned HTTP 200 and the same 375 tokens of thinking as true, and --think=low from the CLI printed the full reasoning and took 124 seconds. On qwen3.5, thinking is on or off.

Qwen coder tags on Ollama

Qwen also ships coding-tuned tags; the best Ollama model for coding guide ranks them against Gemma, gpt-oss, and Devstral by VRAM and agent tool. The library listed these on September 30, 2026:

• Current Qwen coding tags: qwen3.6:27b-coding (18GB) and qwen3.6:35b-a3b-coding (23GB), both 256K with tools, vision, and thinking.
• Dedicated coder models: qwen3-coder:30b (19GB, 256K, tools) and qwen3-coder-next (52GB, 256K, tools).
• Autocomplete-sized: qwen2.5-coder:1.5b (986MB) and qwen2.5-coder:7b (4.7GB), both limited to 32K.

Known issues when running Qwen on Ollama

Most Qwen problems on Ollama trace to three things: tool-call parsing, long agent loops, and memory on Apple Silicon. These were open on GitHub on September 30, 2026:

• Long tool loops on qwen3.8: ollama#17778 reports a 500 error, "no user query found in messages", after many tool calls at around 205K context. It has 47 comments.
• Tool calls on Qwen3 via /api/chat: ollama#14601 reports malformed tool definitions when tools are passed through the tools parameter.
• Long file writes from coder models: ollama#18563 reports the qwen3coder parser rejecting long file-write tool calls.
• Memory growth on MLX: ollama#18620 reports qwen3.6:27b-mlx holding about 0.43 GiB more after each request that ends in a tool call, on a 32GB Mac.

When an agent fails on Qwen, I would check ollama ps first for a context above the 4k default, then retry the GGUF tag instead of the -mlx one.

How I tested this

I ran everything on a MacBook Air with an M1 and 8 GB of unified memory (macOS 26.3), using Ollama 0.35.1 from the official GitHub release, on October 3, 2026 (session 1) and October 4, 2026 (session 2, a repeat of the 4B and 2B speed runs). Normal desktop apps were open, and swap already held 2.82 GiB before the first 4B load.

• Models: qwen3.5:9b (6.6 GB, Q4_K_M), qwen3.5:4b (3.4 GB, Q4_K_M), and qwen3.5:2b (2.7 GB, Q8_0), pulled from the Ollama library.
• Speed runs: a small Python script called /api/generate with a fixed prompt, temperature 0, seed 42, thinking off, and 128 output tokens (10 for the 9B), at the default context unless noted. Each tag got one cold request after unloading all models, then three warm requests, then ollama ps. Cold load times may still benefit from the file sitting in the page cache.
• Thinking runs: one arithmetic question on qwen3.5:4b through /api/chat and /api/generate with think omitted, false, true, and "low"; the CLI with no flag, --think=false, --hidethinking, and --think=low; and an interactive session with /set nothink and /set think.
• Not tested: any machine with more than 8 GB, any discrete GPU, qwen3.8, qwen3.6, the 27B and larger tags, the -mlx and q8_0 tags, tool calling, contexts above 32K, and answer quality. The rows above 8GB in the memory table come from config math, not runs. I also tried llama.cpp's llama-bench (build b11351) on the same model files and it refused to load them, so there is no llama.cpp comparison.

Every measured figure on this page is in the raw results file. Timings are specific to this machine and its memory pressure; the default context, thinking behavior, and KV cache sizes should reproduce on any 8GB Apple Silicon Mac.

FAQ

Is Ollama Qwen 3.5 free?

Yes. Ollama is free to run locally, and the qwen3.5 weights ship with an Apache 2.0 license, which allows commercial use. Running it costs only your own hardware and power, with no account or usage fee.

Is Ollama like ChatGPT?

Partly. Ollama gives you a chat window and an API like ChatGPT, but the model runs on your own computer instead of a cloud service. Answers stay local and work offline once the model is downloaded. The trade-off is that speed and model size depend entirely on your GPU or unified memory.

Related coverage

• Best Ollama Model for Coding: Picks by VRAM and Agent Tool
• KV Cache Size: How Much VRAM It Uses and How to Cut It
• LLM VRAM calculator
• Ollama commands cheat sheet

References

• Hugging Face Qwen configs - https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/config.json
• Ollama context docs - https://docs.ollama.com/context-length
• Ollama FAQ - https://docs.ollama.com/faq
• Ollama library - https://ollama.com/search?q=qwen
• Ollama thinking docs - https://docs.ollama.com/capabilities/thinking
• Ollama v0.35.0 release - https://github.com/ollama/ollama/releases/tag/v0.35.0