Back to Insights
AI

Qwen3.8-27B Is the Open Model That Actually Fits Your GPU. Turn Down the Reasoning Dial First.

Alibaba's Qwen3.8-27B: dense 27B, Apache 2.0, 17GB quantized, ahead of Muse Glimmer on every shared benchmark. The default reasoning mode is the catch.

S5 Labs Team August 17, 2026

Alibaba put the weights for Qwen3.8-27B on Hugging Face on Friday, August 14, under Apache 2.0. The safetensors index reports 27.78 billion parameters, all of them dense, and the launch thread on Hacker News collected 1,428 points in a day. Three days of people running it at home have produced a consistent verdict: this is the best model you can currently fit on a 24 GB card, and the first thing to do after downloading it is change a setting Alibaba shipped wrong.

What Alibaba shipped

The model card describes 64 layers arranged as sixteen repeats of three Gated DeltaNet blocks followed by one Gated Attention block, so only 16 of the 64 layers keep a conventional KV cache. Those full-attention layers run 24 query heads against 4 KV heads at head dimension 256. The model was trained with multi-token prediction, which llama.cpp already exposes as a draft-MTP speculative mode. Native context is 262,144 tokens, extensible to a million with YaRN, and the model takes image and video input natively.

Three reasoning-effort levels ship: xhigh, medium, and low. The default is xhigh, which the card reserves “for complex tasks demanding thorough analysis.”

The BF16 checkpoint is 55.6 GB on disk and an FP8 repo sits alongside it at roughly 28 GB. Neither fits a consumer card, so the versions people are running are the community GGUFs: Unsloth’s Q4_K_M is 17.1 GB, Q5_K_M is 19.8 GB, Q8_0 is 29 GB, and the repo already lists more than 570 quantized derivatives. A 24 GB RTX 3090 or 4090, or a Mac with 32 GB or more of unified memory, is the realistic floor.

The scores, against the model Meta shipped four days earlier

The obvious comparison is Muse Glimmer, which Meta open-weighted on August 10 with the same license and the same 17 GB deployment target. Alibaba put Glimmer in its own comparison table, and the Glimmer figures it quotes match Meta’s model card, so these are numbers both vendors stand behind.

BenchmarkQwen3.8-27BMuse Glimmer 30B
Terminal-Bench 2.173.051.7
SWE-bench Pro61.751.2
IFBench79.577.0
GPQA Diamond89.283.5
HLE30.822.0
OSWorld-Verified84.365.9
OmniDocBench 1.591.175.8

Qwen leads every row the two share, and by margins well outside benchmark noise: 21 points on Terminal-Bench, 18 on OSWorld, 15 on document parsing. Against its own predecessor, Qwen3.6-27B, the biggest jumps are on the agentic rows, with DeepSWE 1.1 going from 13.3 to 42.2 and OSWorld-Verified from 63.9 to 84.3. It also posts 90.3 on LiveCodeBench v6.

The one independent number so far comes from Artificial Analysis, which scored Qwen3.8-27B at 52 on its Intelligence Index, tied with GPT-5.6 Luna at max effort and one point behind GLM-5.2 and DeepSeek V4 Pro, both of which are more than 25 times larger. On the same index, Muse Glimmer at high effort scores 35 and Nvidia’s Nemotron 3.5 Lightning scores 24.

Nemotron 3.5 Lightning, the other model competing for the same slot in your GPU, is a 30B mixture of experts activating about 3B parameters per token, and its own model card reports GPQA Diamond at 75.44 and Terminal-Bench 2.1 at 24.58 against Qwen’s 89.2 and 73.0. Nvidia’s harness differs from Alibaba’s, so treat the gap as approximate, but a 48-point spread on Terminal-Bench does not come from harness variance. Nemotron’s pitch is speed rather than quality, and it does deliver the speed; how much that trade is worth depends on the machine you run it on.

The reasoning dial

Simon Willison ran the 17 GB Q4_K_M build in LM Studio on a 128 GB M5 Max and asked for his standard pelican-on-a-bicycle SVG. At the default xhigh setting the model spent 22,276 reasoning tokens and 21 minutes of wall-clock time before producing 3,223 tokens of output. With reasoning turned off, the same request finished in 137 seconds. His verdict on the default was that it is “absolutely not a good way to run the model,” and he had to raise LM Studio’s context limit from 8,192 to the full 262,144 to make room for the reasoning trace.

The Hacker News launch thread had already reached the same conclusion. Several users reported that xhigh produced sprawling, degraded thinking traces on ordinary prompts, and that dropping to medium improved the behavior substantially. The card says the top setting is for complex analytical work; Alibaba shipped it as the default anyway.

This matters more on local hardware than it would through an API. Reasoning tokens are decode tokens, and decode on a laptop runs at 15 to 30 tokens per second, so twenty-two thousand of them is somewhere between twelve and twenty-five minutes of waiting before the first word of the answer. Set the effort to medium for chat and agent work, low for extraction and classification, and reserve xhigh for the hard problems where you would have waited on a hosted frontier model anyway.

What it costs in memory and speed

The weights fit; the context is where the 24 GB budget gets tight, and it is the one place Qwen’s architecture is less economical than its neighbors. Because 16 layers keep a full-attention cache at head dimension 256, the KV cache runs to roughly 16 GiB at the native 262K context in BF16 by kingy.ai’s arithmetic, or about 2 GB per 32K tokens. One Hacker News commenter measured 2.5 GB at 32K on their card and could not fit 128K of context next to the Q4 weights, whereas Gemma 4 and Glimmer, which use sliding-window attention on most layers, left room for several hundred thousand tokens on the same hardware. If your workload is long-document work at the edge of VRAM, that is a real strike against Qwen; for agentic loops under 64K it is not. The general arithmetic is in our KV cache piece.

Speed depends on which machine you own. On a MacBook M4 Max with 64 GB, developer Tom Greenwald ran Qwen and Nemotron against the same repository with the same 6K-token prompt: Qwen took 32 seconds of prefill and generated at about 15 tokens per second, while Nemotron 3.5 Lightning did 6 seconds of prefill and about 70 tokens per second. That is the dense-versus-MoE gap on a bandwidth-bound machine, and no setting closes it. Willison saw 15 to 30 tokens per second from LM Studio on his M5 Max; on a DGX Spark, he found that llama.cpp with the MTP draft mode enabled beat LM Studio’s default GGUF by about 72% on the same machine.

On a desktop GPU the picture changes. In a 16-problem test on a single RTX 5090 with all three models at Q4_K_M under Ollama, Qwen3.8-27B and Nemotron both solved 14 of 16 and Glimmer solved 15. Qwen finished the suite in 6.3 minutes at an average 171 tokens per second, Nemotron in 8.3 minutes at 216 tokens per second, and Glimmer in 16.2 minutes at 81. Qwen never hit the generation cap and had the lowest median solve time on the problems it got right, 5.9 seconds; its two misses were wrong answers, one of them propped up by an invented OEIS citation. VRAM at 64K context came to 17.5 GB for Qwen, 16.9 for Glimmer, and 25.4 for Nemotron, so on a 24 GB card the MoE is the one that does not fit at that context.

One person’s run on one card is not a study, but it does suggest that Nemotron’s speed advantage is mostly a Mac phenomenon, and that Qwen’s dense quality shows up as fewer wasted tokens rather than faster raw decode.

The download numbers, and the fakes before them

Every outlet has a download figure and they do not agree. Cybernews reported more than 3 million Hugging Face downloads within three days of the August 14 release, and Alibaba Cloud’s own blog says the model became one of the five most-liked on the hub within two days. The official Qwen/Qwen3.8-27B repository counter showed roughly 665,000 downloads and about 11,000 likes when we checked it, while Unsloth’s GGUF mirror alone showed 3.56 million. Hugging Face counts per repository, so a headline number depends entirely on which repos someone summed. If you repeat one of these figures, say which repositories it covers.

The other footnote is what filled the gap before the files arrived. Alibaba pre-announced the 27B in its August 3 Qwen3.8-Max post and said the weights would follow the next week. The Max weights arrived on August 12, the 27B on the 14th. In the meantime, searching Hugging Face for the model turned up placeholder repositories under unrelated accounts, at least one of which admitted in its own README that it held no weights (FontMirror documented these). Anyone who pulled a “Qwen3.8-27B” before the 14th did not get this model.

Which one to run

If you own the 24 GB card and the work is coding agents, terminal sessions, document parsing, or computer use, Qwen3.8-27B is the model to install this week. It beats Glimmer on every benchmark the two share, it beats Nemotron by margins that survive any harness dispute, and it fits in the same 17 GB. The tradeoff you are accepting is decode speed on bandwidth-limited hardware. On a Mac, Nemotron’s 3B active parameters will feel four to five times faster, and for a chat assistant that a person sits and watches type, speed is most of what they notice. On a desktop GPU the gap mostly disappears and Qwen’s habit of solving the problem in fewer tokens takes over, so the dense model is the better choice at the desk and the MoE still makes sense on the laptop.

Whichever way you go, change the reasoning effort before you form an opinion, because the model most people will meet on first run is the one that spends twenty minutes on a task that needs two. Alibaba got the hard part right, a checkpoint that quantizes to a size a small team already owns hardware for, and then shipped it behind a default that hides that. Nothing about the license or the weights should stop you; the default setting is the only real catch.

Sources

Want to discuss this topic?

We'd love to hear about your specific challenges and how we might help.

Start a conversation