How the New Flash Models Work: Inside GLM-5.3-Flash and Qwen3.8-Flash-Next

In most product lines, “Flash” means the smaller, cheaper tier of a model family. Two open-weight releases this year use the word differently. GLM-5.3-Flash and Qwen3.8-Flash-Next are not simply compressed copies of their flagships. Each was built around a new architecture designed to make long-context generation cheaper, and both labs disclosed enough of the design to examine how it works.

Both releases sit next to a flagship that is architecturally conservative by comparison:

Model	Total / active parameters	Attention stack	Residual path	Context	What it is
GLM-5.3	744B / 40B (Z.ai’s figure)	DeepSeek Sparse Attention on MLA, 78 layers	standard	1,048,576	GLM-5.2’s base with more post-training
GLM-5.3-Flash	320B / 18B	34 Kimi Delta Attention + 11 sparse MLA layers	4-stream mHC	1,048,576	new base trained on a 30T-token multimodal corpus
Qwen3.8-2.4T-A95B (open weights behind Qwen3.8-Max)	2.4T / 95B	69 Gated DeltaNet + 23 gated full-attention layers	standard	262,144 native	Qwen3.5-generation hybrid at very large scale
Qwen3.8-Flash-Next	125B / 6B, plus 51B n-gram tables and a 4B MTP module	36 Gated DeltaNet + 12 Qwen Sparse Attention layers	4-stream Gated Residual	262,144 native, about 1M with YaRN	preview of the Qwen4 architecture

Z.ai’s GitHub family table lists GLM-5.3 as 744B-A40B; the Hugging Face checkpoint metadata counts 753B stored parameters, which appears to include tensors outside that headline figure. The S5 Labs release coverage of GLM-5.3 used the 753B number. Qwen’s 125 + 51 + 4 split matters for the same reason: tools report the Flash-Next checkpoint as 176B or 180B depending on whether they count the MTP module.

The rest of this article goes component by component. Both designs attack the same costs but bet differently on where the bottleneck sits, and the comparison says more about where open-model architecture is heading than either release does alone. We assume familiarity with mixture-of-experts routing and the KV cache.

What a generated token costs

A decoded token pays for four things, and a different mechanism addresses each.

1. Weight reads and expert matrix multiplies. Every active parameter is read from memory and used in roughly two floating-point operations per token. An MoE reduces this by activating a subset of experts.
2. History mixing. Attention compares the new token with every earlier token. Over a full sequence that is quadratic in length, and the keys and values for every earlier token sit in the KV cache.
3. Memory traffic. During decode, the GPU does little arithmetic per byte moved. Weight bytes, KV-cache bytes, and residual-stream bytes all compete for HBM bandwidth.
4. Sequential steps. Autoregressive generation runs one forward pass per output token. Each pass carries kernel launch, synchronization, and, in an MoE, all-to-all communication overhead.

MoE addresses only the first. With 6B active parameters, Flash-Next spends roughly 12 GFLOPs per token on its weight matrices by the usual two-FLOPs-per-parameter estimate; GLM-5.3-Flash at 18B spends about 36. Neither number says anything about attention cost at a million tokens, which is where both labs made their biggest changes.

The two stacks, layer by layer

The published configs give the layer-level details.

GLM-5.3-Flash has a 4,096-dimensional hidden state and 45 decoder layers. The attention pattern repeats three Kimi Delta Attention (KDA) layers and then one sparse-attention layer, so layers 3, 7, 11 and so on up to 43 are sparse: 11 in total. The final layer, 44, is linear. The sparse layers are DeepSeek-style Multi-head Latent Attention with no rotary position embedding (NoPE), a 512-dimensional cached latent, 64 heads, and a DeepSeek Sparse Attention indexer that selects 2,048 positions. The first three layers use a dense FFN. The remaining 42 route each token to 8 of 288 experts plus one shared expert, using sigmoid scores and DeepSeek’s auxiliary-loss-free balancing. The vocabulary has 154,880 entries, and there is one multi-token-prediction (MTP) layer. A 24-layer vision encoder feeds the same backbone; Z.ai calls it the first natively multimodal model in the GLM-5 series.

Qwen3.8-Flash-Next is narrower: a 2,560-dimensional hidden state, 48 layers, and a pattern written in the model card as

12 × (3 × (Gated DeltaNet → MoE) → 1 × (Qwen Sparse Attention → MoE))

Each MoE layer holds 512 small experts (intermediate size 640) and routes each token to 10 of them plus a shared expert. The Gated DeltaNet (GDN) layers use 16 key heads and 48 value heads of dimension 128. The Qwen Sparse Attention (QSA) layers use 24 query heads and only 2 key-value heads of dimension 256, plus an indexer with 4 query heads and one shared key head. Vocabulary is 248,320. Bigram and trigram lookups into 51 billion parameters of hashed embedding tables enter the residual stream at layer 2. The config’s architecture class is Qwen4ExpForConditionalGeneration, which matches Qwen’s description of the model as a Qwen4 preview.

Both labs settled on a 3:1 ratio of recurrent to retrieval layers. Each also uses one MTP layer and a widened four-stream residual. Most of their other choices differ.

Linear attention: a fixed-size memory in place of the cache

Softmax attention keeps every past key and value. A linear-attention layer replaces that growing list with a fixed-size matrix that is updated once per token. The delta rule behind both GDN and KDA treats that matrix as an associative memory that is corrected toward each new key-value pair, rather than simply accumulated.

Gated DeltaNet writes the update as

$$S_t = \alpha_t\, S_{t-1}\left(I - \beta_t\, k_t k_t^{\top}\right) + \beta_t\, v_t k_t^{\top}, \qquad o_t = S_t\, q_t$$

Source: S_t = alpha_t * S_{t-1} (I - beta_t k_t k_t^T) + beta_t v_t k_t^T ; o_t = S_t q_t

Here $S_t$ is a $d_v \times d_k$ state matrix per head, $\beta_t$ is a learned write strength, and $\alpha_t \in (0, 1)$ is a learned scalar decay that lets the layer forget. The term $(I - \beta_t k_t k_t^{\top})$ erases what the state currently associates with $k_t$ before writing the new value. That erase step is what separates the delta rule from older linear attention, which could only add to its memory.

Kimi Delta Attention, introduced by Moonshot in the Kimi Linear paper, keeps the delta rule and replaces the scalar decay with a per-channel one:

$$S_t = S_{t-1}\,\operatorname{Diag}(\alpha_t)\left(I - \beta_t\, k_t k_t^{\top}\right) + \beta_t\, v_t k_t^{\top}$$

Source: S_t = S_{t-1} Diag(alpha_t) (I - beta_t k_t k_t^T) + beta_t v_t k_t^T

(The Kimi paper writes the state transposed; the content is the same. Z.ai’s config and the vLLM recipe label Flash’s linear layers KDA without expanding the acronym, so reading them as Kimi Delta Attention is our inference from the name.) A diagonal decay lets each key dimension forget at its own rate, so one head can hold short-lived and long-lived information side by side. Moonshot describes KDA as Gated DeltaNet with finer-grained gating, and reported up to 75% less KV cache and up to 6× decoding throughput at 1M context for its own 48B-A3B hybrid.

The memory consequence is easiest to see with the published configs. The arithmetic below is ours, assumes BF16, takes 1M as 10^6 tokens, leaves out indexer keys, and ignores implementation padding:

Component	GLM-5.3-Flash	Qwen3.8-Flash-Next
Recurrent state per layer	64 heads × 128 × 128 ≈ 1.05M values	48 value heads × 128 × 128 ≈ 0.79M values
Recurrent state, all linear layers	34 layers ≈ 35.7M values ≈ 71 MB	36 layers ≈ 28.3M values ≈ 57 MB
Cached values per token, sparse layers	11 × 512 latent = 5,632	12 × 2 heads × 256 × 2 (K and V) = 12,288
Sparse-layer cache at 1M tokens	≈ 11.3 GB	≈ 24.6 GB

The recurrent state is the same size whether the conversation is ten tokens or a million. Recurrent state is often held in FP32, which doubles those figures, and Flash-Next’s config sets mamba_ssm_dtype to float32. The per-token cache now comes only from the sparse layers. Had every Qwen layer kept the same QSA cache, the 1M-token figure would be about 98 GB; the 3:1 hybrid cuts it by four. For contrast, full GLM-5.3 caches a 512-value MLA latent plus a 64-value decoupled RoPE key (576 values) in each of its 78 layers, about 45,000 values per token or roughly 90 GB at 1M before indexer keys. The per-layer average in our arithmetic falls by about 4.6×, close to the 4.4× KV reduction Z.ai reports for Flash against GLM-5.3 using its own normalization.

GLM’s sparse layers cache less per token than Qwen’s because MLA stores one compressed latent rather than separate keys and values. Z.ai also compared Flash’s per-layer cache with DeepSeek-V4-Flash and Kimi K3. GLM-5.3-Flash had the lowest attention compute in that comparison, but its KV cache was still slightly larger than both, which Z.ai said leaves room for improvement.

Why neither lab went fully recurrent

A fixed-size state cannot store a million tokens verbatim. It compresses, and compression loses exact detail: a variable name defined 400,000 tokens ago, a clause in a contract, the exact text of an error message. Qwen’s report states the design logic directly. The GDN layers compress history, and one global-attention layer in four is retained because it is difficult for any finite-state memory to reproduce earlier tokens exactly.

Z.ai has tested this trade-off before. The GLM-5 technical report tested efficient-attention alternatives on a 9B model, including sliding-window interleaving and GDN variants with half the layers kept as full attention, and found an accuracy gap on fine-grained long-context retrieval at 64K and 128K. The report contrasts those results with DeepSeek Sparse Attention, which it calls lossless by construction. That contrast reads as the case for GLM-5 adopting DSA rather than a recurrent hybrid, though the report does not frame the choice in those words. GLM-5.3-Flash now uses a recurrent hybrid after all, while keeping a sparse retrieval layer every fourth layer. The public material we reviewed does not explain what changed. Plausible candidates include KDA’s finer gating, the new base training run, and the retained sparse layers, but we found no Z.ai ablation that separates them.

Sparse retrieval and the indexer problem

The retrieval layers in both models are learned sparse attention. A small indexer scores past positions for each query, a top-k step chooses which positions to keep, and full attention runs over only the selected keys and values.

GLM inherits DeepSeek Sparse Attention from DeepSeek-V3.2. DeepSeek’s lightning indexer scores a query token $t$ against a past token $s$ with a handful of cheap heads:

$$I_{t,s} = \sum_{j=1}^{H^{I}} w_{t,j}\,\operatorname{ReLU}\!\left(q^{I}_{t,j} \cdot k^{I}_{s}\right)$$

Source: I_{t,s} = sum_j w_{t,j} * ReLU(q^I_{t,j} . k^I_s)

The top 2,048 positions by $I_{t,s}$ go into the main attention. Core attention cost then scales with $k = 2{,}048$ rather than with context length. The indexer itself still touches every past position, though, and at a million tokens its latency and memory overhead matter enough that Z.ai built IndexPool to reduce them.

Both labs therefore made the indexer cheaper as well:

• IndexShare (GLM-5.2, carried into GLM-5.3) computes the indexer’s top-k once and reuses it across a group of four sparse layers. Z.ai reported 2.9× lower per-token FLOPs at 1M context when it introduced the feature, without saying whether that covers the indexer alone or the whole model.
• IndexPool (GLM-5.3-Flash) compresses four indexer key vectors into one through weighted pooling, cutting indexer latency and memory at million-token length. The config flags index_kpool: 4 and index_kpool_always_select_tail, which, going by the flag name, keeps the newest, not-yet-pooled tokens in the selection.
• QSA’s block indexer (Qwen3.8-Flash-Next) splits the history into non-overlapping micro-blocks of $r = 4$ tokens, scores compressed blocks rather than tokens, selects up to 512 blocks, and expands them back to 2,048 token positions plus the incomplete tail block. Indexer cost falls from $O(n^2)$ to $O(n^2/r)$ over a sequence.

IndexPool and QSA’s micro-blocks both operate at a granularity of four tokens, but the two sources describe the selection step differently. Qwen selects whole blocks, which gives the attention kernel contiguous four-token runs to read, at the cost of spending part of the budget on neighbours of the token that mattered. Z.ai’s public material does not say whether GLM’s top-k is applied to the pooled entries or to individual tokens, so we do not compare the two on that point.

Qwen’s ablations suggest the sparse layers did not cost quality. On the continued-pretraining checkpoints, QSA matched or beat the full-attention version on seven of eight benchmarks and raised the average from 75.9 to 76.8. Long-context results moved further: RULER beyond 512K rose from 90.08 to 93.00, and 8-needle MRCR rose from 30.66 to 40.53 at 512K and from 20.71 to 26.44 at 1M. Learned selection may help as well as save compute, possibly because the model attends to fewer distractors, though the report offers no mechanism study. Those are Qwen’s own measurements on its own checkpoints.

At the kernel level, Qwen reports 7.6× faster prefill and 4.9× faster decode for the attention module at 1M context against FlashInfer’s paged GQA kernel, with speedups starting from 64K. Those figures cover the attention module only. End-to-end gains are smaller, because MoE layers, the recurrent layers, and communication are unchanged. Qwen also reports, and NVIDIA’s blog repeats, 8.6× the prefill throughput of Qwen3.7-Plus at 1M context with a 90% prefix-cache hit rate. That figure combines architecture and cache reuse, so it describes a particular workload rather than a general speedup.

Widening the residual stream

The residual-stream change has drawn less discussion than the attention changes, though for training it may matter more. Both models replace the single residual stream of a standard Transformer with four parallel streams, and they control the mixing in opposite ways.

A standard block computes $x_{l+1} = x_l + F(x_l)$. Hyper-connections widen $x$ into $n$ streams and learn how to read a block’s input from them and write its output back. Unconstrained, those learned mixing matrices can amplify or cancel signal across depth, which is the instability the identity residual was invented to avoid.

Manifold-Constrained Hyper-Connections (mHC, from DeepSeek) restore the identity-like property by projecting the stream-mixing matrix onto doubly stochastic matrices, where every row and column sums to one. Schematically,

$$x_{l+1} = H^{\text{res}}_l\, x_l + \left(H^{\text{post}}_l\right)^{\top} F\!\left(H^{\text{pre}}_l\, x_l\right), \qquad H^{\text{res}}_l \in \text{doubly stochastic}$$

Source: x_{l+1} = H_res x_l + H_post^T F(H_pre x_l), with H_res projected to doubly stochastic

A doubly stochastic mixing matrix redistributes signal between streams without changing its total, and a product of such matrices is still doubly stochastic, so the residual path cannot amplify signal with depth and the stream average is carried through intact. GLM-5.3-Flash uses four streams (hc_mult: 4) and 20 Sinkhorn iterations for the projection. Its hyper-connection parameters are kept out of quantization.

Gated Residual (GR), Qwen’s design, drops cross-stream mixing entirely. Each block reads its input as an average of the four branches, each multiplied by an element-wise sigmoid gate computed from the data through a low-rank bottleneck of rank $d/8$ (320 for Flash-Next). It writes its output back to each branch scaled by one data-dependent scalar, $s = 2\sigma(\cdot)$. There is no branch-mixing operator, and the attention and MLP blocks in every layer each get their own GR module.

Qwen’s technical report is unusually specific about what GR buys. In a stress test at four times the optimal learning rate on a 28-layer MoE, an AdamW baseline with the Qwen3.5 structure recorded 183 loss spikes per 10,000 steps and crossed the gradient-clipping threshold on 213 of 19,932 steps. Both Muon runs, with and without GR, stayed under the clipping threshold, and only the configuration with the gated residual recorded zero loss spikes. In a separate ablation at three times the optimal rate, adding the gate cut spikes from 32.0 to 3.2 per 10,000 steps. The report calls the gate a key contributor to the stability margin over Qwen3.5 and says the full-scale run had no loss spikes and needed no QK-clip or SwiGLU-clip.

The gates also bound the values written into the residual state. Qwen reports that storing the four branches in FP8 halves the bytes moved for the residual state relative to BF16 with almost no quality loss. Four BF16 streams would quadruple residual traffic; by our arithmetic FP8 storage brings that back to twice a single BF16 stream, so the gate design recovers part of the widening’s cost at inference time as well as in training.

Qwen’s third tier: n-gram memory in host RAM

Flash-Next’s most unusual component is its n-gram embedding. At layer 2, the model looks up the current bigram and trigram in hashed embedding tables (a base table of 20 million entries, eight hash heads per n-gram order) and injects the result into the residual stream through a contextual gate. The tables hold 51 billion parameters, more than 40% of the size of the neural backbone.

The lookup address depends only on the input tokens. Nothing computed by the network decides which rows are needed, so the runtime knows what to fetch as soon as the tokens arrive. Qwen uses that to keep the tables in host memory and prefetch rows asynchronously. Placing the injection at layer 2 rather than layer 0 gives the transfer the duration of layer 1’s computation to complete. SGLang exposes this as --ple-offload-embedding, which puts the table in pinned CPU memory, and can instead back it with files on NVMe. NVIDIA’s Dynamo recipe asks for at least 51 GB of host memory per worker.

The result is a three-tier parameter hierarchy:

1. Dense and recurrent weights that every token uses, on the accelerator.
2. Expert weights that each token uses sparsely, on the accelerator and spread across GPUs.
3. Lookup tables that each token reads a few rows from, in host DRAM or on disk.

Qwen’s own scaling result adds a caution. In one sweep that added n-gram parameters, enlarging the vocabulary from 20× to 200× the base lowered training loss monotonically, yet downstream accuracy saturated or fluctuated. In a second sweep at a fixed total parameter budget, where expert parameters were cut to pay for the tables, loss was lowest at 10× the base vocabulary (tables at 25% of parameters) and downstream benchmarks showed no clear improvement over an MoE-only baseline. The tables beat having none in the first sweep, but past a modest size, growing them mostly lowered loss rather than improving capability. That divergence between loss and capability recurs throughout the report.

Training the hybrids

Data and curriculum

Z.ai has disclosed more about data. The GLM-5 report describes a base model that began with a 27-trillion-token corpus and totals 28.5T tokens across all base stages. The paper does not reconcile the two figures, but the mid-training stages sum to about 1.55T. Z.ai says GLM-5.3 shares GLM-5.2’s base but has not said how that base relates to GLM-5’s (the configured context window grew from 202,752 to 1,048,576 tokens), so read the details below as the documented GLM-5 recipe, not a confirmed description of GLM-5.3’s base. Web data passed through a sentence-embedding DCLM-style classifier and a separate world-knowledge classifier. Math and science documents were scored by language models for educational value, with synthetic and template-generated text excluded. Mid-training extended context in three stages, 32K for 1T tokens, 128K for 500B, and 200K for 50B, and added about 10 million GitHub issue-PR pairs (roughly 160B unique tokens) built from repository-level code, diffs and discussions.

GLM-5.3 received no new pre-training. Z.ai says it uses the same base as GLM-5.2 and that every gain came from post-training. The benchmark deltas are large for a post-training-only release: Terminal Bench 2.1 from 81.0 to 88.2, DeepSWE v1.1 from 46.2 to 66.9, CyberGym from 77.2 to 84.5, and Terminal Bench 3.0 from 4.6 to 28.3. With the base held fixed, these differences isolate post-training better than a generation-to-generation comparison can, although they remain Z.ai’s numbers under Z.ai’s harness settings.

GLM-5.3-Flash is a new base, trained on what Z.ai calls its latest 30T-token multimodal pre-training corpus. The modality breakdown and training compute have not been published, and there is no dedicated Flash paper; the model card cites the GLM-5 report.

Qwen has not disclosed a total token count for Flash-Next. The report gives only relative figures: about one third of the training tokens and roughly one ninth of the training FLOPs of Qwen3.7-Plus, a 397B-A17B model. With those budgets, Flash-Next-Base led Qwen3.7-Plus-Base on 8 of 14 pre-training benchmarks and trailed on the other six by at most 2.6 points.

Converting dense attention into sparse attention

Both labs train the freshly added indexer before adapting the whole model, because a poorly trained indexer hides information from a backbone that has learned to depend on it. Qwen distills its indexer from dense attention; the GLM-5 report describes a warm-up stage without naming its objective.

GLM-5’s conversion starts from the end of mid-training. A warm-up of 1,000 steps on 14 sequences of 202,752 tokens trains the indexer, followed by a sparse-adaptation stage of 20 billion tokens, against roughly 944 billion for DeepSeek-V3.2’s equivalent stage. A GLM-4.7-Flash ablation shows why the second stage matters. Warming only the indexer while freezing the base dropped RULER at 128K from 79.21 to 71.35. After 150B tokens of joint training, the sparse model beat the dense baseline at 16K, 32K and 64K, though it still trailed by 0.35 points at 128K and sat slightly below at 4K and 8K.

Qwen’s QSA conversion runs during continued pretraining at 256K sequence length. Stage one trains only the indexer for 1,000 steps (about 2B tokens), minimizing the KL divergence between the indexer’s score distribution and the dense model’s attention distribution. Stage two trains indexer and backbone together for 8,000 steps, about 200B tokens. A fused kernel computes the sparse attention output and the KL loss without materializing intermediate tensors.

The GLM-5 report also records a problem relevant to anyone running RL on a sparse-attention model. Non-deterministic top-k operators in CUDA and TileLang caused drastic RL degradation within a few steps, with a sharp entropy drop. Z.ai switched to deterministic torch.topk and froze the indexer parameters during RL. The report attributes the failure to a mismatch between training and inference that a deterministic operator removes. Our reading of why RL is so sensitive: small differences in which tokens are visible change the probabilities the trainer and the sampler assign to the same trajectory.

Optimizers

Both labs use Muon, which orthogonalizes each weight matrix’s momentum update with Newton-Schulz iterations so the update is well conditioned across directions.

GLM-5 introduced Muon Split. Plain Muon on MLA’s projection matrices underperformed grouped-query attention, so Z.ai splits those matrices by attention head and orthogonalizes each head’s slice separately. With Muon Split, MLA roughly matched GQA-8 on average in the report’s ablation, and attention-logit scale stayed stable through pre-training without clipping.

Qwen partitions parameters by role. Muon handles the attention Q, K, V and output projections, the GDN input and output projections, the routed and shared expert matrices, and the n-gram key and value projections. AdamW keeps the input embeddings, the output head, the MoE router, and the low-rank GR parameters. On the router, Qwen found that Muon worsened early-training fluctuations and destabilized routing; applied later in training it caused no instability but gave no significant gain.

With the new stability margin, Qwen refit its learning-rate and batch-size scaling law. On a 48-layer 156B-A7B validation model trained for 419B tokens, the new fit chose an 8.4M-token batch and a learning rate of $1.76 \times 10^{-3}$, against 4.2M and $6.8 \times 10^{-4}$ for the Qwen3.5 recipe. Batch-size warmup was removed; ramping cost 18.8% more optimizer steps with no gain. The report states the limits. The loss optimum is a flat bowl, with settings within about $\sqrt{2}\times$ of the learning rate and +25% batch landing within $7 \times 10^{-4}$ of each other in loss, and the law was validated on a 419B-token run.

When loss and capability disagree

Qwen’s report contains three cases where pretraining loss misled. Larger n-gram tables kept lowering loss while downstream accuracy saturated. Restricting writes to the top two residual branches was nearly free in pretraining loss but degraded quality after post-training. Removing positional encoding from the full-attention layers made little difference in pretraining and then raised the rate of endless generation after post-training. In each case, an ablation scored on pretraining loss alone would have accepted a change that later cost quality. GLM-5.3-Flash ships NoPE sparse layers, which makes Qwen’s NoPE result worth noting: it does not show that GLM’s choice is wrong, since the two architectures differ, but it does show the kind of evaluation needed to defend it.

Post-training

Z.ai’s GLM-5 post-training sequence is documented in detail: multi-task SFT, then Reasoning RL, Agentic RL and General RL, with on-policy cross-stage distillation as the final stage to recover skills from earlier checkpoints. In SFT, erroneous segments of agent trajectories stay in context but are masked from the loss, so the model learns to recover from mistakes without being trained to make them. Reasoning RL uses GRPO with IcePop-style masking of mismatched policy ratios, no KL term, and fully on-policy updates. Agentic RL is asynchronous: inference engines on separate GPUs generate trajectories continuously across more than 10,000 software-engineering and terminal scenarios plus multi-hop search tasks, with weights pushed to them every K gradient updates. That makes it only approximately on-policy.

Z.ai says GLM-5.3’s gains came entirely from post-training on the GLM-5.2 base, but it has not published a GLM-5.3 recipe, so the GLM-5 report is the nearest public description, not a record of what changed for 5.3. Qwen’s Flash-Next report concentrates on architecture and pre-training and describes no post-training recipe.

Decoding several tokens per pass

Both models ship a multi-token-prediction module trained to predict tokens beyond the next one. At inference it becomes a built-in draft model: it proposes several tokens, and the main model verifies all of them in a single forward pass. Each accepted draft token saves one sequential step.

Both labs integrated MTP with sparse attention so that drafting does not repeat indexer work:

• GLM’s config sets index_share_for_mtp_iteration, reusing the indexer’s selections across MTP steps. GLM-5 trained three parameter-shared MTP layers and reported an average acceptance length of 2.76 tokens against 2.55 for DeepSeek-V3.2, measured on Z.ai’s private prompt set with four speculative steps; GLM-5.2 reported up to 20% longer acceptance from an improved MTP layer.
• Qwen’s MTP layer is itself a QSA layer and reuses top-k indices across prediction steps. In four-step speculative decoding, mean accepted length was essentially unchanged by index reuse: 4.06 with full attention, 4.07 with QSA.

Serving recipes differ in how far they push the draft. vLLM’s GLM-5.3-Flash recipe drafts five tokens. SGLang’s Flash-Next cookbook drafts four. On DGX Spark it reports acceptance lengths of 3.5 to 3.7 for non-thinking output and about 2.5 for thinking output. On a single RTX PRO 6000 Blackwell GPU with the NVFP4 checkpoint, MTP cut time per output token for one request from about 11.4 ms to 5.9 ms. The cookbook does not explain the thinking-mode drop; reasoning text being harder to predict than ordinary prose is a plausible cause.

Precision is now assigned per tensor

Neither model fits a description like “an INT8 model.” The released checkpoints and NVIDIA’s quantized variants assign precision tensor by tensor.

• GLM-5.3 and GLM-5.3-Flash ship native FP8 checkpoints (E4M3, 128×128 weight blocks, dynamic activation scaling) alongside BF16 repositories. In the Flash config, the FP8 exclusion list covers the embeddings and LM head, the MoE router gates, the mHC parameters, the vision tower, and the projections, convolutions, decay and gating parameters of all 34 KDA layers. GLM-5’s report describes INT4 quantization-aware training during SFT, with one kernel used for both training and offline weight quantization so that training and inference behave bit-identically.
• NVIDIA’s Qwen3.8-Flash-Next-NVFP4 quantizes the routed experts to 4-bit NVFP4, keeps attention, shared experts and the other main-model layers in BF16, stores the n-gram tables in per-tensor FP8, and puts the MTP experts in block FP8. The result is about 2.7× smaller than BF16.
• NVIDIA’s Qwen3.8-2.4T-A95B-NVFP4 uses NVFP4 experts, FP8 attention and linear-attention projections, and an FP8 KV cache, and leaves the MTP module unquantized. NVIDIA’s card compares the 4-bit build with BF16 on six benchmarks. NVFP4 matches or slightly exceeds BF16 on five (GPQA Diamond 92.58 versus 92.55, SciCode 56.21 versus 54.44, IFBench 81.73 versus 79.93) and trails on HLE, 40.55 versus 41.43. The card reports no run-to-run variance, so gaps this small in either direction should not be read as a gain or a loss.

The configs and cards record which tensors get which format, not why, so the reasons here are our reading. Across these checkpoints, bulk expert weights take the lowest precision; they dominate memory and each one is used rarely. Routers, gates, recurrent decay parameters and residual-mixing parameters stay at high precision; they are small and sensitive. KV and residual state get their own formats because they dominate bandwidth during decode. We found no sign of unstructured pruning in either lab’s material. Their sparsity is structured (experts, top-k attention, table lookups), which maps onto large matrix multiplies and predictable memory access.

Serving: prefill and decode are different problems

Prefill processes the whole prompt in large batches and is mostly compute-bound. Decode produces one token per sequence per step and is mostly bound by memory bandwidth and communication. Serving recipes for both models increasingly run the two phases on separate GPU pools.

vLLM’s GLM-5.3-Flash recipe shows a prefill instance on four GPUs handing KV and recurrent state to a decode instance on four more through NIXL. A hybrid model has to transfer both kinds of state, so the recipe sets layout variables for the convolution state as well as the KV cache. NVIDIA’s Dynamo recipe for GB200 runs two prefill workers and one decode worker, with KV transfer over Mooncake on NVLink. For Flash-Next, NVIDIA’s Dynamo recipe disaggregates one prefill and one decode worker over NIXL and InfiniBand and offloads the n-gram tables to CPU memory.

Expert routing adds all-to-all traffic: after the router picks experts, tokens must reach the GPUs holding them. The bigger the expert set, the more that traffic dominates. NVIDIA makes this argument for its day-zero results for Qwen3.8-Max, with 95B active parameters, on a GB300 NVL72, saying the 72-GPU NVLink domain avoids bottlenecks that appear when expert traffic must cross traditional off-the-shelf networks. Flash-Next reduces the neural traffic but adds a host-memory path that has to keep up with token arrival.

The published numbers, with their caveats

Model	Setup	Reported result	Read it as
Qwen3.8-Flash-Next	FP8, TensorRT-LLM, GB300 NVL72	over 16,000 tok/s per GPU; over 200 tok/s per user	separate points on a throughput-versus-interactivity curve; NVIDIA calls support best-effort day 0
Qwen3.8-2.4T-A95B	FP8, GB300 NVL72	over 4,000 tok/s per GPU; over 350 tok/s per user	also curve points, without additional model tuning
GLM-5.3-Flash	SGLang, B200, NVFP4 checkpoint, one aggregated worker, concurrency 56	579 output tok/s per GPU; 50.7 tok/s per user (P50); TTFT 189 ms	agentic workload subset; speculative acceptance simulated at 2.57 rather than measured
GLM-5.3-Flash	SGLang, GB200, NVFP4 checkpoint, 2 prefill + 1 decode, concurrency 128	620 tok/s per GPU; 80.1 tok/s per user (P50); TTFT 4,680 ms	more per-user speed and throughput, with a much longer wait for the first token; same simulated acceptance length
GLM-5.3-Flash	eight DGX Spark nodes	roughly 80 tok/s generation	concurrency and precision not stated

The rows are not comparable to one another. The Qwen figures are rack-scale Blackwell Ultra results at the best point on a curve. The GLM figures are fixed operating points from a recipe page that uses an NVFP4 checkpoint with an FP8 KV cache, and every one of its rows simulates draft acceptance rather than measuring it. The same recipe also lists an aggregated GB200 configuration at concurrency 56: 590 tok/s per GPU and 51 tok/s per user. Against that row, the disaggregated setup raised per-user decode speed by about 57% at more than twice the concurrency on three times the GPUs (12 versus 4), while its median time to first token was 4.7 seconds.

When evaluating either model, measure time to first token, inter-token latency, per-user decode speed, aggregate throughput, prompt and output lengths, concurrency, and prefix-cache hit rate together. Sparse attention pays off most on long prompts. A benchmark built from short prompts will hide most of the difference between these models and a conventional dense-attention MoE.

What the evaluations do not cover

Qwen’s report is a strong architecture evaluation. It tests each change against baselines in ablations, reports the final base model on a 14-benchmark suite, and publishes variants that failed. The post-trained model card adds vendor-run agent results (its DeepSWE 1.1 score is the better of two harnesses): 62.5 on SWE-bench Pro, 81.0 on SWE-bench Multilingual, 58.7 on DeepSWE 1.1 and 73.9 on Qwen’s in-house CoWorkBench, against 55.8, 75.8, 16.5 and 65.1 for Qwen3.7-Plus on the same card. The Flash-Next release coverage discusses those numbers and the license.

GLM-5.3 has the most notable independent evaluation of the group. NIST’s Center for AI Standards and Innovation called it the most cyber-capable open-weight model released to date and estimated it trails the U.S. frontier by about four months on CAISI’s aggregate cyber index. CAISI measured 40.4% on SEC-Bench Pro, 61.1% on ExploitBench, 9.4% on userspace ExploitGym and 7.7% on OSS-Fuzz, against best U.S. frontier scores of 90.2%, 100%, 44.4% and 23.2%. Z.ai’s model card says cyber capability “developed faster than we expected” as post-training scaled. That result applies to the flagship. NIST did not evaluate Flash.

The gaps are similar for both labs. Neither lab’s model card or technical report that we reviewed includes a standardized hallucination or calibration evaluation for these models, and coding-agent pass rates do not measure whether a model invents a citation or abstains when it should. Sparse retrieval adds a failure mode that aggregate long-context scores can hide: an indexer that consistently misses a rare but important kind of key. A team deploying either model on long documents should test at least:

• factual accuracy and abstention on questions with no answer in the context;
• retrieval accuracy as the target moves across positions and distances, including content near pooling and block boundaries;
• adversarial distractors designed to look relevant to the indexer;
• multi-hop questions whose evidence is spread across the context;
• recovery from tool errors over long agent trajectories;
• the same suite at the reasoning effort and context length planned for production.

Two points on the same frontier

GLM-5.3-Flash keeps more active compute (18B) and a compact MLA cache in its sparse layers. Its model card points to the GLM-5 report, which documents Z.ai’s agent post-training stack in detail, but Z.ai has not published a Flash-specific recipe. Its million-token context is configured in the checkpoint itself, it is natively multimodal, and it is MIT-licensed. The costs are three times Flash-Next’s active compute and more expert weight traffic per token.

Qwen3.8-Flash-Next pushes active compute down to 6B. It adds capacity back through 512 small experts, a four-branch gated residual, and 51B parameters of lookup memory that do not need accelerator memory. Its architecture and optimizer evidence is the more thorough of the two, but more mechanisms have to work well together. A host-memory path joins the serving critical path, and the release is an experimental preview of the architecture Qwen says will underpin Qwen4, so details may change. Its 262K native context needs YaRN to reach a million, and its Qwen Community License requires a separate license for model-as-a-service and standalone coding or office-assistant businesses.

That framing is our reading of the published architectures, not a goal either lab has stated. It does suggest how to choose. If long-document fidelity and licensing simplicity matter most, GLM-5.3-Flash is the more conservative candidate. If cost per token at high utilization is the constraint and the license fits the product, Flash-Next is the more aggressive one. Either way, the comparison is only meaningful at the context lengths and concurrency the application will run.

A single token in either model passes through recurrent state updates, selective retrieval from old context, a handful of experts out of hundreds, multi-stream residuals and, in Qwen’s case, a table lookup that came from host memory. Each of those components exists to remove a different bottleneck. DeepSeek V4’s compressed sparse attention took the same approach with different components. The cost of serving a model now depends on how its computation is arranged as much as on its parameter count.

Sources

• GLM-5.3 model card and benchmark table — Z.ai on Hugging Face
• GLM-5.3 config.json — Z.ai on Hugging Face
• GLM-5.3-Flash model card — Z.ai on Hugging Face
• GLM-5.3-Flash config.json — Z.ai on Hugging Face
• GLM-5.3-Flash overview — Z.ai developer documentation
• GLM-5 family repository — Z.ai on GitHub
• GLM-5: from Vibe Coding to Agentic Engineering — Z.ai and Tsinghua University, arXiv
• Qwen3.8-Flash-Next model card — Qwen on Hugging Face
• Qwen3.8-Flash-Next config.json — Qwen on Hugging Face
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability — Qwen Team, arXiv
• Qwen3.8-Flash-Next: A New Architecture, Towards Ultimate Cost-Efficiency — Alibaba Cloud
• Qwen3.8-2.4T-A95B model card — Qwen on Hugging Face
• Gated Delta Networks: Improving Mamba2 with Delta Rule — arXiv
• Kimi Linear: An Expressive, Efficient Attention Architecture — Moonshot AI, arXiv
• DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models — DeepSeek, arXiv
• Hyper-Connections — arXiv
• mHC: Manifold-Constrained Hyper-Connections — DeepSeek, arXiv
• Muon: An optimizer for hidden layers in neural networks — Keller Jordan
• vLLM recipe: GLM-5.3-Flash — vLLM
• SGLang cookbook: Qwen3.8-Flash-Next — SGLang
• Dynamo recipe: GLM-5.3-Flash — NVIDIA
• Dynamo recipe: Qwen3.8-Flash-Next — NVIDIA
• Experiment with Qwen3.8-Flash-Next on NVIDIA GB300 NVL72 for Agentic Coding — NVIDIA Technical Blog
• Serve Qwen3.8-2.4T-A95B, a 2.4T-Parameter Model, with Configurable Reasoning on NVIDIA GB300 NVL72 — NVIDIA Technical Blog
• Running Frontier-Class AI Models Across 8 NVIDIA DGX Spark Nodes — NVIDIA Technical Blog
• nvidia/Qwen3.8-Flash-Next-NVFP4 — NVIDIA on Hugging Face
• nvidia/Qwen3.8-2.4T-A95B-NVFP4 — NVIDIA on Hugging Face
• CAISI’s Assessment of Z.ai’s GLM-5.3 Cyber Capabilities — NIST