Budget for more than the weights
GPU-addressable memory must hold model weights, runtime buffers and the context cache. Shared memory also serves the operating system. A download that fits on disk is not proof that the model fits on the GPU.
Living Buying Guide
Best Local LLM Machines (2026)
A practical, vendor-neutral guide to the computers that run large language models on your own hardware, compared on the specs that decide the outcome: how much memory the GPU can reach, how fast that memory is, and how mature the local software stack is.
Executive summary
Choose a computer around the model and workload you intend to use. This guide compares nine systems across four form factors, with separate notes for announced hardware, current configuration prices and inference constraints. It is a specification review, not a hands-on benchmark ranking.
Measure your workload
Memory bandwidth often limits single-user decoding. Prompt processing, batching, quantization kernels and attention can change the bottleneck. A bandwidth ratio is not a measured speedup.
Mac Studio is changing generations
Apple says M5 Max and M5 Ultra Mac Studio became available September 22. The 512GB configuration is due in late October. Availability does not establish local inference performance; check delivery for the exact configuration.
Start with a model you can evaluate
Try the same documents, coding tasks and context length on your existing computer first. Buy more capacity when a useful workload needs it, rather than choosing a machine around the largest parameter count.
Before buying
Run a small evaluation before ordering hardware: use representative prompts, record the model and quantization, and test the context length you actually need. A larger checkpoint is useful only if it improves that workload enough to justify the cost.
What matters when buying
Start with memory and software compatibility, then measure latency. Quantization reduces weight storage, but context and runtime allocations still consume memory. Our KV cache and memory wall guide explains why a model that loads can still become impractical with long prompts.
Usable accelerator memory
Count the full quantized checkpoint, not just the active parameters of a mixture-of-experts model. Leave room for context, image encoders, runtime allocations and concurrent requests.
Latency and throughput
Time both the first token and subsequent generation. Long prompts can be compute-bound; decode can be bandwidth-bound. Test the runtime and quantization you intend to deploy.
Supported software
Check the exact model architecture, accelerator, operating system and runtime release together. Support for a GPU family does not mean every model format works on it.
Deployment conditions
Account for cooling, power, storage, support and access controls. A module power limit is not the complete system’s wall power, and a local endpoint still needs authentication if other people can reach it.
Mac Studio availability and configuration prices
Apple’s August 25 announcement replaces the old Mac Studio capacity limits in this guide. Apple announced M5 availability on September 22, with 512GB configurations planned for late October. NVIDIA’s 64GB DGX Spark is due from OEM partners on October 23. Elsewhere, compare exact SKUs: Framework lists DIY system selections before extras, and GMKtec prices its 64GB and 128GB EVO-X2 variants separately. Prices have moved during 2026. NVIDIA raised the DGX Spark MSRP in February, citing memory supply, and now lists its Jetson developer kits at $5,499 and $399. A manufacturer specification page does not prove local stock.
Four form factors, not one market
Unified-memory desktops, GB10 appliances, GPU towers and embedded kits serve different needs. Choose the deployment format before comparing prices within it.
Unified-memory desktops
CPU and GPU share a memory pool. The amount available to inference depends on the operating system, firmware and other applications. AMD’s newer Ryzen AI Max+ PRO 495 systems raise the x86 ceiling from 128GB to 192GB, at a much higher price.
NVIDIA GB10 appliances
Compact Arm systems with NVIDIA GB10. Alongside the 128GB systems, NVIDIA announced a 64GB DGX Spark on October 2, sold only through Acer, ASUS, Dell, Gigabyte, HP and MSI from October 23, starting at $4,999. Check the exact SKU, memory and Arm software support.
Discrete-GPU workstations
A discrete GPU supplies its own VRAM. System RAM is separate; offloading can extend capacity but changes performance.
Edge and embedded systems
Developer kits for robotics, cameras and embedded inference. Choose these for device integration, not as interchangeable desktop chat computers.
Capacity and interactivity are different purchases
A 128GB shared-memory desktop and a 96GB discrete GPU expose different amounts of usable memory to the runtime. Published bandwidth helps explain potential bottlenecks, but it does not measure prompt processing, decoding or concurrent serving. Compare the same checkpoint and workload on both systems before drawing a speed conclusion.
Pick your machine: five buyers, five answers
Most readers fall into one of five buckets. Find yours, then read that machine's full card below before deciding.
A desktop for Apple-silicon inference
Mac Studio, with the delivery date checked
MLX and Metal provide an established local inference path. Compare the new M5 configurations with any older machine you already own. If 64GB is enough, a Mac mini with M5 Pro offers it at 307 GB/s.
Watch out: The 512GB configuration is due in late October. Check configuration-specific delivery and runtime support before committing a workload.
128GB in an x86 desktop
Framework Desktop, HP Z2 Mini G1a or GMKtec EVO-X2
These Ryzen AI Max systems offer substantial shared memory without moving to an Arm machine.
Watch out: Firmware allocation and runtime support matter. Systems with 192GB and the Ryzen AI Max+ PRO 495, such as the Framework Desktop 400 edition and GMKtec EVO-X5 Pro, list from $6,799, roughly double the 128GB models. Compare complete configurations and service terms, not headline base prices.
NVIDIA development in a compact appliance
DGX Spark or ASUS Ascent GX10
GB10 provides CUDA and high-speed networking for documented distributed inference workflows.
Watch out: The 64GB version halves the memory of the 128GB systems. Check Arm-compatible packages and the specific cluster recipe. Linking machines does not guarantee low latency.
A workstation with replaceable graphics
HP Z2 Tower G1i with a specified GPU
A supported discrete GPU gives you dedicated VRAM and a conventional CUDA workstation path.
Watch out: An entry-level tower is not the 96GB configuration. Obtain an exact GPU, power supply and support quote.
AI beside cameras or machinery
Jetson AGX Thor or Orin Nano Super
Jetson combines embedded I/O with NVIDIA’s deployment stack.
Watch out: Choose the kit, carrier and JetPack release for the actual peripherals. Leave memory for the rest of the application.
Spec comparison at a glance
Capacity and bandwidth are manufacturer specifications or explicitly labelled theoretical calculations. Prices are dated US storefront snapshots, before taxes and configuration changes. We rechecked the listed prices on October 6, 2026, except the 128GB DGX Spark checkout price, which we could not confirm. See the methodology for review scope; confirm the exact configuration before buying. Where a current price could not be verified, the table says so.
| Machine | Class | Memory | Bandwidth | Price snapshot | Buying constraint |
|---|---|---|---|---|---|
| Mac Studio M5 Max | Unified desktop | 36–128GB | 460–614 GB/s | From $2,499 | Validate after delivery |
| Mac Studio M5 Ultra | Unified desktop | 96–512GB | 1.2 TB/s | From $5,499 | 512GB due late October |
| Framework Desktop | Unified desktop | 32–128GB | 256 GB/s theoretical | $1,269–$3,449 before extras; 64GB out of stock | GPU allocation dependent |
| HP Z2 Mini G1a | Unified desktop | Up to 128GB / 96GB GPU | ~256 GB/s theoretical | Quote | GPU allocation dependent |
| GMKtec EVO-X2 | Unified desktop | 64 or 128GB | 256 GB/s theoretical | $2,199.99 (64GB); $3,499.99 (128GB) | Check selected variant |
| NVIDIA DGX Spark | GB10 appliance | 128GB; 64GB via OEMs from Oct 23 | 273 GB/s | 128GB: check store; 64GB from $4,999 | Supported quantized models |
| ASUS Ascent GX10 | GB10 appliance | 128GB | 273 GB/s | $5,999 (1TB); out of stock | Supported quantized models |
| HP Z2 Tower G1i | GPU workstation | Up to 96GB VRAM | 1792 GB/s on top GPU | Quote | Depends on installed GPU |
| Jetson AGX Thor | Edge | 128GB | 273 GB/s | $5,499 MSRP | Embedded model recipes |
| Jetson Orin Nano Super | Edge | 8GB | 102 GB/s | $399 MSRP | Small models / short context |
How to read it
A base tower and one fitted with a 96GB workstation GPU are different purchases. Likewise, a developer kit’s power specification does not include every peripheral or establish wall power. Read the selected configuration and deployment requirements together.
Machines to compare
These are complete system families, with the 512GB Mac Studio identified as due in late October. Hardware details link to their primary sources on each card. Model fit is a planning assessment; S5 Labs has not benchmarked these systems. Illustrations are stylized.
Unified-memory desktops
CPU and GPU share a memory pool. The amount available to inference depends on the operating system, firmware and other applications. AMD’s newer Ryzen AI Max+ PRO 495 systems raise the x86 ceiling from 128GB to 192GB, at a much higher price.
Available; 512GB configuration due late October
M5 Max from $2,499; M5 Ultra from $5,499 (US) Apple Mac Studio (M5 generation)
Memory M5 Max: 36, 48, 64 or 128GB; M5 Ultra: 96, 256 or 512GB unified; options depend on the chip configuration
Bandwidth 460 or 614 GB/s (M5 Max); 1.2 TB/s (M5 Ultra)
GPU Integrated Apple GPU: 32/40 cores (Max), 64/80 cores (Ultra)
NPU 16-core Neural Engine (Max); 32-core (Ultra)
Storage Up to 8TB (Max) or 16TB (Ultra) SSD
Networking 10GbE, Thunderbolt 5 (four rear ports; two more on the front with M5 Ultra) and Wi-Fi 7
Use case: A candidate for MLX-based inference; actual model compatibility and latency need validation on the delivered hardware.
Constraint: There are no S5 Labs measurements of the new generation. Memory cannot be upgraded after purchase.
Where the numbers come from: Apple specifications and September 22 availability announcement. The 512GB configuration is due in late October; no S5 Labs performance measurements.
Sources: Apple — September 22 Mac Studio availability; Apple — Mac Studio specifications
Strengths
- Large unified-memory options
- Integrated Apple-silicon software path
Trade-offs
- 512GB configuration not due until late October
- No internal discrete GPU upgrade
Best for
Readers considering an Apple-silicon desktop who can validate their workload and check delivery for their chosen configuration.
A configurable x86 desktop
DIY system selection: $1,269 / $1,959 / $3,449 (32 / 64 / 128GB); 64GB marked out of stock October 6 Framework Desktop
Memory 32, 64 or 128GB non-upgradeable LPDDR5x
Bandwidth 256 GB/s theoretical at LPDDR5x-8000 / 256-bit
GPU Ryzen AI Max 385 or Max+ 395 integrated Radeon graphics
NPU XDNA 2; up to 50 TOPS
Storage Two M.2 2280 slots; SSD selection costs extra
Networking 5GbE and Wi-Fi 7
Use case: Quantized local models that fit the configured GPU allocation, with more room on the 128GB option.
Constraint: Long context and dense-model decoding can require compromises; no universal token rate follows from the memory capacity.
Where the numbers come from: Framework configurator and AMD platform specifications. Listed DIY prices exclude separately selected storage, OS and accessories.
Sources: Framework — Desktop specifications; Framework — Desktop 400 edition (192GB) configurator; AMD — Ryzen AI Max+ 395 specifications
Strengths
- Serviceable storage and standard desktop parts
- Choice of memory configurations
Trade-offs
- Memory is soldered
- Complete build costs exceed the system selection
- 192GB needs the separate 400 edition, a $6,799 pre-order shipping in November
Best for
Readers who want an x86 desktop and are willing to configure their inference environment.
The managed mini workstation
Configuration-specific quote; no current price verified HP Z2 Mini G1a
Memory Up to 128GB unified; HP specifies up to 96GB assignable to the GPU
Bandwidth Up to 8000 MT/s; roughly 256 GB/s theoretical on a 256-bit bus
GPU Up to Ryzen AI Max+ PRO 395 integrated Radeon graphics
NPU Up to 50 TOPS
Storage Dual NVMe; up to 8TB, RAID options
Networking 2.5GbE listed; confirm optional NICs in the quote
Use case: Quantized models within the allocated GPU memory, with capacity reserved for other workstation applications.
Constraint: The full 128GB is not advertised as dedicated VRAM. Check the runtime and driver combination before deploying.
Where the numbers come from: HP product specifications; bandwidth is a bus-rate calculation, not a throughput measurement.
Sources: HP — Z2 Mini G1a specifications
Strengths
- Business workstation purchasing and support options
- Compact unified-memory configuration
Trade-offs
- GPU allocation is below total RAM
- Exact options depend on the SKU
Best for
Teams that value workstation support and compact deployment.
A retail mini PC with listed 128GB prices
US store, October 6: $2,199.99 (64GB / 1TB); $3,499.99 (128GB / 1TB); $3,649.99 (128GB / 2TB) GMKtec EVO-X2
Memory 64 or 128GB onboard LPDDR5x-8000; not upgradeable
Bandwidth 256 GB/s theoretical from the memory rate and bus width
GPU Ryzen AI Max+ 395 with Radeon 8060S
NPU XDNA 2; 50 TOPS
Storage Two M.2 2280 slots; verify the selected SSD capacity
Networking 2.5GbE and Wi-Fi 7
Use case: An x86 option for quantized inference within the available GPU memory.
Constraint: The 128GB variant costs $1,300 more than the 64GB one. Store prices change often, and regional listings can differ.
Where the numbers come from: GMKtec’s US product page, checked October 6. Its storefront and specification table list different SSD choices; confirm the selected SKU.
Sources: GMKtec — EVO-X2 configuration and specifications
Strengths
- 128GB option with a listed price
- Replaceable NVMe storage
Trade-offs
- RAM is fixed
- Variant prices and stock need checking
Best for
Readers comparing retail minis on delivered configuration and warranty.
NVIDIA GB10 appliances
Compact Arm systems with NVIDIA GB10. Alongside the 128GB systems, NVIDIA announced a 64GB DGX Spark on October 2, sold only through Acer, ASUS, Dell, Gigabyte, HP and MSI from October 23, starting at $4,999. Check the exact SKU, memory and Arm software support.
CUDA development on GB10
128GB: NVIDIA set the Founders Edition MSRP at $4,699 in February; current checkout price not verified. 64GB OEM systems from $4,999 (due October 23) NVIDIA DGX Spark
Memory 128GB coherent unified LPDDR5x
Bandwidth 273 GB/s
GPU GB10 Grace Blackwell; up to 1 PFLOP FP4 vendor peak
NPU LLM acceleration uses the Blackwell GPU
Storage Up to 4TB self-encrypting NVMe, depending on the system
Networking 10GbE, Wi-Fi 7 and ConnectX-7 at 200Gbps
Use case: NVIDIA-supported local inference and development recipes, subject to model format and runtime memory requirements.
Constraint: Dense decoding and inter-node communication can limit latency. Check Arm software compatibility.
Where the numbers come from: NVIDIA-published specifications. NVIDIA advertises up to 200B parameters on one 128GB unit, up to 100B on one 64GB unit and up to 700B on four 128GB units; these are workload-dependent capacity claims.
Sources: NVIDIA — DGX Spark specifications; NVIDIA — ConnectX-7 clustering guide; NVIDIA — DGX Spark 64GB announcement (October 2); NVIDIA — February 2026 DGX Spark price change
Strengths
- NVIDIA software and deployment documentation
- Documented multi-system networking
Trade-offs
- 128GB shared with CPU and OS
- Cluster setup and supported model recipes required
Best for
Developers who need the NVIDIA software environment in a compact system.
An ASUS implementation of GB10
US store, October 6: $5,999 with 1TB, out of stock; 2TB and 4TB add $1,000 and $2,000 ASUS Ascent GX10
Memory 128GB unified LPDDR5x
Bandwidth 273 GB/s
GPU Integrated NVIDIA GB10 Blackwell GPU
NPU LLM acceleration uses the Blackwell GPU
Storage 1TB standard; 2TB or 4TB options on the US store; single slot
Networking 10GbE, Wi-Fi 7 and ConnectX-7
Use case: GB10-compatible workloads on the Ubuntu-based NVIDIA software stack.
Constraint: Do not assume a lower price than DGX Spark or identical support terms. Verify the selected system and cluster support.
Where the numbers come from: ASUS specifications, support FAQ and the US store listing checked October 6; no per-model speed claim.
Sources: ASUS — GX10 specifications; ASUS — GX10 support FAQ; ASUS — GX10 US store
Strengths
- 128GB coherent memory
- ConnectX-7 networking
Trade-offs
- US store listing out of stock when checked
- Arm-compatible software required
Best for
Buyers comparing GB10 vendors on actual price, storage and support.
Discrete-GPU workstations
A discrete GPU supplies its own VRAM. System RAM is separate; offloading can extend capacity but changes performance.
Specify the GPU before comparing
Configuration-specific quote; no current price verified HP Z2 Tower G1i
Memory Up to 256GB non-ECC system RAM (128GB with ECC); optional RTX PRO 6000 Blackwell has 96GB VRAM
Bandwidth 1792 GB/s on RTX PRO 6000 Blackwell Workstation Edition
GPU Up to RTX PRO 6000 Blackwell (96GB); HP also lists RTX PRO 5000 (48GB), 4500 (32GB), 4000 (24GB) and 2000 (16GB)
NPU CPU-dependent; separate from the discrete GPU
Storage Up to 36TB across NVMe and HDD options
Networking 1GbE listed; confirm any added NIC in the quote
Use case: Quantized models that fit the selected discrete GPU, with dedicated high-bandwidth VRAM.
Constraint: CPU RAM is not interchangeable with VRAM. Offloading changes the performance profile.
Where the numbers come from: HP chassis specifications and QuickSpecs, plus NVIDIA GPU specifications. HP’s QuickSpecs exclude the 96GB cards and the RTX PRO 5000 from the 500W and 700W power supplies. No bandwidth-to-token-rate conversion is claimed.
Sources: HP — Z2 Tower G1i specifications; HP — Z2 Tower G1i QuickSpecs; NVIDIA — RTX PRO 6000 Blackwell specifications
Strengths
- Replaceable graphics and expandable storage
- Dedicated GPU memory
Trade-offs
- Top configuration requires an exact quote
- GPU, cooling and power supply must match
Best for
Teams needing a conventional, expandable CUDA workstation.
Edge and embedded systems
Developer kits for robotics, cameras and embedded inference. Choose these for device integration, not as interchangeable desktop chat computers.
Embedded inference with sensor I/O
$5,499 NVIDIA MSRP; partner prices and stock vary NVIDIA Jetson AGX Thor Developer Kit
Memory 128GB 256-bit LPDDR5X
Bandwidth 273 GB/s
GPU Blackwell; vendor peak 2070 sparse FP4 TFLOPS
NPU GPU acceleration plus dedicated vision hardware
Storage Developer kit includes 1TB NVMe
Networking 5GbE RJ45 plus QSFP28 carrying 4×25GbE
Use case: Embedded language and vision workloads with a supported JetPack model recipe.
Constraint: Application processes and sensor buffers share memory. Equal bandwidth to GB10 does not establish equal LLM performance.
Where the numbers come from: NVIDIA kit specifications and MSRP list. The kit uses the T5000 module; the separate T4000 module has 64GB. The 40–130W range is a platform power specification, not a measured whole-system wall-power figure.
Sources: NVIDIA — Jetson Thor specifications; NVIDIA — Jetson FAQ and MSRP list
Strengths
- Large memory for embedded applications
- Sensor and robotics interfaces
Trade-offs
- Integration work required
- Different deployment environment from a desktop
Best for
Robotics, inspection and multimodal edge applications.
A small-model development kit
$399 NVIDIA MSRP; partner prices and stock vary NVIDIA Jetson Orin Nano Super Developer Kit
Memory 8GB LPDDR5 shared memory
Bandwidth 102 GB/s
GPU Ampere; vendor peak 67 INT8 TOPS
NPU No separate NPU
Storage microSD or compatible NVMe storage
Networking Gigabit Ethernet
Use case: Small quantized language and vision models with modest context and application memory needs.
Constraint: An 8B model is not an automatic fit in 8GB. Runtime, context and other processes can exhaust the remaining memory.
Where the numbers come from: NVIDIA-published specifications. No per-model token rate or guaranteed maximum parameter count.
Sources: NVIDIA — Orin Nano Super specifications; NVIDIA — Jetson FAQ and MSRP list
Strengths
- Compact Jetson development platform
- Suitable for small-model prototypes
Trade-offs
- Only 8GB shared memory
- Storage and peripherals add to the $399 MSRP
Best for
Learning the Jetson stack and prototyping compact edge applications.
The six models to test first
Use these models as a repeatable evaluation set for general assistance, vision, reasoning, coding and agent work. They are practical examples from the open models guide, rather than a claim that every model is the newest or best. Record the exact download and runtime version so a later comparison measures the same thing.
Qwen3.8-27B
A dense model with vision support to test against your own writing, reasoning and document tasks.
27B language model · Apache 2.0 · Unsloth UD-Q4_K_M: 16.46GB; vision projector adds 0.93GB
Fit: 32GB total is a candidate for modest context if the GPU allocation allows it; 64GB leaves more room.
Muse Glimmer 30B
A local reasoning and tool-use model with an optional perception encoder. Its model card specifies a compatible llama.cpp build and chat-template requirements.
30B · Apache 2.0 plus usage policy · official Q4_K_M: 16.76GB; vision encoder adds 1.40GB
Fit: Evaluate on a 32GB or larger system with adequate accelerator allocation; optional speculative decoding consumes additional memory.
gpt-oss-20b
A compact MoE example for reasoning and tool-use evaluation with a runtime that supports its format.
21B total / 3.6B active · Apache 2.0 plus usage policy · native MXFP4 weights about 13.8GB
Fit: OpenAI describes a 16GB memory deployment. That is not a guarantee for every runtime or long context; allow more headroom.
Qwen3-Coder-Next
Use repository tasks to test whether this larger coding MoE improves on a smaller model enough to justify the memory cost.
80B total / 3B active · Apache 2.0 · official Q4_K_M shards total 48.41GB (45.09GiB)
Fit: 64GB unified memory is conditional on allocation and context; 96–128GB gives more room. A 48GB GPU leaves little runtime headroom.
Llama 4 Scout
A larger MoE example for evaluating memory headroom. Its active-parameter count does not describe the storage required for all experts.
109B total / 17B active · custom Llama 4 license · ideal 4-bit weight floor: 54.5GB, before format overhead
Fit: Evaluate an actual quantized artifact on a 96–128GB system. Do not budget from the ideal floor or assume the advertised maximum context fits.
Nemotron 3.5 Lightning
NVIDIA’s hybrid Mamba-2, attention and MoE model tests whether a small active-parameter count pays off on your runtime. Its model card has a local AI section for RTX 5090, DGX Spark and RTX 6000 Pro.
30B total / 3B active · OpenMDW-1.1 · ggml-org Q4_0 GGUF (a community conversion, not an NVIDIA file): 18.90GB; optional MTP draft adds 1.16GB
Fit: Evaluate on a 32GB or larger system at modest context. The 1M-token nominal context is not part of this estimate.
Model-to-memory fit
“Candidate” means worth testing for one user at a modest context, such as 4K tokens; it does not certify a successful load. “Conditional” needs particular care with GPU allocation and runtime overhead. Sizes use decimal GB unless GiB is stated. Scout’s ideal floor is calculated as 109 billion parameters × 4 bits ÷ 8, and is not a download size.
| Model | Weight size / estimate | 8GB edge | 32GB / 64GB total | 96GB box | 128GB box |
|---|---|---|---|---|---|
| Qwen3.8-27B | 16.46GB + 0.93GB vision | No | 32: conditional / 64: candidate | Candidate | Candidate |
| Muse Glimmer 30B | 16.76GB + 1.40GB vision | No | 32: conditional / 64: candidate | Candidate | Candidate |
| gpt-oss-20b | ~13.8GB MXFP4 | No | Candidate with supported runtime | Candidate | Candidate |
| Qwen3-Coder-Next | 48.41GB Q4_K_M | No | 32: no / 64: conditional | Candidate | Candidate |
| Llama 4 Scout | 54.5GB ideal 4-bit floor; artifact larger | No | Do not plan from this floor | Artifact-dependent | Artifact-dependent |
| Nemotron 3.5 Lightning | 18.90GB Q4_0 + 1.16GB MTP | No | 32: conditional / 64: candidate | Candidate | Candidate |
Fit is not the same as usable
The table is a rough capacity screen, not a runtime guarantee. Download sizes exclude the context cache, image encoders unless stated, runtime buffers and other applications. A shared-memory system may allocate much less than its total RAM to the GPU. Start with a modest context and measure memory before increasing it; the model’s advertised maximum context is not included in these estimates.
Can you connect them together?
Several machines can either share one model through a distributed runtime or serve separate models behind remote endpoints. These are different architectures, with different networking and software requirements.
One model across nodes
NVIDIA documents multi-Spark clusters over ConnectX-7: up to three systems cabled directly, or four through a switch. Its product page advertises models up to 700B parameters on four 128GB systems. Model format, topology and serving software still determine what runs.
General-purpose sharding
vLLM supports tensor and pipeline parallelism across nodes with supported accelerators. This can run a model larger than one node’s memory. Apple describes clustering the M5 Mac Studio over Thunderbolt 5 with RDMA. Its “up to 3x” figure comes from one Apple test: summarizing a 32K-token prompt with a 72B dense model in mlx-lm, four systems against one. Neither approach creates a single transparent RAM pool.
Remote model access
LM Link lets a client use a model running on another machine. The remote host does the inference; this does not distribute that model’s weights across the client computers.
Choose the architecture
If one model exceeds a node’s memory, investigate supported sharding and the required interconnect. If several people need access to a model that already fits, a secured remote endpoint may be enough. Neither design combines machines into transparent, interchangeable system RAM.
Software stacks and compatibility
Check the model architecture, file format, operating system and accelerator together. Desktop launchers simplify installation, but they do not remove backend compatibility limits.
Apple · MLX / Metal
Best for: Apple-silicon inference
MLX targets Apple silicon and unified memory. LM Studio supports MLX models and llama.cpp-compatible formats on supported Macs. Since v0.40.0 (September 25), Ollama automatically uses MLX for supported architectures on Apple silicon.
Check model architecture support; a new chip announcement is not a runtime compatibility test.
NVIDIA · CUDA
Best for: GPU development and serving
DGX Spark ships NVIDIA’s software stack; GX10 lists Ubuntu Linux. Use containers and packages built for the system’s CPU architecture.
A package for an x86 RTX workstation may not run unchanged on Arm GB10.
NVIDIA · JetPack
Best for: Embedded deployment
NVIDIA’s current JetPack 7 documentation covers Thor and Orin. Orin support arrived with JetPack 7.2 in June, so earlier JetPack 7 guides may cover Thor only. Follow the release and carrier-board compatibility instructions for the exact device.
Keep sensor support and deployment libraries aligned with the Jetson Linux release.
AMD · ROCm / Vulkan
Best for: Ryzen AI Max inference
AMD documents Ryzen AI Max inference with ROCm and Ollama. llama.cpp also offers Vulkan; backend and model support must be checked separately.
Record BIOS GPU allocation, OS, driver and runtime versions when comparing results.
Cross-platform shortcut
LM Studio and Ollama provide local inference tools on supported desktop platforms. LM Studio’s LM Link, still in preview, adds encrypted remote access to models hosted on another computer. Verify that the chosen model stays local if privacy is a requirement; tool calls and cloud fallbacks can still leave the device.
Methodology: where these numbers come from
We checked the linked manufacturer pages and developer documentation on September 5, 2026. On September 28, we updated Mac Studio availability against Apple’s September 22 announcement; other systems were not fully re-reviewed. On October 5, we corrected the review-date wording. On October 6, we rechecked prices, stock and specifications for every system against manufacturer pages, stores and NVIDIA’s Jetson MSRP list. We also added NVIDIA’s October 2 announcement of the 64GB DGX Spark. We could not confirm a current NVIDIA Marketplace checkout price for the 128GB DGX Spark, so the guide does not give one. The review corrects obsolete hardware options and removes unsourced token rates. It distinguishes published specifications, observed storefront prices and our workload-planning guidance.
Specifications
Specifications come from the linked manufacturers. The 256 GB/s figure for LPDDR5x-8000 on a 256-bit bus is a theoretical rate, not measured sustained bandwidth. Peak compute figures use the precision stated by the vendor.
Prices and model fit
Storefront prices apply to the stated selection, not every variant. Model sizes describe the named weight artifact; runtime allocations and the context cache are additional. Fit assessments are estimates to narrow an evaluation, not measured results.
House benchmarks
S5 Labs has not performed a controlled cross-system benchmark for this guide. We do not infer tokens per second from bandwidth or repeat community rates without a reproducible test configuration.
A test you can repeat
Record the checkpoint and quantization, runtime version, prompt and output lengths, context limit, batch size and accelerator allocation. Measure time to first token, decode rate and peak memory. Include a longer prompt and concurrent requests if those reflect the intended use; a short single-user chat test will not answer those questions.
Frequently asked questions
How much memory do I need for a local LLM?
Start with the exact quantized checkpoint size, then add runtime and context memory. Many compact models fit below 32GB, while larger quantized models need substantially more. Total system RAM and memory available to the GPU can differ.
Can these machines run a 70B model?
Some configurations have enough memory for quantized 70B weights, but that does not guarantee your target context length or latency. Test the exact model, quantization and runtime; bandwidth alone cannot provide a token rate.
Can I buy a Mac Studio with more than 96GB?
The M5 generation restores larger options: M5 Max reaches 128GB and M5 Ultra reaches 512GB. Apple announced availability on September 22, with the 512GB version due in late October. Check the delivery estimate for your exact configuration.
Do two computers pool their memory?
They remain separate systems. Distributed runtimes can shard one model across them, while remote-serving tools send requests to a model on one host. Both require compatible software; an Ethernet cable alone does not combine RAM.
Does the NPU TOPS number predict LLM speed?
Only if the model actually uses that NPU through a supported runtime, and even then the precision and workload matter. GPU-based inference tools should be compared using their own measured latency, memory use and throughput.
Is local inference private and cheaper?
Local inference can keep prompts on your hardware, but cloud fallbacks, tools, telemetry and remote access must be configured accordingly. Compare hardware, electricity and maintenance with your actual API usage; there is no universal payback period.
Should I start with a desktop or a Jetson?
Use a desktop for general development unless the workload needs embedded interfaces, sensor integration or a device power envelope. Jetson is useful for those deployment constraints, but it requires its own software and hardware integration work.
Sources
Sources include manufacturer specifications and stores, official model cards, and tooling documentation. The page update date does not mean every configuration was rechecked. See methodology for how performance claims are attributed.
Apple — September 22 Mac Studio availability https://www.apple.com/newsroom/2026/09/the-new-mac-mini-and-mac-studio-are-available-today/ Apple — Mac Studio specifications https://www.apple.com/mac-studio/specs/ Framework — Desktop DIY configurator https://frame.work/products/desktop-diy-amd-aimax300 Framework — Desktop 400 edition (192GB) configurator https://frame.work/products/desktop-diy-amd-aimax400/configuration/new AMD — Ryzen AI Max+ 395 specifications https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html HP — Z2 Mini G1a specifications https://www.hp.com/us-en/workstations/z2-mini-a.html GMKtec — EVO-X2 configuration and specifications https://www.gmktec.com/products/amd-ryzen%E2%84%A2-ai-max-395-evo-x2-ai-mini-pc NVIDIA — DGX Spark specifications https://www.nvidia.com/en-us/products/workstations/dgx-spark/ NVIDIA — ConnectX-7 clustering guide https://docs.nvidia.com/dgx/dgx-spark/spark-clustering.html NVIDIA — DGX Spark 64GB announcement (October 2) https://blogs.nvidia.com/blog/local-ai-dgx-spark-64gb-sync/ NVIDIA — February 2026 DGX Spark price change https://forums.developer.nvidia.com/t/2-23-2026-price-change-announcement/361713 ASUS — GX10 specifications https://www.asus.com/us/networking-iot-servers/desktop-ai-supercomputer/ultra-small-ai-supercomputers/asus-ascent-gx10/techspec/ ASUS — GX10 support FAQ https://rog.asus.com/in/support/faq/1056142/ ASUS — GX10 US store https://eshop.asus.com/us/ascent-gx10.html HP — Z2 Tower G1i specifications https://www.hp.com/us-en/workstations/z2-tower.html HP — Z2 Tower G1i QuickSpecs https://h20195.www2.hp.com/v2/GetDocument.aspx?docname=c09133728 NVIDIA — RTX PRO 6000 Blackwell specifications https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000/ NVIDIA — Jetson Thor specifications https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-thor/ NVIDIA — Orin Nano Super specifications https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/nano-super-developer-kit/ NVIDIA — Jetson FAQ and MSRP list https://developer.nvidia.com/embedded/faq NVIDIA — JetPack 7.2 adds Orin support https://developer.nvidia.com/blog/deploy-agentic-ready-ai-at-the-edge-with-memory-efficiency-in-nvidia-jetpack-7-2 NVIDIA — JetPack documentation https://developer.nvidia.com/embedded/jetpack Apple — MLX https://opensource.apple.com/projects/mlx/ Apple — Mac Studio (Thunderbolt 5 RDMA clustering footnote) https://www.apple.com/mac-studio/ Apple — Mac mini specifications https://www.apple.com/mac-mini/specs/ Ollama — v0.40.0 release notes (MLX on Apple silicon) https://github.com/ollama/ollama/releases/tag/v0.40.0 GMKtec — EVO-X5 Pro (192GB) https://www.gmktec.com/products/gmktec-evo-x5-pro-amd-ryzen-ai-max-pro-495-ai-mini-pc Hugging Face — Nemotron 3.5 Lightning model card https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 Hugging Face — Nemotron 3.5 Lightning GGUF (ggml-org) https://huggingface.co/ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF LM Studio — system requirements https://lmstudio.ai/docs/app/system-requirements LM Studio — LM Link https://lmstudio.ai/link AMD — Ryzen AI Max inference guide https://rocm.blogs.amd.com/artificial-intelligence/ryzen-uma-llm/README.html Ollama — context memory requirements https://docs.ollama.com/context-length vLLM — parallelism and scaling https://docs.vllm.ai/en/latest/serving/parallelism_scaling/