InfrastructureCompute, chips & cloud
NVIDIA Extends DGX Spark Clustering to Eight Nodes, Running the 753B-Parameter GLM 5.3 Locally
Eight linked DGX Spark desktops pool 1 TB of memory, enough to run Z.ai's full GLM 5.3 at about 31 tokens/s for one user. A new one-shot all-reduce backend, RoCEnante, improves multi-node decode scaling on switched networks.
From the NVIDIA Generative AI article of October 22, 2026 and cited sources: Eight 128 GB Spark nodes give 1 TB of memory, enough for the roughly 470 GB NVFP4 build of GLM 5.3. [1] [5]
SOURCES[1] NVIDIA Technical Blog: Running Frontier-Class AI Models… developer.nvidia.com[5] Gadget Pilipinas: NVIDIA Local AI Push at GTC Berlin gadgetpilipinas.net
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- Eight 128 GB Spark nodes give 1 TB of memory, enough for the roughly 470 GB NVFP4 build of GLM 5.3. [1] [5]
- Original source
- NVIDIA Generative AI article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- October 22, 2026 article published
- Covered
- Oct 23 edition
- Evidence
- 9 sources across 7 sites
- 1NVIDIA Technical Blog: Running Frontier-Class AI Models…developer.nvidia.com
- 2MONOist: DGX Spark 4台連結可能にmonoist.itmedia.co.jp
- 3byteiota: DGX Spark June 2026, four nodesbyteiota.com
- 4Fanáticos del Hardware: NVIDIA amplía DGX Spark a ocho…fanaticosdelhardware.com
- 5Gadget Pilipinas: NVIDIA Local AI Push at GTC Berlingadgetpilipinas.net
- 6GitHub: spark-nanochat-multinodegithub.com
Reach: not measured yet
Eight 128 GB Spark nodes give 1 TB of memory, enough for the roughly 470 GB NVFP4 build of GLM 5.3. Decode runs at about 31 tok/s with one request. At eight concurrent requests it drops to about 18 tok/s per user, while total throughput rises to 67 tok/s. Today, eight-node clusters have to be set up by hand. Reports disagree on when NVIDIA Sync's Cluster Assistant will automate it: one says October, NVIDIA says by year end. [1] [5]
RoCEnante uses a single GPU kernel. Every rank sends its data to all the others, and each rank sums the results itself. That means fewer communication steps, which pays off for small messages, few nodes and low concurrency. With it, Qwen3.8-27B decode rises from 11.86 to 52.58 tok/s across one to eight nodes (4.43x), versus 3.05x without it. Prefill scales only 1.94x. RoCEnante needs a switched network and falls back to NCCL if its startup check fails. NCCL's ring approach wins at larger scale. [1]
Switchyard sends some requests to a smaller model and the rest to a larger one. On a 92-case tool-use benchmark it scored 91, the same as Qwen alone. Total throughput rose about 1.5x when half of requests went to Qwen and about 3x when a quarter did. Switchyard's library is in beta, and its standalone server is demo-only. [1] [7]
700B-class open models now fit and run on a desk-side cluster at modest per-user speed. Implication: private on-premises prototyping becomes practical. Network design and routing between small and large models may matter more than adding nodes.
Executive brief
NVIDIA now officially supports linking eight desk-sized DGX Spark computers into one cluster. Together they have 1 TB of memory, enough to run Z.ai's full GLM 5.3 (753B parameters) locally. On that model, NVIDIA measured 31 tokens/s for a single user. That is usable but not fast. The more useful finding is about scaling. Decode speed rises from about 12 to 52 tokens/s across eight nodes, but only with a new collective backend on a switched network. Training a small model scales almost linearly. All of these numbers come from NVIDIA (NVIDIA blog).
What changed and event timeline
Four-node Spark clusters announced
At GTC 2026, NVIDIA said up to four DGX Spark units could be linked through RoCE switches ().
Automated four-node clustering ships
A software update added Cluster Assistant for 512 GB clusters. Coverage said four nodes need a managed 200 Gbps-class QSFP switch ().
GTC Berlin raises the limit to eight nodes
NVIDIA also claimed DGX Spark inference gains of up to 30% on Qwen3.8 27B from speculative decoding and multi-token prediction (MTP) work done with vLLM and llama.cpp ().
NVIDIA publishes eight-node benchmarks
The post covers inference scaling, the RoCEnante backend, Switchyard routing and nanochat training ().
64 GB Spark ships, Sync timing unclear
The 64 GB model shipped. One report says the NVIDIA Sync update with eight-system support arrives in October. NVIDIA's blog says "by the end of the year" ().
Capabilities and access
- Memory: 128 GB of unified memory per node, so 1 TB across eight. Eight-node clusters can be set up by hand today.
- Models tested: nvidia/GLM-5.3-NVFP4 (about 470 GB), GLM 5.3 Flash (about 80 tok/s), Qwen3.8-27B-NVFP4, MiniCPM5 2B and Qwen3.8 Flash Next (NVIDIA blog).
- Switchyard is Apache 2.0; its library is in beta and its standalone server is demo-only.
Read the full section
- Memory: 128 GB of unified memory per node, so 1 TB across eight. Eight-node clusters can be set up by hand today. Support in NVIDIA Sync's Cluster Assistant is still pending (NVIDIA blog).
- Models tested: nvidia/GLM-5.3-NVFP4 (about 470 GB), GLM 5.3 Flash (about 80 tok/s), Qwen3.8-27B-NVFP4, MiniCPM5 2B and Qwen3.8 Flash Next (NVIDIA blog).
- Code: the nanochat multi-node script, spark-vllm-docker and Switchyard. Switchyard is Apache 2.0; its library is in beta and its standalone server is demo-only.
Technical analysis for researchers and developers
- RoCEnante: a single GPU kernel sends each rank's data to every other rank, then each rank sums the inputs itself (a one-shot all-reduce).
- Benchmark method: llama-benchy, 8,192 input tokens, 256 output tokens, one request at a time, speculative decoding off.
- Network: each ConnectX-7 port carries about 200 Gbps, but the OS sees two roughly 100 Gbps sub-interfaces.
Read the full section
- RoCEnante: a single GPU kernel sends each rank's data to every other rank, then each rank sums the inputs itself (a one-shot all-reduce). This cuts the number of communication steps, which helps with small messages, few nodes and low concurrency. It moves more data, so NCCL's ring all-reduce wins as messages, node counts or concurrency grow. RoCEnante works only on switched networks. It is merged into FlashInfer and falls back to NCCL if its startup test fails. The original one-shot idea came from a community contributor (Jason Cook/el8) (NVIDIA blog).
- Benchmark method: llama-benchy, 8,192 input tokens, 256 output tokens, one request at a time, speculative decoding off.
- Network: each ConnectX-7 port carries about 200 Gbps, but the OS sees two roughly 100 Gbps sub-interfaces. NCCL has to use both to reach full speed (NVIDIA forums).
Claims and evidence
All of these figures are NVIDIA's own. No independent reproduction of the eight-node results has been published. The nanochat repo includes the code but does not publish results (GitHub).
Read the full section
- Qwen3.8-27B decode, one to eight nodes: 11.86 to 52.58 tok/s (4.43x) with RoCEnante, versus 3.05x without it. Prefill scales only 1.94x (NVIDIA blog).
- Full GLM 5.3: decode falls from 31 tok/s per user at one request to 18 at eight concurrent requests, with total throughput rising to 67 tok/s. Prefill stays at about 1,600 tok/s total.
- Switchyard routing: scored 91/100 on a 92-case tool-use benchmark, matching Qwen alone. Aggregate throughput is about 1.5x when half of requests go to Qwen, and about 3x when a quarter do.
- nanochat training: 7.79x speedup on eight nodes, 97.4% efficiency.
All of these figures are NVIDIA's own. No independent reproduction of the eight-node results has been published. The nanochat repo includes the code but does not publish results (GitHub).
Context and prior work
Community builders were clustering Sparks before the official support arrived. One forum user reported DeepSeek-V4-Flash on four nodes at about 70 tok/s single-stream, with 83% MTP draft acceptance. The same post identified cross-node MoE all-to-all traffic, not compute, as the main bottleneck (NVIDIA forums). NVIDIA's main cluster tooling, spark-vllm-docker, grew out of a project created by one of the post's authors (NVIDIA blog).
Limitations, safety and contested findings
- Network speed: nodes connect over 200 Gbps Ethernet, not NVLink.
- Ring vs. switch: NVIDIA now presents a switchless four-node ring as a good low-cost option. June coverage described four-node setups as needing a switch.
- Narrow conditions: the scaling numbers are for one request at a time, and a 45% gain refers to one dense model.
Read the full section
- Network speed: nodes connect over 200 Gbps Ethernet, not NVLink. That limits latency-sensitive workloads (byteiota).
- Ring vs. switch: NVIDIA now presents a switchless four-node ring as a good low-cost option. June coverage described four-node setups as needing a switch. Running eight nodes, or splitting them into groups, still requires a switch.
- Narrow conditions: the scaling numbers are for one request at a time, and a 45% gain refers to one dense model.
- Speedup claims vary by platform: NVIDIA cites up to 40% on RTX PRO 6000 but up to 30% on Spark, and says results depend on configuration (Fanáticos del Hardware).
Business and practitioner implications
- An eight-node cluster makes 700B-class open models practical for private, on-premises prototyping. It does not deliver serving-grade per-user speed.
- Buy a switch if you plan to run several model pools or use RoCEnante.
- Routing traffic between a small and a large model, as Switchyard does, may add more throughput than adding nodes.
Read the full section
- An eight-node cluster makes 700B-class open models practical for private, on-premises prototyping. It does not deliver serving-grade per-user speed.
- Buy a switch if you plan to run several model pools or use RoCEnante. A ring is the cheaper choice for one model on four nodes.
- Routing traffic between a small and a large model, as Switchyard does, may add more throughput than adding nodes. It depends on how much traffic the small model can take.
- Near-linear data-parallel scaling makes the cluster a credible place to pre-train small models.
- Wait for the Sync eight-node release before planning turnkey deployments.