InfrastructureCompute, chips & cloud
NVIDIA's trtedge tool runs Nemotron-3.5 Lightning with DeepSeek's DSpark decoding on the $4,999 DGX Spark 64GB
NVIDIA's TensorRT Edge-LLM adds a trtedge command-line workflow that downloads, builds and serves a 4-bit 30B mixture-of-experts model on the new 64GB DGX Spark. It uses DeepSeek's DSpark speculative decoding to raise single-user generation speed.
From the NVIDIA Generative AI article of October 30, 2026 and cited sources: A few trtedge commands download Nemotron-3.5 Lightning, compile a TensorRT engine and start either a chat session or a local OpenAI-compatible server. [1] [7]
SOURCES[1] NVIDIA Technical Blog: Run AI Models for Edge Application… developer.nvidia.com[7] TensorRT Edge-LLM Overview nvidia.github.io
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- A few trtedge commands download Nemotron-3.5 Lightning, compile a TensorRT engine and start either a chat session or a local OpenAI-compatible server. [1] [7]
- Original source
- NVIDIA Generative AI article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- October 30, 2026 article published
- Covered
- Nov 2 edition
- Evidence
- 9 sources across 9 sites
- 1NVIDIA Technical Blog: Run AI Models for Edge Application…developer.nvidia.com
- 2Codersera: DeepSeek DSpark Explainedcodersera.com
- 3Hugging Face: NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4huggingface.co
- 4Classmethod: DSpark / DFlash / MTP on DGX Sparkdev.classmethod.jp
- 5Mixed: DGX Spark 64GB at $4,999mixed-news.com
- 6TokenPost: NVIDIA Tests Show DSpark Speeds Nemotron on…tokenpost.com
Reach: not measured yet
A few trtedge commands download Nemotron-3.5 Lightning, compile a TensorRT engine and start either a chat session or a local OpenAI-compatible server. The compiled engine is cached for reuse. The C++ runtime underneath also targets Jetson and DRIVE Thor, so code prototyped on a Spark can move to those embedded devices without changing runtimes. [1] [7]
In NVIDIA's own tests, DSpark raised output speed from 73.4 to 126.9 tokens per second on HumanEval prompts, and from 73.3 to 108.5 on SpeedBench 1K. Time to first token stayed near 98 ms either way. A separate test on the 128GB Spark, using vLLM with thinking turned off, found a similar gain of about 1.5×. [1] [4]
The model has 30B total parameters, of which 3B are active per token. It combines Mamba-2, mixture-of-experts and a few attention layers, and supports context windows up to 1M tokens. The 4-bit NVFP4 checkpoint scores 81.62 on MMLU-Pro, against 81.94 for the BF16 version. [3]
peak memory use was 27.9GB, including the operating system, which is under half the 64GB machine.
Read the full assessment
Implication: developers could run vision, speech or agent tools alongside the model, and test app code against a local OpenAI-style API before deploying on Thor devices.
I'm writing up the dossier now that the research is done.
Executive brief
NVIDIA says a 30-billion-parameter model peaked at 27.9GB of total system memory, OS included, on its new $4,999 DGX Spark 64GB. That is less than half the machine's memory. The model is Nemotron-3.5 Lightning, compressed to 4 bits. With DSpark speculative decoding it generated up to 126.9 tokens per second, according to NVIDIA's tutorial. A new trtedge command-line tool downloads the model, builds the engine and serves it in a few commands. An independent August test on the 128GB Spark, using different software, saw a similar gain from DSpark. No third party has yet reproduced NVIDIA's 64GB figures.
What changed and event timeline
DeepSeek releases DSpark
DSpark is a small draft model that proposes several tokens for the main model to check. It is MIT-licensed and DeepSeek claims 51–400% throughput gains on its V4 models ().
Nemotron-3.5 Lightning ships
NVIDIA released a 30B-total, 3B-active MoE model with an NVFP4 (4-bit) checkpoint under the OpenMDW-1.1 license ().
Independent Spark benchmark
Classmethod compared DSpark, DFlash and MTP on a 128GB DGX Spark using vLLM. DSpark was fastest of the three ().
DGX Spark 64GB launches
It costs $4,999 in the US and is sold only through Acer, ASUS, Dell, Gigabyte, HP and MSI ().
NVIDIA publishes the trtedge workflow
The tutorial covers chat, a local OpenAI-compatible server and DSpark on the 64GB Spark, with benchmark numbers (). repeated the results without testing them.
Capabilities and access
- Model:
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4, about a 21GB download. The DSpark draft model is...-NVFP4-DSpark(NVIDIA). - Software:
tensorrt-edgellm[server]==0.11.1, tested with TensorRT 10.16.1.11. It needs CUDA 13 and Python 3.10–3.12. - Commands:
trtedge chatandtrtedge serve. The server exposes/v1/chat/completionsand/health/ready.
Read the full section
- Model:
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4, about a 21GB download. The DSpark draft model is...-NVFP4-DSpark(NVIDIA). - Software:
tensorrt-edgellm[server]==0.11.1, tested with TensorRT 10.16.1.11. It needs CUDA 13 and Python 3.10–3.12. - Commands:
trtedge chatandtrtedge serve. The server exposes/v1/chat/completionsand/health/ready. - Context window: up to 1M tokens. Reasoning ("thinking") can be switched on or off (model card).
Technical analysis for researchers and developers
- Runtime design: TensorRT Edge-LLM is a C++ runtime with no Python needed at run time. It targets Jetson Thor/Orin, DRIVE Thor and DGX Spark.
- Speculative decoding options: EAGLE3, MTP, DFlash and DSpark.
- Model architecture: interleaved Mamba-2, MoE and a few attention layers (model card).
Read the full section
- Runtime design: TensorRT Edge-LLM is a C++ runtime with no Python needed at run time. It targets Jetson Thor/Orin, DRIVE Thor and DGX Spark. Models reach it two ways: export to ONNX and build TensorRT engines, or the experimental on-device
trtedge build(docs). Orin is limited to FP16, INT8 and INT4. - Speculative decoding options: EAGLE3, MTP, DFlash and DSpark.
- Model architecture: interleaved Mamba-2, MoE and a few attention layers (model card).
- Benchmark setup: batch size 1, one streaming request, a 128-token output cap, thinking on, DGX OS 7.6.0. Short outputs and single-user load favour the speed reported here; throughput with many users is not covered.
- Caching: compiled engines are stored in
~/.cache/tensorrt-edgellmand reused when the configuration matches.
Claims and evidence
The Classmethod test supports the direction of the gain, not NVIDIA's exact numbers. Its software, memory size and settings all differ.
Read the full section
| Claim | Evidence |
| DSpark raises speed from 73.4 to 126.9 tok/s on HumanEval prompts, and from 73.3 to 108.5 on SpeedBench 1K | NVIDIA's own test (NVIDIA) |
| Time to first token is almost unchanged with DSpark (98.1 vs 98.7 ms) | NVIDIA's own test |
| Peak memory is 27.9GB with DSpark | NVIDIA's own test |
| DSpark gives about 1.5× on a Spark | Independent: 81.27 → 124.24 tok/s, vLLM on the 128GB Spark, thinking off (Classmethod) |
| NVFP4 loses almost no accuracy | NVIDIA reports MMLU-Pro of 81.62 vs 81.94 for BF16 (model card) |
The Classmethod test supports the direction of the gain, not NVIDIA's exact numbers. Its software, memory size and settings all differ.
Context and prior work
- DSpark builds on DFlash's approach of drafting blocks of tokens. DeepSeek's DeepSpec framework benchmarks DSpark against DFlash and EAGLE3 under the same conditions (Codersera).
- Before this, NVIDIA's Spark guides pointed developers to the server-class TensorRT-LLM (build.nvidia.com).
- Edge-LLM puts development on Spark and deployment on Thor-class devices onto the same runtime (docs).
Read the full section
- DSpark builds on DFlash's approach of drafting blocks of tokens. DeepSeek's DeepSpec framework benchmarks DSpark against DFlash and EAGLE3 under the same conditions (Codersera).
- Before this, NVIDIA's Spark guides pointed developers to the server-class TensorRT-LLM (build.nvidia.com).
- Edge-LLM puts development on Spark and deployment on Thor-class devices onto the same runtime (docs).
Limitations, safety and contested findings
- All 64GB numbers come from NVIDIA.
- The results come from small workloads: about 139 to 2,247 input tokens and 128 output tokens.
- The baseline speeds differ between tests: 73 tok/s for NVIDIA's Edge-LLM and 81 tok/s for Classmethod's vLLM.
Read the full section
- All 64GB numbers come from NVIDIA. When the 64GB unit launched, nothing had been tested independently, and NVIDIA did not say whether it matches the 128GB model's performance (Mixed).
- The results come from small workloads: about 139 to 2,247 input tokens and 128 output tokens.
- The baseline speeds differ between tests: 73 tok/s for NVIDIA's Edge-LLM and 81 tok/s for Classmethod's vLLM. Because the setups differ, this does not show which runtime is faster.
- Speculative decoding produces exactly the output the main model would have, so it should not change answer quality (Codersera).
Business and practitioner implications
- Memory headroom: a 27.9GB peak leaves room on a 64GB machine for vision or speech models, or for agent tooling alongside the LLM.
- Pricing: the 64GB model starts at $4,999. The 128GB model has risen from $3,999 at launch to $6,950 (Mixed; Tech Insider).
- Prototyping: the OpenAI-compatible endpoint lets existing app code be tested locally before moving to a C++ deployment on Thor.
Read the full section
- Memory headroom: a 27.9GB peak leaves room on a 64GB machine for vision or speech models, or for agent tooling alongside the LLM.
- Pricing: the 64GB model starts at $4,999. The 128GB model has risen from $3,999 at launch to $6,950 (Mixed; Tech Insider).
- Prototyping: the OpenAI-compatible endpoint lets existing app code be tested locally before moving to a C++ deployment on Thor.
- Before committing: benchmark your own workload with more users, longer outputs and the reasoning mode you plan to use. Draft-token acceptance, and so the speedup, depends on the workload.