NVIDIA Walks Through TensorRT Edge-LLM on DGX Spark and Reports DSpark Decoding Rates in Its Own Benchmark

A NVIDIA technical tutorial dated October 30, 2026, walks developers through local prototyping with TensorRT Edge-LLM on a 64 GB DGX Spark. The target is edge application development: models tested on Spark may later be deployed on supported embedded platforms. The example is a text-only Nemotron-3.5 Lightning 30B-A3B NVFP4 model, not every vision, speech or robotics model mentioned as a future possibility. It uses the trtedge command-line tool to download the model, build a TensorRT engine, chat in a terminal and serve a local API. The article includes a vendor-run performance table; that table is evidence of NVIDIA's reported measurements, not an independent test by this publication.

The listed prerequisites are a DGX Spark with a supported software stack, CUDA 13, TensorRT 10.16.1.11 or later within the 10.x series and Python 3.10–3.12. NVIDIA says it tested the steps with TensorRT Edge-LLM 0.11.1 and TensorRT 10.16.1.11. The tutorial creates a Python virtual environment with access to system packages, installs the server extra for the pinned Edge-LLM version, checks the TensorRT version and loads the runtime. This version check matters because an incompatible binding can prevent the example from starting; the post points to TensorRT's installation instructions if needed. The commands are instructions for a reader's machine, and this article does not claim they were run here.

The first trtedge chat command downloads the selected model and builds an engine. NVIDIA estimates the initial model download at about 21 GB and says files and engines are cached under the user's home cache directory by default; later matching launches reuse them. The demonstration asks the model to convert 25°C to Fahrenheit and shows 77°F as a sample response. This is a functional demonstration, not a measurement of accuracy across tasks. The operator can exit the chat and start trtedge serve with an explicit loopback host, 127.0.0.1, and port 8000. A ready health response is checked before posting a streaming request to an OpenAI-compatible chat-completions API. The loopback address keeps the tutorial server local unless the operator deliberately changes its network configuration.

An optional step restarts the server with DSpark speculative decoding. A draft model proposes tokens that the main model verifies; the example configures three speculative tokens. This is a generation-speed optimization, not a claim about electricity generation or a different main model. The documentation allows experimenting with the token count and workload. The post does not say every model or target device supports this exact configuration. Developers should consult the supported-model list and platform-specific requirements before extrapolating from the Spark example to an embedded deployment.

NVIDIA reports median time to first token and mean generation speed in its DGX Spark 64 GB table. For SpeedBench 1K, the table gives 201.0 ms and 73.3 tokens/s without DSpark, versus 203.0 ms and 108.5 tokens/s with it. For SpeedBench 2K it gives 329.5 ms and 72.8 tokens/s versus 331.6 ms and 105.3 tokens/s. For HumanEval prompts it gives 98.1 ms and 73.4 tokens/s versus 98.7 ms and 126.9 tokens/s. The reported generation-speed numbers exclude time to first token. Thus the higher decoding rate in the table did not mean a lower median first-token time in those rows.

The measurement conditions are specific: DGX OS 7.6.0 based on Ubuntu 24.04.4, NVFP4, thinking enabled, batch size one, one concurrent streaming request and a 128-token output limit. Average input lengths were 1,153, 2,247 and 139 tokens for the three workloads respectively. NVIDIA reports a 27.9 GB peak total system-memory use with DSpark, including the operating system. Different models, concurrency, output lengths or edge hardware may change the result. The practical takeaway is a documented path from a local chat to a local application API plus an optional decoding experiment, with performance figures confined to the vendor's stated benchmark setup.