NVIDIA Shows How TensorRT LLM VisualGen Speeds Up FLUX 3 Image Klein on Blackwell: Up to 3.46x Text-to-Image and 4.45x Image-to-Image for Base at 2048x2048 on One B200 Against Uncompiled BF16, but the Numbers Are NVIDIA's Own on a Feature Branch and Faster Settings Change the Images

On November 6, 2026, NVIDIA published a technical tutorial on running FLUX 3 Image Klein faster on Blackwell GPUs with TensorRT LLM VisualGen, a runtime for diffusion models inside TensorRT LLM. FLUX 3 Image Klein is an image generation and editing model from Black Forest Labs (BFL). It comes as an undistilled Base checkpoint, which uses 50 denoising steps and a guidance scale of 3.5, and a distilled checkpoint that uses 16 steps and no guidance. The model card lists a 5.6 billion parameter image transformer, with Qwen3-VL-4B-Instruct as a separate text encoder, up to 10 reference images, BF16, FP8 and NVFP4 variants, and the FLUX Kommunity License, which allows non-commercial use and a commercial-output exception for qualifying users. The post says BFL trained the model on NVIDIA GB200 NVL72 systems and optimized it for NVIDIA GPUs, so it is a vendor tutorial for its own hardware. Every timing below is NVIDIA's own.

The post first breaks down where the time goes. At 2048 by 2048 pixels on one NVIDIA B200 GPU, with NVFP4 weights, torch.compile and standard attention, denoising took 14.24 of 14.70 seconds for Base text-to-image, about 96.8%, and 39.40 of 40.29 seconds for Base image-to-image, about 97.8%. The distilled model took 2.71 seconds for text-to-image, with denoising at 84.0%. NVFP4 weights alone were up to 1.62 times faster for text-to-image and 1.58 times faster for image-to-image than BF16.

It then tests several optimizations, each with its own baseline. Tuned PrimsTS matrix-multiplication kernels were 1.02 to 1.10 times faster than the autotuned TensorRT LLM NVFP4 version for text-to-image. Attention made up about 60% of transformer kernel time, so the authors tried quantized and sparse variants. Compared with standard BF16 attention, cuDNN FP8 gave 1.08 times for Base and 1.09 times for distilled, and VC-Attention FP8 with a repair budget of 0.005 gave 1.18 and 1.16 times. Sol-Attention with its sparsity parameter tau at 1.0 gave 1.24 times faster denoising plus decoding for Base with 16 initial dense steps, or 1.30 times with 12. Cache-DiT, which reuses intermediate outputs across steps when a residual-difference threshold of 0.24 allows, gave 1.97 times for Base and 1.26 times for distilled at 1024 by 1024, and 1.94 and 1.20 times at 2048 by 2048.

Image quality is judged with LPIPS, a perceptual distance, computed against the unoptimized output for the same seed, where lower means closer. Values quoted in the post include 0.033 for Cache-DiT on Base, 0.086 for VC-Attention FP8 on Base, and 0.151 to 0.161 for some distilled or 12-dense-step settings, and the post says a higher tau reduced quality with little extra speed. So the speedups do not come for free in pixels. The authors suggest 16 dense steps for fidelity or 12 for speed, and show example outputs that readers should compare for themselves.

The headline results combine steps on one B200 at 2048 by 2048, compared with uncompiled BF16 for Base: compilation, NVFP4 with PrimsTS kernels, Sol-Attention and Cache-DiT gave 3.46 times for text-to-image, from 22.36 to 6.46 seconds, and 4.45 times for image-to-image, from 54.64 to 12.28 seconds. The distilled model reached up to 2.34 times. On a single RTX PRO 6000 Blackwell Server Edition GPU the gains were 1.73 to 3.25 times. Note the baseline: these are speedups against uncompiled BF16, not against the compiled NVFP4 numbers above, and the individual factors do not simply multiply.

For multiple GPUs, VisualGen uses classifier-free guidance and Ulysses sequence parallelism. On an NVIDIA DGX B200 with eight GPUs at 2048 by 2048, median text-to-image latency fell from 14.90 to 3.00 seconds for Base and from 2.73 to 1.08 seconds for distilled. Denoising sped up 6.01 and 4.44 times, but roughly half of the distilled eight-GPU time is outside denoising, which limits the end-to-end gain. At 1024 by 1024, CUDA Graphs cut Base latency from 1.823 to 0.809 seconds and distilled from 0.791 to 0.366 seconds.

For deployment, the post gives a container command, an offline script, and an OpenAI-compatible /v1/images/generations endpoint started with trtllm-serve, plus a YAML file for each setting. NVFP4 is chosen by setting the variant to nvfp4. On eight GPUs the Base model uses a cfg_size of 2 and a ulysses_size of 4, and the distilled model uses a cfg_size of 1 and a ulysses_size of 8. The reference setup also loads a safety classifier, and the model card warns that short prompts sent as typed give generic output, which is why it asks users to run the upsampler.

The limits matter. Results cover NVIDIA Blackwell data center GPUs and the RTX PRO 6000 Blackwell Server Edition, not other hardware. Averages come from 10 requests after two warmups with seed 0, and the post gives no run-to-run spread. Reported latencies exclude BFL's prompt upsampler, which the model card says to always run, and which is an 18.8 GB model in the reference setup. The container tag and code links point to a feature branch, feat/flux3-image-klein, and not a stable release. LPIPS is an automated measure, and the example images cover only a few scenes, such as a tea scene, a cat scene, a kitchen still life and a canoe edit, so they cannot show how each setting behaves on every prompt. We did not reproduce any figure.

Our reading: this is a practical, detailed recipe with real configuration files, and it shows where the time goes. People who serve FLUX 3 Klein on Blackwell can try NVFP4, compilation and the attention options and check the LPIPS and the images against their own quality bar. Anyone on other GPUs or on a stable release should wait. Treat the largest numbers as best cases from one vendor's tests, and read each factor against its own baseline.