NVIDIA Proposes a Reshard Collective for Reinforcement Learning Weight Refit: Its Own Benchmarks Cut Refit Latency 2.7x to 5.7x, but They Measure Refit Only and Not End-to-End Training, and the NeMo RL Feature Is Experimental and Needs the Megatron Training Backend

On November 6, 2026, NVIDIA engineers published a technical post on the Reshard collective, a communication call meant to speed up one step of reinforcement learning (RL) post-training: copying fresh model weights from the training GPUs to the inference GPUs. The authors call that step refit. The post also introduces NCCL M2N Reshard, an open source reference implementation in the NVIDIA/nccl-extensions repository, and a path for it in the NeMo RL framework. All timings are NVIDIA's own, measured on its own systems.

The problem is size. In a disaggregated RL setup, training and inference run on separate GPUs, and the inference side must receive the updated policy often. The two sides split the model differently, with different data, tensor, expert and pipeline parallelism. The usual method is gather-and-broadcast: the training GPUs gather each full parameter, broadcast it, and each inference GPU keeps only the shard it needs. NVIDIA estimates that this moves about twice the model's weight size. For BF16 weights it gives 5.6 TB for Kimi K3 (2.8 trillion parameters) and 4.8 TB for Qwen 3.8 2.4T A95B, or roughly 11.2 TB and 9.6 TB of logical traffic, which the post says must happen at every training step.

The idea is to describe both layouts as PyTorch DTensors, a device mesh plus placements, and treat refit as resharding one DTensor into another. Reshard then sends only the overlapping pieces between source and destination shards. NCCL M2N Reshard also groups destination replicas by NVLink domain: in the illustrated case with all four destination GPUs in one NVLink domain, each shard crosses the network once and the receivers swap the rest over NVLink; across multiple domains a copy must be forwarded to each domain, with the two steps pipelined. In NeMo RL, NVIDIA applies a hybrid design. Feed-forward projection weights, which it says are around 95% of parameters in recent models, take the reshard path, and smaller weights such as QKV projection or Mamba weights keep gather-and-broadcast so that coverage stays complete.

The reported results are four refit latencies measured in NeMo RL. On NVIDIA DGX H100, Qwen 3 235B falls from 6.4 s to 1.9 s, and DeepSeek v3 671B from 14.9 s to 2.6 s. On NVIDIA DGX GB200, DeepSeek v3 671B falls from 6.1 s to 2.3 s, and Nemotron Ultra v3 from 8.2 s to 2.3 s. The post states a speedup of 2.7x to 5.7x, and the table agrees with that range. A separate refit-only microbenchmark moved the Qwen3.8 2.4T BF16 model in 4.3 seconds, but the post gives no gather-and-broadcast time for that model, so there is no ratio for it. The table’s training→inference layouts are H100 NVL8 Qwen 3 235B TP4×PP8×EP16→TP8×DP16 (6.4→1.9 s); H100 NVL8 DeepSeek v3 PP16×EP16→TP32×DP8 (14.9→2.6 s); GB200 NVL72 DeepSeek v3 with that same layout (6.1→2.3 s); and GB200 NVL72 Nemotron Ultra v3 TP8×EP32×PP2→TP8×DP8 (8.2→2.3 s).

Read the numbers for what they measure. They are refit latency, not training throughput or time to a target reward, and the post does not say how much of each RL step refit takes, so we cannot translate 2.7x to 5.7x into an end-to-end speedup. The comparison is with NVIDIA's own conventional path, not with other refit methods in other frameworks. Four configurations on two NVIDIA systems are a small set, the post does not report run-to-run spread, and we did not reproduce any of it.

The NeMo RL documentation adds the limits. It labels `nccl_reshard_refit` experimental. The setting is `refit_transport: nccl_reshard` with `policy.generation.colocated.enabled` set to false, because it works only with a non-colocated setup. It requires the Megatron training backend and a vLLM or Megatron generation backend, and says SGLang and TRTLLM are not supported yet. A vLLM destination still needs expert tensor parallel size 1. The post adds that when nodes join or leave during failure recovery, NeMo RL currently destroys and recreates the communication group, and that using NCCL shrink and grow APIs is planned for later.

Our reading: this is a credible engineering idea with real numbers, and the abstraction is useful because refit cost grows with model size. Teams that train very large mixture-of-experts models with Megatron and vLLM, with a non-colocated training/generation resource configuration, could test it, starting from the experimental flag. Others should wait for broader backend support, measure what share of their step time refit takes, and treat the headline speedup as a best-case hint, not a guarantee.