InfrastructureCompute, chips & cloud
NVIDIA open-sources Reshard, an NCCL collective that speeds weight refit in RL post-training
NVIDIA's Reshard is an open-source NCCL collective that moves only the overlapping weight slices between training and inference GPU layouts during reinforcement learning post-training, cutting the refit step for large mixture-of-experts models from many seconds to a few.
From the NVIDIA Generative AI article of November 6, 2026 and cited sources: Reshard treats refit as converting each parameter from one DTensor layout to another. [1]
SOURCES[1] Accelerating Reinforcement Learning Weight Refit with the… developer.nvidia.com
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- Reshard treats refit as converting each parameter from one DTensor layout to another. [1]
- Original source
- NVIDIA Generative AI article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- Studies
- arXiv 2510.27656 (Oct 2025) [3] date from arXiv ID
- When
- November 6, 2026 article published
- Covered
- Nov 9 edition
- Evidence
- 10 sources across 10 sites
- 1Accelerating Reinforcement Learning Weight Refit with the…developer.nvidia.com
- 2Perplexity: Weight Transfer for RL Post-Training in under…perplexity.ai
- 3arXiv 2510.27656 (TransferEngine)arxiv.org
- 4Emergent Mind: 2510.27656emergentmind.com
- 5AI Weekly: NeMo-DCR cuts 1T RL weight syncaiweekly.co
- 6NeMo RL: NCCL Reshard Refit (Experimental)docs.nvidia.com
Reach: not measured yet
Reshard treats refit as converting each parameter from one DTensor layout to another. From each side's global shape, device mesh and placements, it works out where source and destination shards overlap, so each GPU sends only the slices it needs. Replicas in an NVLink domain each pull a different slice over the network and then swap slices over NVLink, so each shard crosses the network only once. [1]
In NVIDIA-run NeMo RL benchmarks on DGX H100 and GB200, refit ran 2.7x to 5.7x faster than gather-and-broadcast on Qwen 3 235B, DeepSeek v3 671B and Nemotron Ultra v3. DeepSeek v3 on H100 dropped from 14.9 s to 2.6 s. Only feed-forward weights go through Reshard; attention, embedding, norm and router weights are still broadcast. [1] [6]
The NCCL M2N Reshard library ships in nccl-extensions and needs NCCL 2.30.5 or newer. vLLM exposes it as an nccl_m2n weight-transfer engine. In NeMo RL it is an experimental option for Megatron training on separate GPUs, with vLLM or Megatron for generation, and supports only unquantized, blockwise FP8 and MXFP8 weights. [6] [7] [8]
Perplexity has reported a 1.3 s refit for the trillion-parameter Kimi-K2 using point-to-point RDMA on different hardware, so the two sets of figures cannot be compared directly. NVIDIA's separate NeMo-DCR shrinks the payload by sending only changed weights, which could in principle be combined with Reshard's removal of redundant traffic. [2] [3] [5]
Refit happens every RL training step, and the reported tests show it getting several times faster on large models like DeepSeek v3.
Read the full assessment
If that holds on other clusters, disaggregated RL setups could keep inference GPUs busier, and Megatron and vLLM users can turn it on with one config flag.
Executive brief
NVIDIA has released "Reshard," a new collective communication primitive for one job: moving updated weights from training GPUs to inference GPUs during reinforcement learning (RL) post-training, a step called "refit." In NVIDIA's own NeMo RL benchmarks, refit ran 2.7x to 5.7x faster than the usual gather-and-broadcast method. On DGX H100, refit for DeepSeek v3 671B dropped from 14.9 s to 2.6 s (NVIDIA blog). NVIDIA estimates that naive refit for today's largest open-weight models moves about 10 TB per training step. The reference implementation, NCCL M2N Reshard, is open source, and NeMo RL marks the feature as experimental. No independent benchmarks were found.
What changed and event timeline
Perplexity reports 1.3 s trillion-parameter refit
Perplexity used one-sided RDMA WRITE with a static transfer schedule to update Kimi-K2 across 256 BF16 training GPUs and 128 FP8 inference GPUs ().
TransferEngine paper posted
Perplexity's arXiv paper describes the same point-to-point approach and claims it is more than 100x faster than earlier methods (;).
NVIDIA NeMo-DCR reported
This is a separate NVIDIA approach that sends only the weights that changed. It reportedly cuts a 1T-parameter sync from 87.5 minutes to 150 seconds ().
Reshard collective published
NVIDIA presents refit as a transfer from one DTensor layout to another (DTensor is PyTorch's sharded-tensor format) and open-sources NCCL M2N Reshard. NeMo RL adds
refit_transport: nccl_reshard(;).
Capabilities and access
- The NCCL M2N Reshard library ships through
nccl-extensionsand is imported asnccl.m2n. It needs NCCL 2.30.5 or newer (vLLM docs; PyPI). - NeMo RL (experimental): works only when training and inference run on separate GPUs, with Megatron for training and vLLM or Megatron for generation.
- vLLM has an
nccl_m2nweight-transfer engine. It does not support pipeline-parallel trainers or CUDA graph capture during reshards (vLLM docs).
Read the full section
- The NCCL M2N Reshard library ships through
nccl-extensionsand is imported asnccl.m2n. It needs NCCL 2.30.5 or newer (vLLM docs; PyPI). - NeMo RL (experimental): works only when training and inference run on separate GPUs, with Megatron for training and vLLM or Megatron for generation. Supported precisions are unquantized, blockwise FP8 and MXFP8. Pipeline parallelism greater than 1 on the generation side is not yet supported (design doc).
- vLLM has an
nccl_m2nweight-transfer engine. It does not support pipeline-parallel trainers or CUDA graph capture during reshards (vLLM docs).
Technical analysis for researchers and developers
- Each side describes a parameter as a DTensor: its global shape, a device mesh and placements.
- Replicas are grouped by NVLink domain. Each replica receives a different slice over the network, then the replicas exchange slices over NVLink.
- Attention, embedding, norm and router weights still use broadcast. Quantized weights are converted to BF16 for transfer and requantized afterwards (design doc).
Read the full section
- Abstraction. Each side describes a parameter as a DTensor: its global shape, a device mesh and placements. Reshard works out exactly where source and destination shards overlap, so each GPU sends only the slices that are needed. The blog's example: an 8×8 matrix split by rows across 2 GPUs and by columns across 4 GPUs needs a 4×2 region per source–destination pair.
- Topology awareness. Replicas are grouped by NVLink domain. Each replica receives a different slice over the network, then the replicas exchange slices over NVLink. Both steps are pipelined, so each shard crosses the network only once instead of once per replica (NVIDIA blog).
- Hybrid path. Feed-forward projection weights go through Reshard. Attention, embedding, norm and router weights still use broadcast. Quantized weights are converted to BF16 for transfer and requantized afterwards (design doc).
- Implementation details. The library includes source files for mesh topology, load balancing and memory-window registration (DeepWiki).
Claims and evidence
All of these are NVIDIA's own figures. No independent reproduction was found.
Read the full section
| Claim | Evidence |
| 2.7x–5.7x refit speedup (Qwen 3 235B, DeepSeek v3 671B, Nemotron Ultra v3) | NVIDIA-measured in NeMo RL on DGX H100 and DGX GB200 (blog) |
| 4.3 s refit for a 2.4T-parameter BF16 model | NVIDIA refit-only microbenchmark; full configuration not documented |
| About 10 TB payload per naive refit | NVIDIA arithmetic (parameters × 2 bytes × 2 stages), not a measurement |
| DeepSeek v3 refit of 1.93–2.59 s on GB200; payload efficiency of 66–97% | NeMo RL docs; efficiency is lower for FP8 (design doc) |
All of these are NVIDIA's own figures. No independent reproduction was found.
Context and prior work
- Perplexity TransferEngine: reported 1.3 s for 1T-parameter Kimi-K2 using point-to-point RDMA (Perplexity).
- Kimi checkpoint-engine: reportedly about 20 s for a Kimi-K2 update across thousands of GPUs (checkpoint-engine mirror).
- NeMo-DCR: compresses the payload by sending only changed weights. Reshard removes redundant traffic instead, so the two approaches could in principle be combined (AI Weekly).
Read the full section
- Perplexity TransferEngine: reported 1.3 s for 1T-parameter Kimi-K2 using point-to-point RDMA (Perplexity). The hardware and setup differ from NVIDIA's, so the numbers cannot be compared directly.
- Kimi checkpoint-engine: reportedly about 20 s for a Kimi-K2 update across thousands of GPUs (checkpoint-engine mirror).
- NeMo-DCR: compresses the payload by sending only changed weights. Reshard removes redundant traffic instead, so the two approaches could in principle be combined (AI Weekly).
Limitations, safety and contested findings
- The blog says feed-forward weights are about 95% of parameters. The design doc says the bulk path covers about 97–98% of weights.
- The feature has precision limits, no generation-side pipeline parallelism, and no ModelOpt real quantization (design doc).
- The engine caps each parameter at 16 sources and 64 targets (vLLM docs).
Read the full section
- Inconsistent share figures. The blog says feed-forward weights are about 95% of parameters. The design doc says the bulk path covers about 97–98% of weights.
- Experimental status. The feature has precision limits, no generation-side pipeline parallelism, and no ModelOpt real quantization (design doc).
- vLLM limits. The engine caps each parameter at 16 sources and 64 targets (vLLM docs).
- Failure recovery. If GPU membership changes, NeMo RL currently tears down and rebuilds the communicator. Using NCCL's shrink and grow APIs is planned but not yet available (blog).
- Unexplained gaps. The baselines are NVIDIA's own gather-and-broadcast path, and the reported times are still above Perplexity's 1.3 s claim. The reviewed sources do not explain the difference.
Business and practitioner implications
- Higher GPU utilization in disaggregated RL. Refit runs every training step.
- Low adoption cost for Megatron + vLLM users. Enabling it is a single config flag.
- A shared primitive across frameworks.
Read the full section
- Higher GPU utilization in disaggregated RL. Refit runs every training step. Cutting it from seconds to fewer seconds on large mixture-of-experts models frees inference GPUs that would otherwise sit idle during weight updates.
- Low adoption cost for Megatron + vLLM users. Enabling it is a single config flag. Teams should measure payload efficiency on their own layouts, especially with FP8.
- A shared primitive across frameworks. Packaging refit as a library collective, already exposed in both NeMo RL and vLLM, means framework teams no longer need to write custom resharding code for each parallelism layout.
The source trail.
Sources (10)
Accelerating Reinforcement Learning Weight Refit with the Reshard Collective
Article text retrieved; extracted text may omit tables or interactive elements.
developer.nvidia.com