Accelerating Reinforcement Learning Weight Refit with the Reshard Collective
During reinforcement learning (RL) post-training, training trajectories are generated on the fly by running inference with the current policy model. For...During reinforcement learning (RL) post-training, training trajectories are generated on the fly by running inference with the current policy model. For training efficiency, it is common to run inference and training in parallel on separate GPU resources in a disaggregated setup. Although the policy model is updated only on the training GPUs, the system must periodically refresh the weights on the…