DMuon: Scaling Matrix Optimizers in Data Parallelism with Topology-Aware Matrix Ownership and Communication
arXiv:2611.07084v1 Announce Type: new Abstract: Matrix optimizers such as Muon update whole weight matrices, which conflicts with the element- or row-wise shard…