Data, Tensor & Pipeline Parallelism
The three axes of splitting a model and batch across devices — and when to use each.
advanced#parallelism#tensor-parallel#pipeline
Full write-up in progress
This topic is on the roadmap and its detailed page — theory, math, code, quizzes and projects — is being authored. Its metadata, prerequisites and links are ready below.