Glossary term
Glossary term
Infrastructure and Serving
Abbreviation for single program / multiple data.
Created for this library
An ML platform team uses SPMD execution in JAX to run the same training step across many devices on different data shards.
A research engineer uses SPMD with pjit to declare how parameters and activations are sharded over a device mesh.
An ML platform team uses SPMD-style sharding in its very large training jobs to scale across TPU pods.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License