Glossary term
Glossary term
Infrastructure and Serving
When training an ML model on accelerator chips (GPUs or TPUs), the part of the system that controls both of the following:
The overall flow of the code.
The extraction and transformation of the input pipeline.
The host typically runs on a CPU, not on an accelerator chip; the device manipulates tensors on the accelerator chips.
Created for this library
An ML engineer profiles host-side data loading to confirm the GPU is not stalled waiting for input batches.
An ML platform team monitors host memory usage during training to spot leaks before they crash long-running jobs.
A research engineer offloads optimizer state to host memory to fit a larger model on the same accelerator device.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License