Glossary term
Glossary term
Infrastructure and Serving
An overloaded term with the following two possible definitions:
A category of hardware that can run a TensorFlow session, including CPUs, GPUs, and TPUs.
When training an ML model on accelerator chips (GPUs or TPUs), the part of the system that actually manipulates tensors and embeddings. The device runs on accelerator chips. In contrast, the host typically runs on a CPU.
Created for this library
An ML platform team allocates GPUs across training jobs by tagging each job with the device type it requires for fair scheduling.
A speech recognition team profiles its model on CPU, GPU, and TPU devices to choose the cheapest device that meets the latency target.
A mobile team benchmarks its on-device model across several handset devices so the worst-case device still meets the user-experience latency budget.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License