Glossary term
Glossary term
Foundations
Broadly speaking, an array of floating-point numbers taken from any hidden layer that describe the inputs to that hidden layer. Often, an embedding vector is the array of floating-point numbers trained in an embedding layer. For example, suppose an embedding layer must learn an embedding vector for each of the 73,000 tree species on Earth. Perhaps the following array is the embedding vector for a baobab tree:
An embedding vector is not a bunch of random numbers. An embedding layer determines these values through training, similar to the way a neural network learns other weights during training. Each element of the array is a rating along some characteristic of a tree species. Which element represents which tree species' characteristic? That's very hard for humans to determine.
The mathematically remarkable part of an embedding vector is that similar items have similar sets of floating-point numbers. For example, similar tree species have a more similar set of floating-point numbers than dissimilar tree species. Redwoods and sequoias are related tree species, so they'll have a more similar set of floating-pointing numbers than redwoods and coconut palms. The numbers in the embedding vector will change each time you retrain the model, even if you retrain the model with identical input.
Created for this library
A search team stores the embedding vector of every document in a vector index so semantic retrieval can find relevant passages by similarity.
A retail recommendation team caches a user embedding vector after each visit so subsequent recommendations are personalized with no extra computation.
A help-desk team builds an embedding vector of every past ticket so new tickets can be routed to the team that solved similar issues before.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License