Glossary term
Glossary term
Agentic Systems
A group of prompts for evaluating a large language model. For example, the following illustration shows a prompt set consisting of three prompts:
Good prompt sets consist of a sufficiently "wide" collection of prompts to thoroughly evaluate the safety and helpfulness of a large language model.
See also response set.
Created for this library
An LLM evaluation team curates a prompt set per business domain so model selection is grounded in domain-relevant tasks.
A research lab maintains a prompt set of edge cases that previous models failed on to drive future improvements.
A SaaS team maintains a prompt set per product feature so every prompt change can be evaluated against the same baseline.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License