Glossary term
Glossary term
Agentic Systems
A metric for evaluating an ML model's generated text. The fraction of successes is the number of "successful" generated text outputs divided by the total number of generated text outputs. For example, if a large language model generated 10 blocks of code, five of which were successful, then the fraction of successes would be 50%.
Although fraction of successes is broadly useful throughout statistics, within ML, this metric is primarily useful for measuring verifiable tasks like code generation or math problems.
Created for this library
An RL team in robotics reports the fraction of successes on grasping in simulation as the headline metric across training checkpoints.
A logistics RL team reports the fraction of successes on delivery completion in its simulator to compare policy versions.
A trading research team reports the fraction of successes in hitting target prices when comparing execution agents in backtests.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License