Glossary term
Glossary term
Infrastructure and Serving
Open-source tool for running LLMs locally on macOS, Linux, and Windows with a simple CLI and REST API.
Ollama has over 100,000 GitHub stars and is used by 2M+ developers to run Llama 3, Mistral, Phi-3, Gemma, and 100+ other models locally with a single command, making on-device LLM inference accessible without cloud APIs.
A law firm uses Ollama to deploy Llama 3.1 8B on attorneys' MacBook Pro M3 laptops, enabling offline document review without sending confidential client materials to external cloud providers.
Cursor AI and Continue.dev support Ollama as a local model backend, allowing developers to run code-completion assistants locally on their workstation without latency introduced by cloud round-trips, used widely in air-gapped development environments.