llama.cpp
A free engine for running AI models on your own computer. It's for people who write code. It's free.
Get patch notes
Same daily newsletter as the rest of the site. One email a day.
No affiliate link on file. Outbound goes to github.com. Identity checked October 21, 2026. Score null is from the run on October 21, 2026.
Grade
We tried it ourselves. Grade B.
- Grade
- B · We tried it
- Score
- 7/10
- Price
- Free
Best for
Jobs people use this for.
Test details
C/C++ LLM inference engine for GGUF models on CPU or GPU. MIT license.
Hands-on review · 2026-10-21
TestedB7/10- CPU-only box: 8 cores, 15 GB RAM, no GPU.
- Bench variance high; another process may have competed
llama.cpp built cleanly once cmake was installed and gave solid CPU throughput, about 400 tokens/s reading prompts and 13-18 tokens/s generating with 1.5-3B models. The 3B coder answered our palindrome task correctly. Generation speed varied a lot between runs (±5.5 t/s on the 1.5B), so take the exact numbers loosely. It's more hands-on than Ollama but gives you full control.
Fast and correct on CPU. Compile step and shifting CLI flags add friction.
- Coverage
- Build from source, CPU llama-bench, and one coding prompt
- What we tested
- Build-from-source friction, CPU speed (llama-bench), and one coding prompt.
Method
git clone, cmake -DGGML_NATIVE=ON, build llama-cli and llama-bench. Same Qwen2.5-Coder GGUFs as the Ollama test. llama-bench -p 512 -n 128 -t 8 -r 3. llama-cli -st with the palindrome prompt.
Measured
- buildFirst try failed (cmake not installed). After apt install cmake: 179 s build
- bench_1.5bpp512 402 ± 62 t/s, tg128 18.2 ± 5.5 t/s
- bench_3bpp512 190 ± 20 t/s, tg128 12.8 ± 1.3 t/s
- cli_codingqwen2.5-coder-3b answer passed 6/6 unit cases; 105 t/s prompt, 12.2 t/s generation
- cli_flag_noteOlder tutorials' -no-cnv flag is rejected; the new single-turn flag is -st
Pros
- Pure CPU speed and full control
- Clean, correct output
- Built-in benchmarking
Cons
- Needs cmake and a 3-minute compile
- CLI flags change between versions
- Status
- Not run
- Date
- October 21, 2026
- Time to first output
- —
- Cost
- —
- Result
- —
- Evidence
- Log linked below
Queued task. Answer a fixed research brief. Log whether claims cite sources, time to a usable answer, and cost.
FAQ
What is the agent-tested verdict for llama.cpp?
A free engine for running AI models on your own computer. It's for people who write code. It's free. We tried it ourselves. Grade B.
How much does llama.cpp cost?
Free. Free, open source
Do sponsors outrank this tool?
No. A sponsored slot is labeled and sold separately from tiers. It cannot move this listing.