AgentTested hands-on review
tool: llama.cpp
slug: llama-cpp
date: 2026-10-21
score: 7/10

Coverage: Build from source, CPU llama-bench, and one coding prompt
What we tested: Build-from-source friction, CPU speed (llama-bench), and one coding prompt.
Method: git clone, cmake -DGGML_NATIVE=ON, build llama-cli and llama-bench. Same Qwen2.5-Coder GGUFs as the Ollama test. llama-bench -p 512 -n 128 -t 8 -r 3. llama-cli -st with the palindrome prompt.

Notes:
- CPU-only box: 8 cores, 15 GB RAM, no GPU.
- Bench variance high; another process may have competed

Results:
build: First try failed (cmake not installed). After apt install cmake: 179 s build
bench_1.5b: pp512 402 ± 62 t/s, tg128 18.2 ± 5.5 t/s
bench_3b: pp512 190 ± 20 t/s, tg128 12.8 ± 1.3 t/s
cli_coding: qwen2.5-coder-3b answer passed 6/6 unit cases; 105 t/s prompt, 12.2 t/s generation
cli_flag_note: Older tutorials' -no-cnv flag is rejected; the new single-turn flag is -st

Pros:
- Pure CPU speed and full control
- Clean, correct output
- Built-in benchmarking

Cons:
- Needs cmake and a 3-minute compile
- CLI flags change between versions

Verdict: llama.cpp built cleanly once cmake was installed and gave solid CPU throughput, about 400 tokens/s reading prompts and 13-18 tokens/s generating with 1.5-3B models. The 3B coder answered our palindrome task correctly. Generation speed varied a lot between runs (±5.5 t/s on the 1.5B), so take the exact numbers loosely. It's more hands-on than Ollama but gives you full control.

Score line: Fast and correct on CPU. Compile step and shifting CLI flags add friction.