Gemma 4

• CPU-only box: 8 cores, 15 GB RAM (about 4-10 GB free during tests), no GPU.
• Small battery
• Tested E2B only, not 26B or 31B

The small Gemma 4 model passed every task in both runs on a plain CPU, the first local model to get a perfect score on our battery. The cost is time: it reasons before answering, so a trick question that hosted models answer in a second took up to 77 seconds here. We could not test the bigger hosted Gemma 4 models because the free tier was rate-limited on every call.

8/8 offline on CPU, but slow because it thinks at length. The 31B hosted model is still untested.

Coverage

Same 4-task battery, 2 runs, on local Gemma 4 E2B. Hosted OpenRouter Gemma 4 calls were rate-limited.

What we tested

Same 4-task battery as earlier chat models (code with 6 unit checks, 2 trick questions, 60-word summary of a 700-word text), 2 runs. We first retried the hosted OpenRouter Gemma 4 models; every call was rate-limited, so we ran the small Gemma 4 model locally on CPU.

Method

unsloth/gemma-4-E2B-it-GGUF Q4_K_M (3.1 GB, 25 s download from Hugging Face), imported with `ollama create` (12 s; Ollama 0.40.7 recognised the gemma4 architecture, template and tool support). The `ollama pull gemma4:e2b` registry download was blocked by the box network. Temperature 0, seed 1, default thinking on. Script _battery/battery.py.

Measured

• pass_rate8/8
• generation_speed19.5-23.4 tokens/s on CPU
• latency10-77 s per task (run1 code includes model load)
• hosted_retryOpenRouter google/gemma-4-31b-it:free (3 more runs) and google/gemma-4-26b-a4b-it:free (2 runs): all 20 calls returned 429 'temporarily rate-limited upstream' (Google AI Studio). Logs in or-gemma-4-31b-it/run4-6.log and or-gemma-4-26b-a4b-it/.
• run1.codepassed, 6/6 unit cases, wall 39.73 s, 656 out tokens, 19.5 tok/s
• run1.reason1passed, expect 5, wall 10.9 s, 240 out tokens, 23.4 tok/s
• run1.reason2passed, expect 5, wall 48.37 s, 1048 out tokens, 21.8 tok/s
• run1.summarypassed, 49 words, 3/3 key terms, wall 25.14 s, 467 out tokens, 22.1 tok/s
• run2.codepassed, 6/6 unit cases, wall 29.34 s, 637 out tokens, 22.4 tok/s
• run2.reason1passed, expect 5, wall 10.15 s, 231 out tokens, 23.3 tok/s
• run2.reason2passed, expect 5, wall 76.58 s, 1670 out tokens, 21.9 tok/s
• run2.summarypassed, 46 words, 3/3 key terms, wall 21.61 s, 471 out tokens, 21.9 tok/s

Raw test log