AgentTested hands-on review
tool: AutoGen
slug: autogen
date: 2026-10-21
score: 6/10

Coverage: Price tool, unknown SKU, and tool-then-math. Hosted and local.
What we tested: The site's agents task, as for LangChain: a get_price tool that knows MUG-01 and errors on unknown SKUs, plus a tool-then-math question. Hosted and local backends.
Method: autogen-agentchat / autogen-ext[openai] 0.7.5, AssistantAgent with reflect_on_tool_use, OpenAIChatCompletionClient pointed at OpenRouter nemotron-3.5-super-vl:free and at Ollama gemma4-e2b. Temperature 0.

Notes:
- CPU-only box: 8 cores, 15 GB RAM (about 4-10 GB free during tests), no GPU.
- 3 questions
- Archived upstream

Prompts:
- What is the price of SKU MUG-01? Use the tool.
- What is the price of SKU XYZ-99? Use the tool.
- What is the total price of 3 units of SKU MUG-01? Use the tool, then compute.

Results:
install: 28 packages, about 1.5 s with a warm uv cache
import_s: 1.5
pass: 3/3 on both backends. The hosted XYZ-99 answer ('not recognized in our system') was graded a pass by hand; our keyword check missed it
openrouter.mug: tool MUG-01, final 'The price of SKU MUG-01 is $12.50.', 24.5 s, passed
openrouter.unknown: tool XYZ-99, final 'I'm sorry, but SKU XYZ-99 is not recognized in our system.', 1.3 s, keyword check failed
openrouter.total: tool MUG-01, final 'The price for SKU MUG-01 is 12.50 per unit. Total price for 3 units: 3 × 12.50 = **37.50**', 1.8 s, passed
local.mug: tool MUG-01, final 'The price of SKU MUG-01 is $12.50.', 11.6 s, passed
local.unknown: tool XYZ-99, final 'I am sorry, I cannot find the price for SKU XYZ-99. The system returned an unknown SKU.', 14.3 s, passed
local.total: tool MUG-01, final 'The total price for 3 units of SKU MUG-01 is $37.50.', 20.7 s, passed

Pros:
- Tool calls and error handling worked with both a hosted and a local model
- Little code for a tool-using agent
- Error from the tool reached the model and became a plain answer

Cons:
- The project is archived; Microsoft says to use Microsoft Agent Framework for new work
- First hosted call took 24.5 s (upstream latency)
- Needs a hand-written model_info block for non-OpenAI models

Verdict: AutoGen's agent called our price tool correctly every time, handled the unknown-SKU error gracefully, and did the multiply-after-lookup step, with both backends. It did better than LangChain did on the same error case. The catch is that Microsoft has archived AutoGen and points new users to Microsoft Agent Framework, so it won't get fixes.

Score line: 3/3 on both backends, but the project is archived.