LangChain

• CPU-only box: 8 cores, 15 GB RAM, no GPU.
• Local tool-calling failures were partly due to the test setup.
• LLM was a free hosted model; quality is partly the model's

LangChain's plumbing worked. With a free hosted model it made the right tool call and answered correctly in under 2 seconds. On the failure case the error reached the model, but the final reply was empty, so the user would see nothing. That's partly the model and partly the missing retry or fallback logic you'd have to write yourself. Local tool calling failed in our setup because of how we imported the model.

Happy path passed, failure case gave an empty reply, and local tool calling didn't work in our setup.

Coverage

One tool call and one failure case, plus a local model that could not call tools

What we tested

One tool call plus one failure case (the site's agents task): a get_price tool that knows MUG-01 and raises an error for unknown SKUs.

Method

langchain + langchain-openai pointed at OpenRouter's free MiniMax M3.1 Flash, bind_tools, one tool round trip. Also tried langchain-ollama with our imported qwen2.5-coder-3b.

Prompts

1. What is the price of SKU MUG-01? Use the tool.
2. What is the price of SKU XYZ-99? Use the tool.

Measured

• minimax MUG-01tool call sku MUG-01, final: The price of SKU **MUG-01** is **$12.50 USD**., 1.7 s, passed
• minimax XYZ-99tool call sku XYZ-99, final empty, 2.2 s, failed
• ollama_imported_qwen3b0 tool calls. The model printed the call as JSON text because our GGUF import had no tool template (our setup, not LangChain)

Raw test log