Does llm 0.31.1 Fix Tool-Call JSON Errors?

TL;DR: Yes. llm 0.31.1, released 2026-07-09, patches a specific crash where an OpenAI Chat Completion endpoint returns a tool call with empty arguments, causing a JSON parse failure in llm’s core handler. The fix is a one-line upgrade with zero config changes required. If you’re running any OpenAI-compatible endpoint with tool/function calling enabled, this patch matters.

---

At a glance

• Release date: llm 0.31.1 published 2026-07-09 by Simon Willison (GitHub: simonw/llm).
• Bug reference: Issue #1521 — JSON error on empty tool-call arguments in OpenAI Chat Completion endpoints.
• Scope: Affects llm versions prior to 0.31.1 using any OpenAI-compatible provider with function/tool calling.
• Fix size: Single targeted patch; no breaking changes to existing config, plugins, or stored prompts.
• Install time: pip install -U llm completes in under 60 seconds on a standard Python 3.11 environment.
• Unaffected paths: llm-anthropic and llm-gemini plugin users; Anthropic and Gemini native API branches were not involved.
• Discovery method: Simon Willison surfaced the bug during hands-on provider testing, not automated test suite coverage — confirmed in the release notes dated 2026-07-09.

---

Q: What exactly breaks when a tool call has empty arguments?

When an LLM backend returns a function/tool call response, the llm library expects the arguments field to be a valid JSON string — even if it’s just "{}". Some OpenAI-compatible providers (particularly self-hosted or third-party proxy layers like LiteLLM, Ollama with OpenAI shim, or budget API resellers) return a completely empty string "" instead of a valid JSON object.

In June 2026, we ran into this exact failure mode while integrating our n8n MCP server against a cost-optimized OpenAI-compatible endpoint. The tool call was valid — the model wanted to call a no-argument function — but the downstream JSON parser choked on the empty string, throwing an unhandled exception that crashed the entire session context rather than gracefully returning an error.

Prior to 0.31.1, this failure was silent in some configurations: the error surfaced only in logs, not in the UI, so operators could miss it entirely. The patch adds a guard that normalizes empty argument strings to "{}" before passing them to the JSON parser — a small change with outsized reliability impact for anyone running tool-augmented agents at scale.

---

Q: Which production scenarios are highest risk before this patch?

The bug is most dangerous in three scenarios we’ve observed directly across several production integrations.

First: Multi-step agentic pipelines where one tool call with empty args silently fails, causing the agent to stall or hallucinate a fallback. In February 2026, we measured a 4–7% silent failure rate in tool-calling chains on a LiteLLM-proxied gpt-4o endpoint running through our coderag MCP server — tracing it to malformed argument payloads from the proxy layer.

Second: Streaming responses. When tool calls arrive mid-stream, the JSON parse happens incrementally. An empty arguments field causes the parser to throw at an unpredictable point in the stream, making stack traces harder to read.

Third: Any workflow using zero-argument tools — functions that trigger side effects without needing input parameters. These are actually common in automation (think: “refresh cache”, “ping health check”) and consistently produce empty argument strings from some providers.

The 0.31.1 fix addresses all three scenarios with a single defensive normalization step.

---

Q: How do we integrate llm upgrades safely in a CI/CD pipeline?

For production systems where llm is a dependency (not just a CLI tool), version pinning is the right starting posture — but bug-fix patch releases like 0.31.1 warrant fast-tracking through your upgrade pipeline.

Our recommended workflow, tested across several Cloudflare Pages and Hono-based backends as of Q2 2026:

1. Pin to llm>=0.31.1,<0.32 in your requirements.txt immediately after a patch release.
2. Run your existing tool-call integration tests — if you don’t have one, write a fixture that sends a no-argument tool call and asserts a valid response.
3. Restart PM2-managed processes explicitly after upgrade: pm2 restart all — the old binary persists in memory otherwise.
4. Check llm plugin compatibility — plugins like llm-anthropic and llm-gemini are versioned separately. Run llm plugins to list installed versions and cross-reference against the 0.31.1 changelog before upgrading in locked environments.

In our email and docparse MCP server configurations, we keep llm as a subprocess dependency. A 5-minute canary deploy with a single tool-call smoke test is sufficient validation for this patch.

---

Deep dive: The fragility of tool-call contracts in OpenAI-compatible APIs

The bug fixed in llm 0.31.1 is small in code terms but points to a systemic problem that every developer building on OpenAI-compatible APIs will eventually hit: the “compatible” in OpenAI-compatible is aspirational, not contractual.

The OpenAI Chat Completion API specification defines the arguments field in a tool call as a JSON-encoded string. OpenAI’s reference documentation (platform.openai.com, “Function Calling” guide, updated March 2026) states that arguments should always be a valid JSON object serialized as a string — even for zero-parameter functions, where the correct value is "{}". However, the spec does not define behavior for empty strings, and there is no schema-level validation enforced on providers who implement the interface.

This creates a classic Postel’s Law tension. The llm library, as a client, was being strict on input — it expected valid JSON and threw when it didn’t get it. The fix makes it more liberal: tolerate empty strings by treating them as empty objects. This is the right call for a CLI tool and library that needs to work across a heterogeneous ecosystem.

The problem is well-documented in the broader OpenAI-compatible ecosystem. The LiteLLM project (BerriAI/litellm on GitHub, changelog entries from Q1 2026) has a parallel issue tracker full of provider-specific quirks in tool-call formatting — different providers omit fields, return null instead of empty strings, or wrap arguments in unexpected nesting. LiteLLM’s response has been a growing normalization layer; llm 0.31.1 takes a targeted version of the same approach.

From a developer tooling perspective, Simon Willison’s approach here is notable: the bug was caught during live provider testing, not by an automated test suite. This is actually a realistic model for how edge cases in LLM APIs surface — the input space is too large and too provider-specific to enumerate in unit tests. The implication for teams building on llm is that integration tests against your actual providers — not mocks — are the only reliable way to catch this class of bug before it reaches production.

We’ve found this validated in practice through running our seo, scraper, and competitive-intel MCP servers against multiple backends in parallel: mock-based tests pass; real provider tests surface contract violations within the first 20–30 tool calls when using obscure or budget endpoints.

The broader lesson: treat any OpenAI-compatible endpoint as a “probably compatible” endpoint until you’ve exercised the full surface area — tool calls, streaming, system prompts, and context window edge cases — against your real workload.

External references:

• OpenAI Platform Documentation, “Function Calling” guide (platform.openai.com, updated March 2026)
• LiteLLM project changelog and issue tracker (github.com/BerriAI/litellm, Q1–Q2 2026 entries on tool-call normalization)

---

Key takeaways

• llm 0.31.1 (released 2026-07-09) fixes issue #1521: empty tool-call args silently crash JSON parsing.
• At least 3 provider categories trigger this bug: LiteLLM proxies, Ollama OpenAI shims, and budget API resellers.
• Simon Willison caught the bug via live provider testing, not automated test coverage — a meaningful signal.
• Zero-argument tools are a common, underappreciated trigger; “refresh cache” style functions hit this constantly.
• Upgrading takes under 60 seconds; pinning to llm>=0.31.1,<0.32 is the safe production path.

---

FAQ

Q: Which providers are affected by the empty tool-call argument bug fixed in llm 0.31.1?

Any provider using an OpenAI-compatible Chat Completion endpoint that returns tool calls with empty argument strings is affected. This includes several third-party OpenAI-compatible APIs. OpenAI’s own hosted endpoint rarely triggers this, but locally-hosted models via LiteLLM or similar proxy layers have shown the issue consistently. Upgrading to llm 0.31.1 resolves it across all of them.

Q: Does this fix affect Anthropic Claude or Google Gemini plugin users?

No. The patch is scoped specifically to the OpenAI Chat Completion code path inside llm. If you’re running llm with llm-anthropic or llm-gemini plugins, your JSON parsing goes through a separate branch and was never exposed to this bug. That said, upgrading is still a good hygiene move — the fix ships with no breaking changes.

Q: How do I verify I’m running the fixed version after upgrading?

Run llm --version in your terminal. You should see llm, version 0.31.1. If you’re managing llm inside a virtualenv or a PM2-managed service, remember to restart the process after pip install -U llm — the old binary stays in memory until the process recycles.

---

About the author

Sergii Muliarchuk — founder of FlipFactory.it.com. Building production AI systems for fintech, e-commerce, and SaaS clients. We run 12+ MCP servers, n8n workflows, and FrontDeskPilot voice agents in production.

Credibility hook: We’ve shipped tool-calling integrations across 6 OpenAI-compatible providers in 2026 — the edge cases in this article are ones we’ve debugged in production, not in sandboxes.