Fine grained tool calling

You turn on streaming for a tool-calling response, expecting the smooth token-by-token trickle you saw for text responses. Instead, output arrives in strange bursts — nothing for several seconds, then a wall of text, then nothing again, then another wall. You haven't broken streaming; the API is doing exactly what it's designed to do, and there's a specific flag for when you want it to stop.

Streaming tool arguments — the InputJsonEvent

When you enable streaming with stream=True or use client.messages.stream(), you're already familiar with ContentBlockDelta events for text. For tool calls, you get a new event type: InputJsonEvent, which carries chunks of the JSON arguments as Claude builds them.

Each InputJsonEvent has two fields worth caring about:

• partial_json — the new chunk that just arrived.
• snapshot — the cumulative JSON assembled from every chunk so far.

Basic handling looks like:

for chunk in stream:
    if chunk.type == "input_json":
        print(chunk.partial_json, end="")
        current_args = chunk.snapshot  # cumulative so far

That's the API surface. The interesting behaviour is what partial_json actually contains by default.

The default: buffered, validated bursts

By default, the Anthropic API buffers streaming tool arguments and only releases them after a complete top-level key-value pair has been validated against your schema. It's a feature — the API is doing the work of making sure the JSON you assemble from the stream is always structurally valid.

Say your tool schema expects:

{
  "abstract": "This paper presents a novel approach...",
  "meta": {
    "word_count": 847,
    "review": "This paper introduces QuanNet..."
  }
}

Here's what the API does on a streaming call:

1. Claude starts generating the value for abstract. The API buffers the chunks.
2. When abstract is complete, the API validates the key-value pair against your schema and emits all the buffered chunks in one burst.
3. Claude starts generating meta. The API buffers.
4. When the entire meta object is complete and validates, another burst.

This is why tool streams feel bursty. You see nothing for the duration of a field, then everything for that field at once. For most applications this is fine — sometimes even desirable — because every chunk you receive is guaranteed to be part of valid JSON.

Fine-grained tool calling: disable the validation

When you need real-time tokens without the buffering — say, to show a user live feedback as Claude builds a long argument, or to start processing a partial result as soon as the first field appears — you can enable fine-grained tool calling. It does one specific thing: disables the API-side JSON validation buffer, so chunks arrive as soon as Claude generates them.

run_conversation(
    messages,
    tools=[save_article_schema],
    fine_grained=True,
)

With fine_grained=True:

• Chunks stream as Claude generates them — character-level granularity.
• Top-level key-value pair buffering is gone.
• The stream feels like a normal text stream, not a series of bursts.
• Your code is responsible for handling invalid JSON because the API will no longer catch it.

The trade-off: invalid JSON is now your problem

With validation disabled, Claude may briefly emit JSON that doesn't parse — "word_count": undefined mid-generation, for example, before it self-corrects to "word_count": 847. The snapshot field will contain that temporarily invalid state.

Wrap your parsing in a try/except:

try:
    parsed_args = json.loads(chunk.snapshot)
    # use parsed_args
except json.JSONDecodeError:
    # snapshot is not yet valid JSON — skip and wait for the next chunk
    continue

For most applications, "wait and retry on the next chunk" is the right strategy. For UI updates, you might parse optimistically and swallow errors; for downstream side effects, wait for the full response before acting.

When to enable fine-grained tool calling

The default behaviour is the right choice for the vast majority of applications. Reach for fine-grained tool calling when one or more of these is true:

• You need to show live token-by-token progress to a user during tool argument generation — e.g. a UI that reveals a generated article as it's being produced.
• You want to start processing partial tool results as quickly as possible — e.g. beginning a downstream computation the moment the first field is available.
• The buffering delays measurably hurt user experience and you've confirmed it's the bursts, not overall latency, that users are complaining about.
• You're comfortable implementing robust JSON error handling because the safety net is gone.

When none of those apply, keep the default. You don't want to spend engineering effort on JSON validation handling unless you're actually getting value from the extra responsiveness.

Summary: two knobs, one trade-off

Streaming tool calls comes with one knob — fine_grained=True — and one trade-off.

| Mode | Behaviour | Invalid JSON risk | Use when | |---|---|---|---| | Default | Buffered until top-level key-value is valid | Handled by API | Most applications | | Fine-grained | Immediate chunk delivery | Your responsibility | Real-time UI or partial processing |

The buffering isn't a bug — it's a thoughtful default that hides a class of edge cases from you. Turning it off gives you back some responsiveness at the cost of taking on those edge cases yourself. Like most production trade-offs in this course: measure first, then decide.