Fixing interleaved thinking for local Claude Code

There’s a thread on r/LocalLLaMA where someone builds a pet project with Qwen3.5 35B A3B running inside Claude Code, tallies up what the session would have cost on Sonnet 4.6 ($10.85, per ccusage), and lands on their actual bill: two minutes of 400W electricity. That’s the promise of pointing Claude Code at your own hardware, and it mostly delivers.

Mostly. Because the same thread carries a recurring complaint: the model just stops. It works for a while, makes a few tool calls, then goes silent right where the next tool call should be. No error. The turn ends at the tail of a thinking block, and Claude Code sits there waiting.

The workaround people converged on was to disable thinking. It works. It’s also the wrong lesson, and the actual bug sat in llama.cpp’s Anthropic compatibility layer, where I eventually found and fixed it.

The symptom

I was running the model in Claude Code, testing how far its agentic coding could go in a frontier harness. My first suspicion was the chat template: newly released models ship with template bugs often enough that it’s the reasonable first guess. But the same model drove OpenClaw fine. Same weights, same server, different harness. So I opened a Claude Code session with Opus, handed it my llama.cpp container, and it had the cause in about fifteen minutes.

The model stopped abruptly before emitting a tool call, at the end of a thinking block. An <|im_end|> where a tool_use should be, with empty content. I hit it on both Qwen 3.5 models I run; nothing about it is Qwen-specific, and without the fix it would most likely have hit Qwen 3.6 and every other interleaved-thinking model the same way.

What the server was actually doing

With thinking enabled, an agent’s single logical turn is a chain: thinking → tool_use → thinking → tool_use, all the way to the final answer. That’s interleaved thinking, and it’s how the Anthropic API expects agentic loops to run — Claude Code sends the whole chain back to the server on every request so the model can see its own reasoning trail.

llama.cpp’s /v1/messages endpoint speaks the Anthropic Messages format on the wire — that’s what Claude Code sends — but the rest of the server thinks in OpenAI-style messages, so the endpoint’s real job is translation. The mapping is mostly mechanical: a thinking block should land in reasoning_content, a tool_use block becomes an entry in tool_calls, and a tool_result comes back as a role: "tool" message. Except the first one didn’t happen. The conversion dropped the thinking blocks. Every one of them:

So here’s what the model saw: a transcript where its previous tool calls in the current turn appear to have happened with no thinking lead-in, because it was deleted. The conversion rewrites the chat history so the model believes it never emitted thinking. Then, when it’s time for the next tool call, the model begins a fresh thinking block, trying to re-establish its plan. But the prompt now teaches it that tool calls happen without the reasoning scaffolding. That mismatch shifts the distribution: instead of transitioning from thinking into a tool_use, it often terminates the message right there. EOS.

From the user’s perspective that reads as “Claude Code stalls after tool calls.” The mechanical cause is that dropping interleaved thinking blocks breaks the learned cadence that leads into the next tool call, so the model ends its turn prematurely.

This is also why disabling thinking “fixes” it: with no thinking blocks to drop, the tool-call sequence stays internally consistent, and the model keeps emitting tool calls instead of bailing out. You’ve traded the model’s reasoning for its stability, and you shouldn’t have to.

There’s a template subtlety underneath that confused the diagnosis further. Qwen’s chat template doesn’t keep all historical thinking: it strips <think> blocks from older turns while keeping them for the current tool-use chain. That’s correct behavior: the model was trained that way. The bug was upstream of the template: the endpoint stripped thinking everywhere, so the template never even got the blocks it needed for the active chain.

The fix

PR #20120: in convert_anthropic_to_oai(), collect thinking blocks into reasoning_content on the converted message instead of discarding them, and stop dropping assistant messages whose only content was thinking. The diff is 8 lines added, 2 removed. The tests are 133 lines.

It merged fast, and another user independently confirmed it also resolved the autoparser 500 errors they’d been seeing. Those 133 lines only exist because a maintainer reminded me to write tests. I shouldn’t have needed reminding: the whole point of a fix this small is pinning the behavior down so it can’t quietly regress.

If your local agent “randomly” stalls

Check what your serving layer does to the conversation before the model sees it. The failure mode generalizes: any proxy or compatibility layer that silently edits chat history is gaslighting the model — showing it a version of its own behavior that never happened, then acting surprised when its next-token distribution goes somewhere strange. Reasoning models make this worse, because the part most likely to be “cleaned up” in conversion is exactly the part that carries the model’s plan.

Model releases always come with quirks like this one, and every one of them makes local models look weaker than they are. The only way they get fixed is the people actually running this stuff sending the patch upstream. Fix a few, and you find out the local models were far more capable than they looked at first sight.