Some bugs swallow an entire workday while every system involved reports success and apparently nothing happens.
Our coding agent replied OK. Quite a few times, in fact. It also produced a tidy Markdown summary of the lines it claimed to have changed, complete, of course, with a section titled "Verification." Meanwhile, git diff showed an untouched file.
We spent almost a day looking for the cause, were wrong six times in six interesting ways, and eventually found a genuine defect in Laguna S 2.1. We could even reproduce it through poolside's own hosted API. A few weeks later, we encountered the same defect in an entirely unrelated model from another vendor.
We covered our Laguna infrastructure in Post #8: TP4 across four GPUs, 262k versus 1M context, and the PP3 scaling bug where the server starts cleanly and then quietly produces nonsense. This article is about one specific, tightly bounded defect in the model itself.
The short version:
Six strings are single tokens in Laguna's vocabulary. Once any of them appears as data in source code, Laguna can neither read nor write it correctly. Sometimes the attempt even makes the model terminate its own tool call halfway through the JSON.
What took us longest to realize was that this is not unique to Laguna. Qwen3.6-35B-A3B has the same defect with the same tags, and our workaround initially missed one critical channel.
The Setup That Failed
We were integrating DeepSeek-V4-Flash into TensorSharp as a full-fledged OpenCode model: chat template, tool history, output parsing, reasoning separation, the whole package. We used a local Laguna instance for the work, with code-laguna implementing and reviewer-laguna reviewing. TP4, 262k context, four GPUs, sharing the same rig with two other models. Nothing unusual so far.
Naturally, the target codebase contained plenty of lines like these:
int closeIdx = content.IndexOf("</think>", StringComparison.Ordinal);
int openIdx = content.LastIndexOf("<think>", closeIdx, StringComparison.Ordinal);
In hindsight, the cause is sitting there in plain sight. It did not look that way to us for several hours.
The symptoms arrived in this order:
reviewer-lagunarepeatedly returned empty responses for code that posed no unusual challenge.code-lagunareported successful changes that had never happened.- In one particularly memorable run, the model corrupted seven string literals in a file, four of them in code completely unrelated to the task.
Throughout all this, the server looked perfectly healthy: Running: 1 reqs, Waiting: 0. KV utilization was 11.6 percent. Prefix cache hit rate was 94.4 percent. No preemption. Every single request ended with 200 OK.
Wrong Turn One: It Must Be the Reasoning Parser
vLLM logs this warning every time it starts with Laguna:
WARNING [vllm.py:1486] Auto-initialization of reasoning token IDs failed.
Please check whether your reasoning parser has implemented the
`reasoning_start_str` and `reasoning_end_str`.
A warning about reasoning tokens, on a model with an obvious <think> problem. This looked like a quick case to solve.
We sent five research agents through the vLLM source in parallel. Working independently, all five returned the same well-supported answer. They had, in fact, found a bug:
# vllm/reasoning/basic_parsers.py
model_output_parts = model_output.partition(self.start_token)
model_output = (
model_output_parts[2] if model_output_parts[1] else model_output_parts[0]
)
str.partition splits at the first occurrence anywhere in the output. parts[0], meaning everything before the match, is never assigned anywhere. That content reaches neither content nor reasoning; it simply disappears.
The reasoning parser also runs over the entire raw model output before the tool-call parser gets a chance to separate the arguments:
# vllm/parser/abstract_parser.py
reasoning, content = self.extract_reasoning(model_output, request)
tool_calls, content = self._extract_tool_calls(content=content, ...)
Any <think> inside a tool argument is therefore swallowed by a structure-blind string search before a component that understands tool calls ever sees it. It was a good find, devastating for the parser and thoroughly documented across five independent reports.
It just did not explain our bug. The parser defect is real nonetheless; we later reported it as vllm#50901. We will return to it at the end of the article.
To their credit, two of the five agents flagged the discrepancy themselves: the predicted damage did not match what we had observed. Under the parser theory, the entire tool call should have been destroyed. Our tool call remained intact while precisely seven characters were cut out and replaced with a single space. And reasoning_content had length zero, even though the theory required it to contain something.
That mismatch finally put us on the right track.
The Measurement That Settled the Case
When asked, vLLM returns the raw token IDs of its output ("return_token_ids": true). It also exposes /tokenize and /detokenize. Together, these let us separate three questions cleanly: What went in, what did the model do, and what came out?
First, the tokens in question:
[18] -> <think> [23] -> <assistant> [25] -> <tool_call>
[19] -> </think> [24] -> </assistant> [26] -> </tool_call>
Six individual tokens. These are not seven characters that happen to resemble a tag. Each string is an atomic vocabulary entry.
Here is how the tokenizer handles ordinary input text:
'X<think>Y' -> [2, 125, 18, 126]
'grep -c "<think>" /tmp/a.cs' -> [2, 21565, 419, 136, 444, 18, 71, 778, 6684, 7935, 11905]
'X< think >Y' -> [2, 125, 97, 1981, 981, 126] <- no token 18
A <think> appearing as ordinary text in the prompt becomes exactly the same token that the chat template uses to open a reasoning block. At the token level, the two are indistinguishable. The generation prompt also ends with <assistant><think> = [..., 23, 18], so the model always begins its output inside an open reasoning block.
We then asked Laguna to generate grep -c "<think>" /tmp/a.cs and compared the actual token sequence with the expected one:
correct : ... 136(c) 444(") 18(<think>) 71(") 778(/tmp) ...
measured: ... 136(c) 444(") 444(") 778(/tmp) ...
Token 18 never appears. Three runs at temperature 0 produced zero hits. Neither skip_special_tokens=false nor enable_thinking=false changed that.
The parser had deleted nothing. The model simply never generated token 18 and chose a second token 444 instead. That is why the command came back as grep -c " " /tmp/a.cs. The mysterious space for which we could find no cause in the vLLM source was not a deletion artifact. Laguna had written it.
Once we looked at how the token is used, the reason became rather mundane. In the training data, token 18 appears almost exclusively in a structural position, emitted by the template rather than carried as data inside a string. There is therefore almost no signal for "generate token 18 as content inside a tool argument." The model reaches for something else.
For one rare moment, the bug was even funny. Asked to repeat the literal string X<think>Y, Laguna wrote in its own reasoning:
Okay, the user wants me to return exactly the string "XY" and nothing else.
The model read token 18 and reproduced it as token 26. Laguna does not merely struggle to write these tokens; it confuses them with one another.
Exhibit A: The Model Hangs Up on Itself
The clearest failure came from an edit tool. Laguna was supposed to generate this line as a tool argument:
int closeIdx = content.IndexOf("</think>", StringComparison.Ordinal);
This is literally what came over the wire:
{"filePath": "/tmp/x.cs", "oldString": " int closeIdx = content.IndexOf(\"</tool_call>
^^^^^^^^^^^^
While trying to write </think>, the model selected </tool_call>. That token terminates the tool call. Generation stops in the middle of a JSON string because, from the harness's perspective, the model has just declared itself finished.
The JSON remains incomplete and cannot be parsed. On our local server, the vLLM parser discards the malformed call altogether. What remains is finish_reason: stop, an empty response, and an agent that shrugs and reports OK.
That explained the whole mystery: a silent failure on a server that looked completely healthy.
Wrong Turn Two: Then It Must Be Our Stack
We now understood the mechanism, but not who was responsible. We were using INT4, a particular vLLM version, and --reasoning-parser poolside_v1. Any of these components could have been the actual source of the problem.
So we ran the same three tests against poolside's hosted model on OpenRouter, once on the free tier and once on the paid tier, and compared both with our local INT4 instance.
| Test | local vLLM INT4 | OpenRouter :free |
OpenRouter paid |
|---|---|---|---|
Echo X<think>Y |
content='' |
content='' |
content='' |
| Literal in a tool argument | grep -c " " |
grep -c " " |
grep -c " " |
| Edit with literal in the payload | no tool call | malformed JSON | malformed JSON |
Test 2 is wrong in exactly the same way, byte for byte, on all three endpoints.
That one table cleared four suspects at once: our vLLM server, our INT4 quant (the hosted endpoints do not use INT4), the poolside_v1 parser (OpenRouter runs a different stack), and OpenCode.
Had we made these three curl calls in the first hour rather than the sixth, we would have saved most of the day. We have followed one rule ever since: When we suspect a model defect, we test the same model through a second provider before dismantling our own stack.
The complete reproduction:
curl -s $ENDPOINT/v1/chat/completions -H 'Content-Type: application/json' -d '{
"model":"'"$MODEL"'",
"messages":[{"role":"user","content":"Call the bash tool with exactly this command: grep -c \"<think>\" /tmp/a.cs"}],
"tools":[{"type":"function","function":{"name":"bash","description":"Run a shell command",
"parameters":{"type":"object","properties":{"command":{"type":"string"}},"required":["command"]}}}],
"tool_choice":"auto","max_tokens":2000,"temperature":0}'
Expected: grep -c "<think>" /tmp/a.cs. Delivered: grep -c " " /tmp/a.cs.
What Does Not Fix It
After our earlier guesses, we measured each proposed fix separately:
| Attempt | Result |
|---|---|
enable_thinking:false per request |
model stops after one character and generates EOS |
reasoning_effort:"none" |
same result |
skip_special_tokens:false |
no effect, because token 18 is never generated in the first place |
--reasoning-config with explicit delimiters |
suppresses the startup warning and changes nothing else |
--reasoning-parser identity |
that name is not registered; the server does not start |
| Upgrade from vLLM v0.25.1 to v0.26.0 | the relevant files are SHA-256-identical |
The last point deserves some explanation. We hashed poolside_v1_reasoning_parser.py, deepseek_v3_, deepseek_r1_, and poolside_v1_tool_parser.py in v0.25.1, v0.26.0, and main. Every file was byte-identical. The Poolside reasoning parser has had exactly one commit in its entire history. Taking the production risk of another 411 commits would have gained us precisely nothing.
The startup warning that sent us down the first wrong path also proved irrelevant to this case. Across the entire vLLM tree, the token IDs derived there have exactly one consumer: the thinking_token_budget sampler. The warning is cosmetic. It has also been reported as issue #49379, on our exact version and with almost identical flags. The issue has been open since July 21 and has zero comments. Three related PRs are also open, unmerged, assigned to no milestone, and without human review. It is a fairly sober picture of long-tail model support in a fast-moving inference engine.
The Fix: We Lie to the Model
The defect lives in the weights, where we cannot repair it. The practical requirement, however, is narrow and mechanical:
The six strings must never enter the context as their structural tokens, and the model must never have to generate them.
That turns a model defect into a transport-encoding problem, for which proven solutions exist. OpenCode provides a plugin API with hooks around tool execution and message handling:
| Hook | Purpose |
|---|---|
chat.params |
tracks which sessions are protected and which marker set they use |
tool.execute.after |
encodes the six strings in tool results before the model sees them |
tool.execute.before |
decodes them in tool arguments before anything is executed |
experimental.chat.messages.transform |
encodes them in user messages before the model sees them |
We added the fourth hook later, after an expensive lesson. More on that under "Wrong Turn Four." The model no longer encounters any structural token and never has to emit one. The file on disk still contains the real bytes. For every other model, the data stream remains unchanged. We explicitly verified that behavior with Claude, GPT, and Qwen, as well as with unknown sessions where the plugin stays out of the way when uncertain.
One distinction matters here because it is easy to misunderstand: This is not a skill. A skill is Markdown that explains something to the model. Our code runs in the harness process and changes data in transit. No instruction could give Laguna an output path for a token it cannot produce as payload.
We checked our choice of sentinel directly against the tokenizer:
<think> -> [18] structural token
%%LAGUNA_THINK_OPEN%% -> [3834, 7836, 2683, ...] ordinary BPE
Wrong Turn Three: The Elegant Sentinel
Our first sentinel was a pretty one: a zero-width space before the closing angle bracket, yielding <think\u200B>. Invisible to humans, still recognizable to the model as a tag, and no longer a match for the tokenizer's atomic entry. Measurement confirmed that token 18 was absent.
We took a refactoring that had previously failed deterministically and ran it four times with the sentinel in place. Three runs were flawless. In the fourth, Laguna wrote:
private const string ThinkOpen = "<think\u200B>";
The model had "helpfully" normalized the invisible character into a C# escape sequence. The code compiles, but creates an eight-character string where seven characters are required. Every IndexOf silently misses its target.
A silent semantic error is worse than a loud crash. We discarded the elegant sentinel and switched to the ugly ASCII version. It contains no invisible character that could be normalized, and it does not invite the model to tidy it up.
The decoder is deliberately more tolerant than the encoder because we had measured how Laguna damages long tokens:
%%LAGUNA_THINK_OPEN%% -> <think> canonical
%%laguna_think_open%% -> <think> lowercased
%LAGUNA_THINK_OPEN%% -> <think> dropped character
50%% off and 100% sure -> unchanged
Four new runs of the same refactoring produced these results:
| without plugin | ZWSP sentinel | ASCII sentinel | |
|---|---|---|---|
| structural literals correct | 0 of 2 | 3 of 4 | 4 of 4 |
| sentinel reaches file | – | 1 of 4 | 0 of 4 |
Wrong Turn Four: The Plugin Was Not Enough
A few weeks later, the same defect surfaced in a completely different model, Qwen3.6-35B-A3B. Extending the plugin to cover it looked like a five-minute job. It was not.
The tokenizer story was the same: <think> and </think> are individual vocabulary entries that the model cannot emit as payload. It took us three attempts to set up the experiment correctly. We selected four excerpts of roughly 420 bytes each from the same production file. This kept the payloads comparable while changing only which tags they contained:
| Excerpt | local llama.cpp Q4 | Parasail vLLM FP8 |
|---|---|---|
| no structural tag (580 B) | 3/3 exact | 3/3 exact |
<think> only |
3/3 exact | 0/3 |
</think> only |
0/3 | 0/3 |
| both | 0/3 | 0/3 |
The control excerpt passes on both stacks and is also the largest of the four. The tag, not the length, causes the failure. We had incorrectly blamed payload size in two earlier experiments: the small excerpt happened to contain only the opening tag, while the larger ones added the closing tag. Our setup had confounded size with tag content.
We enabled the plugin, watched the transcription tests jump from 0/5 to 5/5, and pointed a real agent at a real file. It changed nothing. Twice. On the third attempt, it generated 142 kB of soliloquy and froze the editor.
The placeholders were the obvious suspect. The agent even talked about them:
The terminal is rendering it as an XML tag and hiding it, which is why the text display shows
%%SHIELD_THINK_OPEN%%
So we added a system prompt: These are placeholders; treat them like the tag, write them back unchanged, and do not investigate them further. The explanation sounded plausible. The measurements disagreed:
| Task wording | without hint | with hint |
|---|---|---|
| refers to existing code | 3/3 | 3/3 |
| spells out the marker | 1/3 | 0/3 plus Doom Loop |
The hint never helped and made one formulation substantially worse. We removed it.
The missing channel was the prompt. Our own task text contained the raw </think>. The plugin encoded tool results and tool arguments, but not user messages. We had asked the model to search for a string it was fundamentally unable to perceive, then watched it search for twenty thousand tokens.
| correct | Doom Loops | output | |
|---|---|---|---|
| raw marker in prompt | 1/3 | 2 | 20–25 kB |
| encoded marker in prompt | 3/3 | 0 | 6215 B, deterministic |
Another hook fixed it: the same task that had produced two empty diffs and one frozen editor now delivered exactly the requested change. experimental.chat.messages.transform receives an empty input object along the way, so the session has to be reconstructed from the message payload.
Two controls made the problem diagnosable. The same model correctly handled a comparable task in a file without markers; otherwise, it would simply have looked generally incompetent. We also checked git diff instead of trusting the agent's report. The report claimed it had written %%SHIELD_THINK_CLOSE%% to the file, but the file actually contained the real tag.
Our transparency assumption was that no calling agent should need to know about the encoding. That held for simple pass-through. The moment the agent had to reason about the meaning of a literal, the assumption failed. Until then, our end-to-end test had covered pass-through and nothing else.
Two More Strings, Found by Asking the Obvious Question
Once we understood the mechanism, an obvious question followed: Which other strings in this vocabulary are single tokens? For Qwen, one call to /tokenize was enough:
<|im_start|> -> [248045] single token
<|im_end|> -> [248046] single token
<|assistant|> -> [27, 91, 74455, 91, 29] ordinary text, fine
Two hits, both found in exactly the sort of code a coding agent would work on: chat-template rendering. They were missing from the plugin's list.
We repeated the previous experiment with four similarly sized excerpts from a real file. The control mattered especially this time because it was not flawless either:
| Excerpt | exact | marker intact |
|---|---|---|
| no marker (control) | 1/3 | 3/3 |
<|im_start|> only |
0/3 | 0/3 |
<|im_end|> only |
0/3 | 0/3 |
| both | 0/3 | 0/3 |
The control excerpt drops a line by itself in two out of three runs. Without that control, we would have blamed the tag for damage this file provokes anyway. The signal is in the last column: 0 out of 9 without the shield, 9 out of 9 with it.
The two failure modes differ, and that distinction matters:
<|im_start|> -> " :" substituted. Compiles fine. Silently breaks the
prompt format, which no build will catch.
<|im_end|> -> the output simply stops at the marker, finish_reason=stop
The first case is nastier. sb.Append(" :system\n...") is valid C#, but produces the wrong prompt. The same class of silent failure from the start of our investigation had found another way in.
We drew one deliberate line in the implementation. These two tokens are encoded for Qwen only. Laguna's endpoint was unreachable at the time, so we have no measurement for that model. Encoding also has a cost: it changes the tokenization of the input. We observed two failures per ten runs in our measurements. The marker set therefore applies per model family. The decoder, by contrast, accepts every variant, because a sentinel that reaches a file must remain decodable if a different model encounters it later.
Bonus Round: A Second Defect and a Premature Rebuttal
While measuring the first problem, we found a second, independent defect that no clever encoding can fix. Laguna drops characters during verbatim transcription, and the effect becomes much stronger as the payload grows. We measured it on real C#, with two runs per payload and character-level differences counted as errors:
| Payload | Errors |
|---|---|
| 388 B | 2 |
| 600 B | 3 |
| 1003 B | 3 |
| 1656 B | 27 |
| 2402 B | 27 |
Typical outputs included IndexOf -> Indexof, StringComparison -> Stringcomparison, IsNullOrEmpty -> IsNullorEmpty, and LastIndexOf -> LastIndexIf.
There is a clear jump between 1000 and 1650 bytes.
Now for the embarrassing part. Our existing rule of thumb was: "Keep Laguna edit payloads below roughly 1.5 kB." Midway through the investigation, we declared that rule disproved because a synthetic 4000-byte test block produced no errors at all.
The synthetic block consisted of one hundred nearly identical lines. It was trivial for a language model, and the test measured practically nothing. The old rule was right, our rebuttal was worthless, and we put the rule back in place.
A measurement is only as good as its hardest case. If the test data is easier than production, a green result means nothing at all.
At least the second defect has one pleasant property: LastIndexIf does not compile. It fails loudly. Compared with the first defect, where nothing happened and the agent congratulated itself afterward, that is substantial progress.
What This Means for Running Laguna
From the calling system's perspective, the plugin makes the first defect disappear. That was the point. A model that forced every calling agent to know three special rules would, quite reasonably, see little use.
What remains is a short set of practical operating rules:
git diffis the referee, never the agent's report. In successful runs, reports were sometimes empty or fragmentary even when the work was flawless. The reverse also happens. The agent's self-assessment is reliable in neither direction.- We run the build. The transcription defect produces code that does not compile, but only someone actually compiling it will find out.
- We keep edit payloads small. One method per task, not an entire class.
- We never put raw markers into a prompt ourselves. Inside the harness, the plugin handles them. Outside it, for example in a
curlreproduction or a manually written task description, there is no protection. This mistake cost us more time than anything else in the investigation.
For context on how often this matters in practice: Of 57 code-laguna tasks in the affected project, 9 made no change at all. That is sixteen percent, with no visible error. The codebase happened to process chat protocols, making it the worst possible case. In ordinary application code, the rate is effectively zero because </tool_call> does not normally appear as a string literal there.
It would be easy to conclude from all this that Laguna is broken and should not be used. Our experience says otherwise. The model is exceptionally good at difficult debugging work; that has not changed since Post #8. It has one narrow, unpleasant defect that detonates spectacularly in a particular kind of codebase. We hit it so reliably because our target code was a chat-template parser.
Postscript: Six Wrong Turns and a Method
Our six incorrect explanations were:
- The vLLM reasoning parser caused the failure. Five agents documented that case thoroughly and were still wrong.
- Our own serving stack was suspect. Three
curlcalls would have cleared it. - The zero-width sentinel was an elegant solution. Briefly, yes, until it silently corrupted a constant.
- The 1.5 kB rule had been disproved. Our rebuttal used data the model could barely fail on.
- Payload size explained the failure in the second model. Our experiment had confounded size with tag content.
- The placeholders in the file confused the agent. The raw marker was actually in our own prompt, and the supposed system-prompt fix made one case worse.
Every explanation was plausible. Every one rested on evidence that was too thin, came from a test that was too easy, or was taken from the wrong layer. The findings looked convincing, which is precisely why the explanations survived for a while.
Each time, a measurement along the real path settled the matter: /tokenize instead of interpreting source code, a second provider instead of more local troubleshooting, real code instead of synthetic data, and an actual restart instead of an import that bypassed the loader.
We knew all of that already. That is the irritating part.
Both Defects Have Now Been Reported
Neither problem in this article had been documented anywhere before. We therefore reported both.
The model defect: poolside/Laguna-S-2.1, discussion #35. The curl command from "Wrong Turn Two" is the complete bug report. The defect probably also explains #29, where someone using OpenCode encountered the same empty response without being able to see what had happened.
The vLLM parser defect: vllm-project/vllm#50901. This is the real but, in our specific case, innocent bug from "Wrong Turn One." Reporting it required further research, which produced the nicest detail of the entire investigation: minimax_m3 already overrides extract_reasoning and preserves the discarded prefix correctly. The solution exists in the same repository. It simply never made it into the base class inherited by deepseek_r1.
Our expectations for both reports were realistic. poolside does not appear to respond to Hugging Face discussions; in #15, an employee was tagged by name and never replied. The closest vLLM issue to ours, #49379, had been open since July 21 without a single comment. The reports were still worth filing. At least the next person to encounter the defect may be able to discover what is happening. We did not have that advantage.
Update. The vLLM issue saw activity within an hour. Another contributor confirmed the discarded prefix by inspecting the source and proposed exactly the narrow scope we had recommended. A third opened PR #50918: content_before is preserved and prepended to content, with two tests that fail on the old code.
One detail from that exchange stayed with us. The contributor who confirmed the defect also pointed out a downstream interaction. Step3p5ReasoningParser processes the base class's return value with content.removeprefix("\n"). Once the fix lands, that line strips the restored prefix rather than the newline after the closing tag. The same contributor then approved the PR without checking whether this point had been addressed. It remained open in #50918. The loss is small, but it occurs at the exact position the PR restores. The existing tests cannot detect it because it becomes visible only after the prefix is preserved.
Status as of August 17, 2026. We reproduced this interaction at code level and reported issue #51164. Contributor khushali9 implemented the fix with regression tests in PR #51201, and a reviewer approved it. A vLLM collaborator later requested a reproduction on a running server with the affected Step3.5 model. The problem remains latent on main: it becomes observable only after #50918 starts preserving the prefix. An end-to-end demonstration therefore has to apply both changes and compare #50918 alone against #50918 plus #51201.
Both fixes remain unmerged. PR #50918 is waiting for code-owner review and is also blocked by two commits without DCO sign-off. PR #51201 has a clean DCO status but still awaits the maintainers' decision on the requested live reproduction. No question remains open for us; the remaining work lies with the PR authors and maintainers.
Meanwhile, there has been no response to the model defect. That matched our expectations.
A postscript to the postscript: Before reporting the removeprefix observation, we had to verify it ourselves. Passing along someone else's unconfirmed theory was, after all, the method behind the first four wrong turns in this article.
The Laguna figures were measured on a rig with 8× RTX 3090: Laguna S 2.1 INT4 under vLLM 0.25.1 with TP4 and 262k context, cross-checked against poolside-hosted endpoints on both the free and paid tiers. The Qwen figures came from a budget rig with 2× RTX 3060: Qwen3.6-35B-A3B-UD-IQ4_XS under llama.cpp b10143, cross-checked against DeepInfra and Parasail FP8.