Lenient Tool-Call Parsing: Bridging Weak-Tool-Call Models to Native Tool Catalogs

1. Introduction

The boundary between a model’s emission and the runtime’s dispatch layer is the highest-friction surface in any agent platform. The dispatcher needs structured argument trees; the model produces a token stream. Most production runtimes settle on the strictest available bridge: validate against JSON Schema, reject on failure, surface the error, retry next turn. This is the default for OpenAI function calling (OpenAI, 2023), Anthropic tool use (Anthropic, 2024), and the LangChain function-call wrapper (Chase, 2022).

Strict rejection works because frontier models are post-trained against a single canonical emission shape; validation failure rates against properly-described schemas are small enough that rejection cost is dominated by post-training convergence — already paid by the upstream lab.

The picture inverts for the open-weight tier. Kimi K2 (Moonshot AI, 2024) emits tool calls in a model-specific <|tool_calls_section_begin|> envelope. DeepSeek V3 (DeepSeek, 2024) wraps JSON in <tool_call> text sentinels. Llama 3.x (Meta, 2024) and Qwen 2.x (Alibaba, 2024) mix native-shaped JSON with XML invoke blocks depending on system-prompt phrasing. Mistral variants drift across releases. Worse, several occasionally role-play the post-tool-call protocol: the assistant turn emits a malformed call and a fabricated <function_results> block as if a tool had returned data, then reasons on top of the fabrication.

A strict-reject runtime presented with this output has two failure modes. Rejection-then-retry converges slowly or not at all because the fine-tuned distribution reproduces the same malformed shape on every retry. Leaking the malformed envelope to the user as visible content corrupts the transcript and — in the hallucinated-results case — poisons the next turn, because the model treats its own fabrication as ground truth.

Memagen™‘s lenient parser bridges the gap. It is a streaming state machine that recognises four production-observed emission patterns, normalises them to a canonical tool_call_delta shape, applies per-tool argument coercion to non-canonical argument shapes, fuzzy-resolves paraphrased tool names against the declared catalog, and detects-and-suppresses hallucinated <function_results> blocks before they propagate. The parser is the subsystem that makes BYOLLM — bring your own LLM — economically defensible: a customer can deploy Memagen™ against open-weight models on commodity hardware without sacrificing tool reliability. The implementation ships under MIT in the Lite distribution.

This paper is a specialty companion to the Memagen™ master paper (whose §4 introduces the parser at the capability level) and to the LLM router paper (which describes the splice point at which the parser wraps a provider stream). Sections below cover the detection algorithm (§4), per-tool argument coercion (§5), the catalog gate and fuzzy resolver (§6), hallucinated-results detection (§7), implementation notes (§8), and what we have and have not measured (§9).

2. Related work

2.1 Function-calling specifications

The OpenAI function-calling spec (OpenAI, 2023) is the first widely-adopted contract for LLM-driven tool invocation: tools are described by name, JSON-Schema arguments, and a natural-language description; the assistant message carries a tool_calls array; the runtime executes and replies with role=tool messages keyed by tool_call_id. Anthropic native tool use (Anthropic, 2024) follows the same conceptual shape but emits structured tool_use content blocks rather than a flat array. The Gemini function-calling format (Google, 2024) is a third variant. The MCP specification (Anthropic, 2024) standardises the server side of this contract — tool advertisement and dispatch — and defers the client side to the host runtime.

2.2 JSON Schema validation

The dominant validator stack is JSON Schema Draft 7 (Wright et al., 2019). Production agent runtimes typically wire a validator (pydantic, jsonschema, ajv) into the dispatch path: validation passes, dispatch proceeds; validation fails, the error is surfaced and the model retries on the next turn. The contract is clean. The cost is brittleness against weak-tool-call models.

2.3 Parser-tolerance work

Several lines of work address tolerance in structured-output parsers. Constrained decoding (Beurer-Kellner et al., 2023; Willard & Louf, 2023) constrains emission at decode time to remain within a CFG, which removes the malformed-output problem at its source — but requires control over the inference engine, which BYOLLM customers running NIM, vLLM, or remote OpenRouter endpoints generally do not have. Best-effort JSON parsers (json5, dirty-json) recover from common syntactic errors (trailing commas, unquoted keys, single quotes) but do not address the higher-level shape mismatches of §5. The closest prior work is the community-contributed function-calling-format translation layer for open-weight models, which translates one or two emission shapes; we extend coverage to four observed shapes plus the hallucinated-results case.

2.4 Why strict-reject became default

Strict rejection has three justifications. Training-time alignment: a canonical emission shape is a measurable target, and the post-training community has converged on rejection-during-RLHF as the convergence signal (Ouyang et al., 2022). Observability: a strict reject produces a clean structured error suitable for dashboards. Conservatism: a malformed call parsed leniently might dispatch the wrong action. Memagen™ accepts the first two and answers the third by composing the lenient parser with the V3-P capability vault (master paper §5) — lenient parsing never bypasses capability enforcement.

3. The four production-observed formats

We document the four emission patterns the parser handles, each with a representative example string from production observations during the V3-Q development wave.

3.1 Format A — native

OpenAI-style function-calling JSON, Anthropic-style tool_use blocks, and Gemini-style function calls all arrive on the structured side of the streaming protocol — i.e., not in delta.content. The router-level event protocol carries them as ("tool_call_delta", ...) events directly. The parser passes these through untouched. Example (OpenAI shape, abbreviated):

{"id": "call_abc123", "type": "function",
 "function": {"name": "widget_create",
              "arguments": "{\"name\":\"hello\",\"spec\":{\"kind\":\"markdown\"}}"}}

The wrap() generator recognises that non-content events bypass the state machine entirely:

async for kind, chunk in stream:
    if kind != "content":
        yield (kind, chunk)
        continue

This is the no-op path for frontier models and the reason the parser introduces zero overhead on canonical providers.

3.2 Format B — XML invoke

<invoke> blocks within prose, wrapped in a <function_calls> envelope. Common output for Llama 3.x and Qwen variants when prompted with XML-flavoured system prompts.

<function_calls>
<invoke name="widget_create">
<parameter name="name">hello</parameter>
<parameter name="spec">{"kind":"markdown","props":{"text":"Hi"}}</parameter>
</invoke>
</function_calls>

The parser recognises the <function_calls> opener, accumulates until the matching closer, and walks the section via xml.etree.ElementTree to extract each <invoke> and its <parameter> children. Parameter values stay strings — XML carries no richer type — and the per-tool argument coercer (§5) handles type recovery downstream.

3.3 Format C — JSON in sentinel

A JSON object wrapped in <tool_call>...</tool_call> (or <function_call>...</function_call>) text sentinels. This is the format Memagen™‘s system-prompt hint (build_tool_hint()) recommends for weak-tool-call models, because it is the format weakest models converge to most reliably under our hint:

<tool_call>{"name": "widget_create",
            "arguments": {"name": "hello",
                          "spec": {"kind": "markdown"}}}</tool_call>

The parser scans for <tool_call> openers, accumulates to the matching closer, and parses the body as JSON. Multiple sequential blocks in a single chunk are flattened into one tool_call_delta list, matching the way the native streaming protocol batches sequential calls.

3.4 Format D — special-token envelope

Models in the Moonshot family emit tool calls in a model-specific bracketed envelope using literal special tokens (the tokenizer emits these as single-token IDs; they appear in delta.content as text):

<|tool_calls_section_begin|>
<|tool_call_begin|>functions.widget_create:0<|tool_call_argument_begin|>{"name":"hello","spec":{"kind":"markdown"}}<|tool_call_end|>
<|tool_calls_section_end|>

The parser recognises <|tool_calls_section_begin|> as an opener, accumulates to the matching <|tool_calls_section_end|> closer, and runs a regex walk over each <|tool_call_begin|>...<|tool_call_end|> sub-block. The functions. prefix on the tool name (mirrored from the OpenAI convention) is stripped. Arguments are JSON-validated; on parse failure, the raw bytes are wrapped in {"_raw": ...} so the dispatcher surfaces the failure rather than silently dropping the call.

The four format detection routines dispatch from a single registry:

_OPENERS: list[tuple[str, str, str]] = [
    (_OPEN_D, _CLOSE_D, "kimi_special"),
    (_OPEN_B, _CLOSE_B, "xml_invoke"),
    (_OPEN_C, _CLOSE_C, "xml_sentinel"),
    (_OPEN_HALLUC, _CLOSE_HALLUC, "hallucinated_results"),
]

The fourth entry is the hallucinated-results sentinel (§7) — a suppressed format that participates in the same state machine but never produces a tool_call_delta.

3.5 Architecture: the parser state machine

The parser is one streaming state machine with two states (PASSTHROUGH, BUFFERING) and four format-specific (opener, closer, fmt-tag) triples. Format A bypasses the machine entirely; formats B/C/D produce synthetic tool_call_delta events; the hallucinated-results format produces a hallucinated_results_detected event and triggers a one-shot retry directive on the next round.

stateDiagram-v2
    [*] --> PASSTHROUGH

    PASSTHROUGH --> PASSTHROUGH: content chunk\n(no opener seen)
    PASSTHROUGH --> PASSTHROUGH: tool_call_delta\n(Format A passthrough)

    PASSTHROUGH --> BUFFERING_B: <function_calls>
    PASSTHROUGH --> BUFFERING_C: <tool_call>
    PASSTHROUGH --> BUFFERING_D: <|tool_calls_section_begin|>
    PASSTHROUGH --> BUFFERING_HALLUC: <function_results>

    BUFFERING_B --> EMIT_B: </function_calls>
    BUFFERING_C --> EMIT_C: </tool_call>
    BUFFERING_D --> EMIT_D: <|tool_calls_section_end|>
    BUFFERING_HALLUC --> SUPPRESS: </function_results>

    EMIT_B --> PASSTHROUGH: yield tool_call_delta\n(fmt=xml_invoke)
    EMIT_C --> PASSTHROUGH: yield tool_call_delta\n(fmt=xml_sentinel)
    EMIT_D --> PASSTHROUGH: yield tool_call_delta\n(fmt=kimi_special)

    SUPPRESS --> PASSTHROUGH: yield\nhallucinated_results_detected\n(drop block, set retry directive)

    note right of SUPPRESS
      Retry loop:
      halluc_retries < 2
      => augment next-round
         system prompt
      Otherwise: cap retries,
      keep suppressing.
    end note

The diagram is normative: every code path in core/llm/lenient_parser.py maps to exactly one labelled transition above.

4. Detection algorithm

The detection logic lives in a streaming state machine implemented as the wrap() async generator in core/llm/lenient_parser.py. The machine has two macro-states:

PASSTHROUGH --opening sentinel-->  BUFFERING
BUFFERING   --closing sentinel-->  PASSTHROUGH (emit synthetic delta)

In PASSTHROUGH, the parser yields content chunks to the consumer. In BUFFERING, it accumulates bytes silently until a closer is found.

4.1 Opener detection

Per chunk, the parser searches for the earliest opening sentinel via _find_first_opener():

def _find_first_opener(buf: str) -> tuple[int, str, str, str] | None:
    best: tuple[int, str, str, str] | None = None
    for opener, closer, fmt in _OPENERS:
        p = buf.find(opener)
        if p == -1:
            continue
        if best is None or p < best[0]:
            best = (p, opener, closer, fmt)
    return best

Ties resolve by document order — whichever sentinel actually starts first wins. All four sentinels have distinct opener strings, and a model emitting a mixed-format response (rare but observed) signals format intent through which sentinel it emits first.

4.2 Lookahead buffer for split-frame openers

Streaming providers do not align chunk boundaries with sentinel boundaries. A model might emit <|tool_calls in one chunk and _section_begin|> in the next; yielding the first chunk verbatim would render a corrupt prefix to the user. The defence is a lookahead buffer: in PASSTHROUGH, after no full opener is found, the parser checks whether any suffix of the working buffer is a strict proper prefix of any opener:

def _suffix_could_open(buf: str) -> int:
    best = 0
    n = len(buf)
    for opener in _ALL_OPENERS:
        max_check = min(len(opener) - 1, n)
        for k in range(max_check, best, -1):
            if buf.endswith(opener[:k]):
                best = k
                break
    return best

The matching suffix is held back in lookahead and prepended to the next chunk. The held-back tail never exceeds len(longest_opener) - 1 bytes, bounding the latency cost to one character less than the longest sentinel.

4.3 Prefix-collision defence: <function_calls> vs <function_results>

The subtlest collision is between Format B’s <function_calls> (a real opener) and the hallucinated-results sentinel <function_results> (a suppressed opener we want to detect, not parse as Format B). Both share <function_. The parser registers each as a distinct entry in _OPENERS and relies on _find_first_opener()’s literal substring search on the full opener — the two strings differ at the byte level, so the search disambiguates without prefix-tree work.

4.4 Closer detection and tail handling

In BUFFERING, the parser searches for the format-specific closer via candidate.find(active_close). If absent, the candidate is stashed in xml_buf and the next chunk is awaited. If found, the section is parsed (or, for hallucinated-results, suppressed), state returns to PASSTHROUGH, and the post-closer tail is processed in the same loop iteration — handling two sentinel sections in a single chunk.

4.5 Stream-end flush

When the upstream stream is exhausted with the parser still in BUFFERING, the parser attempts graceful degradation: parse what is in xml_buf and emit any deltas recovered. If the parse yields nothing, the raw buffer is surfaced as content so the user at least sees what the model said rather than silently losing it. The hallucinated-results case is special: an incomplete hallucinated block at stream end is treated as a detection (we saw the opener, even though we never saw the closer) and the partial content is suppressed rather than leaked.

5. Per-tool argument coercion (V3-Q.1)

The parser produces a tool call with a name and an argument string. The next problem is that weak-tool-call models routinely emit recognizable but non-canonical argument shapes — well-formed JSON, correct tool name, but an argument tree with the wrong key names or the wrong nesting level. Three patterns dominate live observations against open-weight models running on NIM:

• Key alias: type → kind (English-natural synonym for an enum field).
• Shape flatten/nest: {"text": "..."} → {"props": {"text": "..."}} (omitted wrapper layer the canonical schema requires).
• Cross-turn ID resolution: a name passed where the canonical schema requires the integer ID returned by a previous tool call’s result.

A schema-driven coercer — one that reads JSON Schema and applies generic key-rename rules — is appealing but brittle: rules generalise poorly across tools, and the schema does not encode cross-turn semantics. Memagen™‘s coercer is registry-based: each tool that needs coercion registers a per-tool function in core/agent/arg_coercion.py:

ToolCoercer = Callable[[dict, list], dict]

COERCERS: dict[str, ToolCoercer] = {
    "widget_create": _coerce_widget_create,
    "workspace_create": _coerce_workspace_create,
    "workspace_apply": _coerce_workspace_apply,
}

The history argument grants the coercer access to prior tool results so it can resolve cross-turn references. The coerce_args dispatcher is best-effort: tools without entries pass through unchanged, and a coercer that raises returns the original loose args — coercion never blocks dispatch.

A representative coercer is _coerce_widget_create:

def _coerce_widget_create(args: dict, history: list) -> dict:
    out = dict(args)

    # Step 1: type -> kind rename (top-level alias)
    if "type" in out and "kind" not in out:
        out["kind"] = out.pop("type")

    # Step 2: any pre-existing 'spec' is canonical; harmonise + return early
    if isinstance(out.get("spec"), dict):
        spec = dict(out["spec"])
        if "kind" not in spec and "kind" in out:
            spec["kind"] = out.pop("kind")
        out["spec"] = spec
    else:
        # Step 3: build spec from flat top-level shape.
        spec_keys = {"kind", "props", "data",
                     "observable_node_id", "slot", "description"}
        spec_payload: dict = {}
        carry_top: dict = {}
        for k, v in list(out.items()):
            if k == "name":
                continue
            if k in spec_keys:
                spec_payload[k] = v
            else:
                carry_top[k] = v
        if carry_top:
            existing_props = (
                spec_payload.get("props")
                if isinstance(spec_payload.get("props"), dict)
                else {}
            )
            spec_payload["props"] = {**existing_props, **carry_top}
        out = {"name": out.get("name"), "spec": spec_payload}

    # Step 4: synthesize a name when missing
    if not out.get("name"):
        # ... derive from kind + first prose hint in props ...
        pass

    return out

The coercer reshapes a flat {type, text} into the canonical {name, spec: {kind, props: {text}}} shape required by WIDGET_CREATE_SCHEMA. The name field is synthesized when missing (the canonical schema requires it); synthesis derives a slug from the widget kind plus a fragment of any human-readable prose field (text, title, label).

Cross-turn resolution is illustrated by _coerce_workspace_create, which converts {widgets: [{loose object}, ...]} into {widget_ids: [int, ...]} by walking the conversation history for prior widget_create results:

def _coerce_workspace_create(args: dict, history: list) -> dict:
    out = dict(args)
    if "widgets" in out and "widget_ids" not in out:
        widgets = out.pop("widgets")
        widget_ids: list[int] = []
        if isinstance(widgets, list):
            for w in widgets:
                if isinstance(w, int):
                    widget_ids.append(w)
                elif isinstance(w, dict):
                    if isinstance(w.get("id"), int):
                        widget_ids.append(w["id"])
                    else:
                        resolved = _resolve_widget_id_from_history(w, history)
                        if resolved is not None:
                            widget_ids.append(resolved)
        if not widget_ids:
            widget_ids = _all_recent_widget_ids_from_history(history)
        out["widget_ids"] = widget_ids
    return out

The history scan handles both the native shape (role=tool messages keyed by tool_call_id) and the lenient shape (role=user messages containing <function_results> blocks parsed via _RESULT_BLOCK_RE). The branching is in _scan_recent_tool_results(). This duality matters because the lenient parser, when normalising tool results back into history, formats them as <function_results> blocks rather than role=tool messages — see format_tool_result() in §8.

Registry-based coercion is deliberate. Schema-driven rules would have to encode equivalence classes (type ≡ kind, text ≡ props.text, name → ID via history) declaratively, and we have found no declarative form that generalises across the tool surface without producing wrong coercions on unfamiliar tools. Registry-based coercion is more code per tool but correct by construction: a missing coercer passes through. The cost is per-tool work; the benefit is that coercers cannot misfire on tools they do not know about.

6. Catalog gate and fuzzy resolver (V3-Q.2)

Even after format detection and argument coercion, weak-tool-call models will sometimes produce a tool call whose name does not match any catalog entry. Two patterns dominate:

• Hallucinated tool: the model emits graph_write when only widget_create is offered. The model has invented a name from a fine-tuning prior or from a different system’s vocabulary.
• Paraphrased tool: the model emits create_markdown_widget when the catalog entry is widget_create. The tool concept is correct; the string has drifted because the model is paraphrasing from the description.

Memagen™ handles these with two complementary mechanisms.

6.1 Fuzzy resolver

Inside wrap(), before a tool_call_delta event is yielded, the parser runs each detected name through _fuzzy_resolve_tool_name() against the declared catalog. The resolver computes four signals per candidate and takes the maximum:

• Substring containment (either direction): score 0.9 if emitted in candidate or candidate in emitted. Catches widget_create against agent_widget_create.
• Token-set subset: score 0.85 if either token set (split on _) is a subset of the other. Catches create_markdown_widget against widget_create — every catalog-name token is present plus a paraphrase noise token (markdown).
• Token-set Jaccard: |∩| / |∪| over the underscore-split token sets. Catches partial overlaps.
• difflib.SequenceMatcher ratio: stdlib Levenshtein-style ratio (Ratcliff & Obershelp, 1988). Catches typos and minor spelling drift.

If the best score across all candidates is ≥ 0.7 (the _FUZZY_THRESHOLD), the canonical name is substituted; otherwise the emitted name is preserved verbatim and is rejected by the catalog gate downstream. The resolver logs every rewrite at INFO level for observability.

6.2 Catalog gate

After the parser produces tool calls and after argument coercion runs, the agent loop applies a strict catalog gate when self.tool_filter is set:

if self.tool_filter is not None and tool_calls:
    catalog_names = {t.get("name", "") for t in self.tool_filter}
    allowed = []
    rejected = []
    for tc in tool_calls:
        if tc.name in catalog_names:
            allowed.append(tc)
        else:
            rejected.append(tc)
    for tc in rejected:
        err_payload = {
            "error": f"tool '{tc.name}' not in this turn's catalog",
            "available": sorted(catalog_names),
            "hint": "Use ONLY the exact tool names listed in the schema; "
                    "any other names will fail. Retry with the correct name.",
        }
        # ... append structured error to history as a tool result ...
    tool_calls = allowed

The structured error is inserted into history via format_tool_result() with the same format tag the call carried (native, kimi_special, xml_invoke, xml_sentinel). On the next turn, the model sees the rejection and available-names list inline with its prior turn, which in production observation produces self-correction within one or two retries. The error is a JSON object (error, available, hint) rather than a flat string, so the model attends to available without parsing prose.

The two mechanisms compose: the fuzzy resolver catches paraphrases with high recall and threshold-controlled precision; the catalog gate catches the residue (hallucinated tools and low-confidence paraphrases) with a structured error that drives convergence on the canonical name.

7. Hallucinated <function_results> detection (V3-Q.3)

The novel contribution of the lenient parser is the hallucinated-results detector. Some open-weight models — Kimi K2 in particular — occasionally hallucinate not just tool calls but tool results, emitting role-played <function_results> blocks as if a tool had returned data:

I'll search for that for you.

<function_results>
<result name="code_search" id="x">
{"matches": [{"path": "src/auth.py", "line": 42, "text": "..."}]}
</result>
</function_results>

Based on the search results, the authentication logic is in...

The model has emitted both the question and its own fabricated answer in a single turn. Without detection, the consequence is severe: the transcript contains a fake tool-result message, and on the next turn the model treats the fabrication as ground truth and continues reasoning on top of it.

Memagen™ registers <function_results> as a fourth opener in the parser’s state machine. The format-tag is hallucinated_results; the closer is </function_results>:

_OPEN_HALLUC = "<function_results>"
_CLOSE_HALLUC = "</function_results>"

_OPENERS: list[tuple[str, str, str]] = [
    (_OPEN_D, _CLOSE_D, "kimi_special"),
    (_OPEN_B, _CLOSE_B, "xml_invoke"),
    (_OPEN_C, _CLOSE_C, "xml_sentinel"),
    (_OPEN_HALLUC, _CLOSE_HALLUC, "hallucinated_results"),
]

When the active format is hallucinated_results, the section is suppressed: never yielded as visible content, never producing a synthetic tool_call_delta. The parser instead emits a single hallucinated_results_detected event whose payload carries the running count of hallucinated blocks seen so far in the stream:

if active_fmt == "hallucinated_results":
    halluc_count += 1
    yield (
        "hallucinated_results_detected",
        json.dumps({"count": halluc_count}),
    )

The agent loop in core/agent/loop.py listens for the event:

elif kind == "hallucinated_results_detected":
    yield {"type": "hallucinated_results_detected",
           "payload": payload}
    if halluc_retries < _HALLUC_MAX_RETRIES:
        halluc_retries += 1
        halluc_directive_pending = True

When the pending bit is set, the next round’s system prompt is augmented with a one-shot corrective directive:

_HALLUC_DIRECTIVE = (
    "You emitted `<function_results>` blocks. Those are reserved "
    "for actual tool results sent back to you AFTER you make a "
    "real tool call. Do not generate them yourself. Use real tool "
    "calls and wait for actual results."
)

The directive is one-shot — appended for only the next round; the following round returns to the unmodified prompt. The retry counter is capped at _HALLUC_MAX_RETRIES = 2. If hallucinations continue after two corrective directives, the runtime stops re-trying and lets the conversation proceed (suppression continues, directive injection halts).

Two design choices matter:

Suppression, not rejection. A strict-reject runtime would discard the entire assistant turn. We chose suppression because the rest of the turn — surrounding prose and any real tool calls — is usable. The hallucinated block is the only part dropped, which is strictly better when the assistant turn is mixed.

One-shot, not persistent. A persistent directive would remain in the system prompt for the rest of the session, eventually biasing the model against legitimate references to tool results (e.g., summarising what a tool returned). One-shot injection avoids the bias.

Detector behaviour and directive injection are covered by a nine-test suite in tests/llm/test_lenient_parser_hallucination.py (opener/closer detection, suppression, count tracking, mixed-format streams, stream-end flush).

8. Implementation notes

8.1 Tool-result formatter and the fmt tag

Format detection produces a tool call. The runtime executes it. The result must be inserted back into conversation history in a shape the model recognises. For native-format calls, this is a role=tool message keyed by tool_call_id. For lenient-format calls, the model never saw a tool_call_id in the protocol sense, so we synthesise a role=user message containing a <function_results> block keyed by tool name and synthetic ID:

def format_tool_result(tc, result_str, *, fmt):
    if fmt == "native":
        return ChatMessage(role="tool", content=result_str,
                           tool_call_id=tc.id)
    wrapped = (
        f"<function_results>\n"
        f"<result name=\"{tc.name}\" id=\"{tc.id}\">\n"
        f"{result_str}\n"
        f"</result>\n"
        f"</function_results>"
    )
    return ChatMessage(role="user", content=wrapped)

The fmt tag travels with the call. The router’s accumulator extracts _lenient_fmt from the synthetic delta and stores it in a side-channel keyed on the tool-call ID (_TC_FORMAT_BY_ID); the agent loop reads it back when constructing the ToolCall. This roundtrip is what allows downstream history-replay and result-injection to know which dialect was used.

A practical consequence: _scan_recent_tool_results() in the argument coercer must handle both shapes. The <function_results> wrapper used for lenient-format results is the same wrapper the hallucinated-results detector suppresses on the outbound stream. The detector matters because without it, the model would see its own fabricated <function_results> blocks treated as real tool results in subsequent turns — exactly the corruption to prevent.

8.2 Side-channel format tracking

The _TC_FORMAT_BY_ID: dict[str, str] module-level dict is a deliberate side-channel. The streaming protocol carries tool_call_delta events as JSON strings; adding a new top-level field (_lenient_fmt) to the delta would change the over-the-wire shape consumed by the router accumulator and any external consumer. Keeping the format tag as a side-channel keyed on the synthetic ID lets the agent loop reattach it via _LFMT.get(tc.get("id", ""), "native") without changing the protocol. The cost is global state, acceptable here because the side-channel is single-process and the IDs are namespaced (lenient_{turn_idx}_{call_idx}). This is the same splice pattern the router uses for _ACTIVE_MODEL_FOR_HINT and _ACTIVE_TOOL_CATALOG_FOR_HINT.

8.3 System-prompt hint

For models that need lenient parsing (the _WEAK_TOOL_CALL_PATTERNS allowlist: kimi-k2, kimi, qwen, llama-3, mistral-, deepseek-r, moonshotai), build_tool_hint() produces a system-prompt addendum that nudges the model toward Format C (the JSON-in-sentinel format weakest models converge to most reliably) and, when a catalog is provided, lists the exact tool names available for the turn:

You MUST use the EXACT tool names provided in the schema. The available
tools for this turn are:
  - widget_create
  - workspace_create
  - workspace_apply

Do NOT invent or paraphrase tool names. Only the names listed above will work.

For frontier models, the hint returns None so we do not waste tokens explaining a format their native API already handles. The hint is best-effort: weak models still occasionally ignore it, which is exactly why we need the parser, the coercer, the fuzzy resolver, the catalog gate, and the hallucinated-results detector — defence in depth.

8.4 Edge cases

• Split-frame openers: handled by the lookahead buffer (§4.2). Held-back tail bounded by len(longest_opener) - 1.
• Partial-open fragments at stream end: the wrap() flush logic surfaces unparseable buffered bytes as content rather than dropping them silently, with the special case for hallucinated-results which is treated as a detection.
• Mixed-format streams: each section is processed independently in the while candidate loop within wrap(); format-A native deltas pass through; formats B/C/D produce synthetic deltas; the hallucinated format produces detection events.
• JSON parse failures inside Format D: raw bytes are wrapped in {"_raw": ...} and emitted; the dispatcher surfaces the failure rather than silently dropping the call.
• XML parse failures inside Format B: the section is dropped; downstream the absence of any tool call drives the same on-no-progress reflexion path the loop uses for any other empty turn.

8.5 Test suite

The test suite lives in:

• tests/llm/test_lenient_parser.py — format A/B/C/D detection, fuzzy resolver, edge cases.
• tests/llm/test_lenient_parser_fuzzy.py — fuzzy-name resolution corner cases.
• tests/llm/test_lenient_parser_hallucination.py — V3-Q.3 nine-test suite covering hallucinated-results detection.
• tests/llm/test_router_lenient_wiring.py — end-to-end router wiring.

The argument coercer has its own coverage in the agent-test layer (tests/agent/test_arg_coercion.py); the catalog gate is exercised by the loop’s broader integration tests.

9. Evaluation

We are explicit about what is and is not measured.

What is measured. The nine-test V3-Q.3 hallucination-detection suite passes. The Format A/B/C/D detection tests pass. The fuzzy resolver tests pass against synthetic catalogs. The argument coercer tests pass against the three registered tools (widget_create, workspace_create, workspace_apply). The end-to-end live acceptance test against an open-weight model on NIM was the success criterion for the V3-Q development wave and passes on the supported provider configuration.

What is not measured. No public benchmark exists for tool-call recovery on weak-tool-call models. The format mix in production is from internal observations during V3-Q development; we do not claim a representative distribution across the open-weight ecosystem. Production deployment at scale has not happened — Memagen™ is pre-launch at the time of writing. The fuzzy-resolver threshold (0.7) was chosen by inspection of paraphrase patterns observed during development; we have not run a precision–recall sweep across a labelled corpus.

Per-format coverage. We catch the four formats described in §3. There are emission patterns we have not seen yet — see §11 on Format E. The catalog gate (§6.2) is the safety net for unknown patterns: any tool name that fails to match a catalog entry is rejected with a structured error rather than silently dispatched.

We list these limits because the alternative — claiming benchmark numbers we have not measured — is dishonest. The master paper §12 takes the same position.

10. Discussion

Lenient parsing gives up strictness as a runtime-enforced contract. A strict-reject runtime is a stronger guarantee that output is canonical, and a stronger guarantee is in some sense better. The cost is a runtime that only works against frontier models, which prices out the BYOLLM economics that justify the open-core distribution.

What lenient parsing gains:

• BYOLLM economics. Customers run Memagen™ against open-weight models on commodity hardware without sacrificing tool reliability — the primary commercial justification.
• Faster iteration on smaller models. A new open-weight model can be tested against the existing tool surface without waiting for post-training to converge on a canonical emission shape; if the model emits any of the four supported formats, the parser handles it.
• Dev-experience on weak models. A developer prototyping against a local Llama 3.x or Qwen 2.x deployment sees their tools work, rather than being forced onto a frontier model to debug.

Strict rejection and lenient recovery are not mutually exclusive: capability enforcement (the V3-P vault, master paper §5) is strict and runs after the lenient parser. A leniently-parsed call still passes the capability gate to dispatch, so a malformed envelope cannot bypass capability checks.

11. Conclusion and future work

The lenient parser bridges weak-tool-call models to the native tool catalog by recognising four production-observed emission patterns, applying registry-based per-tool argument coercion, fuzzy-resolving paraphrased tool names, gating off-catalog calls with a structured error, and detecting-and-suppressing hallucinated <function_results> blocks. The implementation is open-source under MIT in the Memagen™ Lite distribution.

Future work:

• Format E. Emission patterns we have not yet observed. The catalog gate is the current safety net; a richer telemetry layer that flags unknown opener-like sequences for human review would let us extend coverage proactively rather than reactively.
• Coercer-as-LLM. The registry-based coercer is per-tool work. A small dedicated model could in principle learn the coercion mapping from observed (loose, canonical) pairs and generalise to new tools without per-tool registration. The trade-off is one more model in the dispatch path; whether the latency cost is acceptable depends on tool-call frequency.
• Streaming-aware coercion. The current coercer operates on the fully-assembled argument string. A streaming-aware coercer could begin reshaping arguments before the closing sentinel arrives, reducing dispatch latency by a few hundred milliseconds on long argument blobs.
• Cross-provider format normalisation. The fmt tag is currently per-call; a richer tagging scheme could record which provider, which model variant, and which catalog the call was issued against, supporting per-provider observability dashboards (see LLM router paper).
• Hallucinated-results telemetry. The _HALLUC_MAX_RETRIES = 2 cap is a fixed bound. A more nuanced policy — e.g., escalate to provider switching after two hallucinations on the same turn — is plausible future work that depends on the multi-provider router’s switching costs.

12. References

1. Levitas, T. (2026). Memagen™: An Open-Core Agent Platform with Lenient Tool Routing, Capability-Gated Execution, and a Compound-Graph Memory Substrate. Master paper, Memagen™.
2. Levitas, T. (2026). Multi-Provider LLM Routing for Tool-Calling Agents. Specialty paper, Memagen™.
3. Anthropic. (2024). Tool use with Claude. https://docs.anthropic.com/claude/docs/tool-use
4. Anthropic. (2024). Model Context Protocol Specification. https://modelcontextprotocol.io
5. Beurer-Kellner, L., Fischer, M., & Vechev, M. (2023). Prompting is programming: A query language for large language models. PLDI 2023.
6. Chase, H. (2022). LangChain: Building applications with LLMs through composability. https://github.com/langchain-ai/langchain
7. DeepSeek-AI. (2024). DeepSeek-V3 technical report. arXiv:2412.19437.
8. Google. (2024). Function calling with the Gemini API. https://ai.google.dev/docs/function_calling
9. Meta AI. (2024). The Llama 3 herd of models. arXiv:2407.21783.
10. Moonshot AI. (2024). Kimi K2 technical report. Moonshot AI Research.
11. OpenAI. (2023). Function calling and other API updates. https://openai.com/blog/function-calling-and-other-api-updates
12. Ouyang, L., Wu, J., Jiang, X., et al. (2022). Training language models to follow instructions with human feedback. NeurIPS 2022.
13. Qwen Team, Alibaba. (2024). Qwen2 technical report. arXiv:2407.10671.
14. Ratcliff, J. W., & Obershelp, D. E. (1988). Pattern matching: The Gestalt approach. Dr. Dobb’s Journal, 13(7), 46–51.
15. Willard, B.T., & Louf, R. (2023). Efficient guided generation for large language models. arXiv:2307.09702.
16. Wright, A., Andrews, H., Hutton, B., & Dennis, G. (2019). JSON Schema: A media type for describing JSON documents. IETF Internet-Draft draft-handrews-json-schema-02.