protolabsai-protoagent-302-304

total 468
drwxr-xr-x 1 root root   4096 Nov  4 22:09 .
drwxr-xr-x 1 root root   4096 Nov  4 22:09 ..
drwxr-xr-x 1 root root   4096 Nov  4 22:08 .automaker
drwxr-xr-x 1 root root   4096 Nov  4 22:09 .beads
-rw-r--r-- 1 root root   1106 Nov  4 22:09 .dockerignore
drwxr-xr-x 1 root root   4096 Nov  4 22:09 .git
drwxr-xr-x 1 root root   4096 Nov  4 22:09 .github
-rw-r--r-- 1 root root   1691 Nov  4 22:09 .gitignore
-rw-r--r-- 1 root root   4287 Nov  4 22:09 Dockerfile
-rw-r--r-- 1 root root  12551 Nov  4 22:09 README.md
-rw-r--r-- 1 root root   8631 Nov  4 22:09 TEMPLATE.md

-rw-r--r-- 1 root root  73569 Nov  4 22:09 a2a_handler.py
drwxr-xr-x 1 root root   4096 Nov  4 22:08 apps
-rw-r--r-- 1 root root   3858 Nov  4 22:09 audit.py
-rw-r--r-- 1 root root   8945 Nov  4 22:09 autostart.py
-rw-r--r-- 1 root root   3563 Nov  4 22:09 cache.py
-rw-r--r-- 1 root root  49119 Nov  4 22:09 chat_ui.py

…[3260 chars omitted]…
te_code.py
./tests/test_fallbacks.py
./tests/test_github_tools.py
./tests/test_goal_controller.py
./tests/test_goal_store.py
./tests/test_goal_turn.py
./tests/test_goal_verifiers.py
./tests/test_guardrails.py
./tests/test_hot_memory.py
./tests/test_hybrid_store.py
./tests/test_knowledge_ingest.py
./tests/test_llm.py
./tests/test_manual_subagents.py
./tests/test_mcp_tools.py
./tests/test_memory_load.py

"""Structured output protocol for protoAgent — `<scratch_pad>` / `<output>` tags.

The model is instructed to wrap internal deliberation in ``<scratch_pad>``
and the user-facing answer in ``<output>``. Server-side, we parse those
tags and forward only the ``<output>`` content to consumers (A2A
artifacts, Gradio chat, subagent return values).

We deliberately do NOT parse the protocol mid-stream — chunk-boundary
tag splitting turned that into a state-machine rabbit hole and the
per-token text rendering consumers were doing didn't add real value.
Instead, ``_chat_langgraph_stream`` accumulates the model's tokens
silently while still emitting tool-start / tool-end status events, then

passes the complete text through ``extract_output`` once on the
terminal ``done`` frame. The consumer sees tool progress during the run
and the clean final artifact at completion.

``_strip_reasoning`` also 
…[8628 chars omitted]…
ed only reasoning (`<scratch_pad>`/`<think>`) — "
    "no tool call and no `<output>` block, so it was dropped. Pick up where "
    "you left off: if you were about to call a tool, call it now; if you have "
    "enough to answer, write the answer in `<output>` directly. Do not emit "
    "another bare reasoning block without committing to one of those paths."
)
===== TESTS =====
test_output_format.py

1	"""Tests for graph.output_format — <scratch_pad>/<output> protocol.
     2	
     3	Covers the three shapes of traffic we see live:
     4	
     5	1. Well-behaved model — emits both tags in the documented order.
     6	2. Mixed — emits `<scratch_pad>` but forgets `<output>` wrapper.
     7	3. Native thinking — provider (MiniMax, DeepSeek, Qwen3) leaks
     8	   `<think>...</think>` regions that the filter must also strip.
     9	
    10	The one-shot terminal path runs the complete text through ``extract_output``.
    11	The incremental path (``stream_visible_output``) streams the user-facing
    12	``<output>`` region token-by-token without leaking ``<scratch_pad>``; the

    13	terminal ``extract_output`` reconciles any held-back tail. Both are covered.
    14	"""
    15	
    16	from __future__ import annotations
    17	
    18	from graph.output_format import (
    19	    OUTPUT
…[7847 chars omitted]…
= ""
   195	    seen = ""
   196	    for c in chunks:
   197	        raw += c
   198	        vis = stream_visible_output(raw)
   199	        assert vis.startswith(seen) or seen.startswith(vis)  # never diverges
   200	        seen = vis
   201	    assert seen == "Hello world"
   202	    # The terminal extractor agrees with the final streamed text.
   203	    assert extract_output(raw) == "Hello world"

./docs/explanation/a2a-protocol.md:35:The assistant's answer streams incrementally: `artifact-update` frames with `append: true` carry each new suffix as the model generates it, and a final `append: false` frame replaces the artifact with the authoritative full text. Only the user-facing `<output>` region streams — the server's `stream_visible_output` holds back the `<scratch_pad>` and any partial trailing tag, and the terminal `extract_output` reconciles the result. Consumers that only want the final answer can ignore the deltas and read the last `append: false` frame.
./docs/explanation/output-protocol.md:15:Server-side, `graph/output_format.py::extract_output` parses tags and forwards only the `<output>` content to consumers (A2A artifacts, Gradio chat, subagent return values).
./docs/explanation/output-protocol.md:54:3. On the terminal `done` frame, `extract_output` runs once on t
…[7555 chars omitted]…
noise</think>B

# Output protocol

The template instructs the model to wrap its response as:

```
<scratch_pad>
Internal reasoning — which tools to call, what you're learning from
each result, how you'll assemble the answer. Not shown to the user.
</scratch_pad>
<output>
The user-facing answer. Clean, scannable, markdown-formatted.

text = _SCRATCH_RE.sub("", text)
    text = _ORPHAN_SCRATCH_OPEN_RE.sub("", text)
    # Confidence tags ride a DataPart, never the user-facing text. Strip them
    # in case the model emits them inside (or right after) <output>.
    text = _CONFIDENCE_EXPL_BLOCK_RE.sub("", text)
    text = _CONFIDENCE_BLOCK_RE.sub("", text)
    return text


_ORPHAN_OUTPUT_OPEN_RE = re.compile(r"<output>([\s\S]*)$", re.IGNORECASE)

def stream_visible_output(raw: str) -> str:
    """The portion of the user-facing ``<output>`` that's safe to show mid-stream.

    Given a *partial* (still-streaming) raw response, returns only the text
    inside the first (possibly still-open) ``<output>`` block, with reasoning
    stripped and any partial trailing tag held back — so a half-written
    ``

{"id":"bd-1tf.5","title":"Tests + docs (configuration.md, goal-mode guide)","status":"closed","priority":2,"issue_type":"task","created_at":"2026-05-28T08:17:41.902931862Z","created_by":"claude","updated_at":"2026-05-28T08:32:32.950437927Z","closed_at":"2026-05-28T08:32:32.950111063Z","close_reason":"30 tests (store/verifiers/controller) green; full suite 554. Docs: configuration.md goal section + guides/goal-mode.md + sidebar entry.","source_repo":".","compaction_level":0,"original_size":0,"dependencies":[{"issue_id":"bd-1tf.5","depends_on_id":"bd-1tf","type":"parent-child","created_at":"2026-05-28T08:17:41.902931862Z","created_by":"claude","metadata":"{}","thread_id":""}]}
{"id":"bd-3up","title":"React/Tauri operator console","description":"Replace the Gradio operator UI with a React console using Ava-style multi-chat, manual subagent launch, notes, beads task rendering, and eventua
…[8630 chars omitted]…
ed/wont-do; reopen if protoAgent moves to self-hosted inference.","source_repo":".","compaction_level":0,"original_size":0,"labels":["deferred","infra","perf","spike"],"dependencies":[{"issue_id":"bd-pe2.9","depends_on_id":"bd-pe2","type":"parent-child","created_at":"2026-05-27T09:44:14.003070235Z","created_by":"claude","metadata":"{}","thread_id":""}]}
categories.json
settings.json
{
  "version": 1
}

The answer mentions `<scratch_pad>` here.

1	"""Tests for the confidence-v1 A2A extension wiring."""
     2	
     3	import pytest
     4	
     5	from a2a_handler import (
     6	    CANCELED,
     7	    COMPLETED,
     8	    CONFIDENCE_MIME,
     9	    A2ATaskStore,
    10	    TaskRecord,
    11	    _confidence_payload,
    12	    _terminal_artifact_parts,

    13	)
    14	from graph.output_format import extract_confidence, extract_output
    15	
    16	
    17	def _now() -> str:
    18	    from datetime import datetime, timezone
    19	
    20	    return datetime.now(timezone.utc).isoformat()
    21	
    22	
    23	def _record(**kw) -> TaskRecord:
    24	    d = dict(
    25	        id="t", context_id="c", state=COMPLETED,
    26	        created_at=_now(), updated_at=_now(), message_text="hi",
    27	    )
    28	    d.update(kw)
    29	    return TaskRecord(**d)
    30	
    31	
    32	# ── extract_confidence ────────────
…[3090 chars omitted]…
, state="working"))
   109	    await store.set_confidence(rec.id, confidence=1.7, explanation="  over  ")
   110	    got = await store.get(rec.id)
   111	    assert got.confidence == 1.0  # clamped
   112	    assert got.confidence_explanation == "over"
   113	
   114	    await store.set_confidence(rec.id, confidence=-0.5)
   115	    got = await store.get(rec.id)
   116	    assert got.confidence == 0.0

==================================== ERRORS ====================================
__________________ ERROR collecting tests/test_confidence.py ___________________
ImportError while importing test module '/repo/tests/test_confidence.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_confidence.py:5: in <module>
    from a2a_handler import (
a2a_handler.py:44: in <module>
    import httpx

E   ModuleNotFoundError: No module named 'httpx'
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__ini
…[30 chars omitted]…
ng: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
ERROR tests/test_confidence.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
1 warning, 1 error in 0.49s

...........................                                              [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
27 passed, 1 warning in 0.21s

[project]
name = "protoagent"
version = "0.3.57"
description = "protoAgent — LangGraph + A2A template for spawning protoLabs agents"
requires-python = ">=3.11"

[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
pythonpath = ["."]
Python 3.12.13

The answer mentions `<scratch_pad>` here.

timeout: failed to run command ‘curl’: No such file or directory
EXIT:0

URLError <urlopen error [Errno -3] Temporary failure in name resolution>

helpers inserted

``, and
    orphaned variants) from a complete response.

    Idempotent — real user content should ne
…[1566 chars omitted]…
text:
    raise SystemExit("strip function not found")
path.write_text(text.replace(old, new, 1))
print("strip updated")
PY
python -m py_compile graph/output_format.py
<returncode>0</returncode>

<output>
strip function not found

return text, []
    parts.append(text[last:])
    return "".join(parts), protected


def _restore_code_span_tags(text: str, protected: list[str]) -> str:
    if not protected:
        return text
    return _PROTECTED_TAG_RE.sub(
        lambda m: protected[int(m.group(1))], text,
    )

def _strip_reasoning(text: str) -> str:
    """Remove all reasoning markers (``<think>``, ``<scratch_pad>``, and
    orphaned variants) from a complete response.

    Idempotent — real user content should never contain literal tag
    markers, so applying this twice is safe.
    """
    text = _THINK_RE.sub("", text)
    text = _ORPHAN_THINK_OPEN_RE.sub("", text)
    text = _ORPHAN_THINK_CLOSE_RE.sub("", text)
    text = _SCRATCH_RE.sub("", text)
    text = _ORPHAN_SCRATCH_OPEN_RE.sub("", text)
    # Confidence tags ride a DataPart, never the user-facing text. Strip them
    # in case the mod
…[469 chars omitted]…
ibly still-open) ``<output>`` block, with reasoning
    stripped and any partial trailing tag held back — so a half-written
    ``

strip updated

stream updated

extract/dropped/confidence updated
 graph/output_format.py | 169 ++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 140 insertions(+), 29 deletions(-)

"""Structured output protocol for protoAgent — `<scratch_pad>` / `<output>` tags.

The model is instructed to wrap internal deliberation in ``<scratch_pad>``
and the user-facing answer in ``<output>``. Server-side, we parse those
tags and forward only the ``<output>`` content to consumers (A2A
artifacts, Gradio chat, subagent return values).

We deliberately do NOT parse the protocol mid-stream — chunk-boundary
tag splitting turned that into a state-machine rabbit hole and the
per-token text rendering consumers were doing didn't add real value.
Instead, ``_chat_langgraph_stream`` accumulates the model's tokens
silently while still emitting tool-start / tool-end status events, then

passes the complete text through ``extract_output`` once on the
terminal ``done`` frame. The consumer sees tool progress during the run
and the clean final artifact at completion.

``_strip_reasoning`` also 
…[7929 chars omitted]…
tail at the end. Monotonic —
    as ``raw`` grows, the result only ever extends (until ``

# Unclosed spans run to EOT while streaming: the closing backtick may
    # simply be in a future chunk.
    masked, protected = _protect_code_span_tags(raw, allow_unclosed=True)
    low = masked.lower()
    start = low.find("<output>")
    if start == -1:
        return ""  # still in scratch_pad — nothing user-facing yet
    after = masked[start + len("<output>") :]
    close = after.lower().find("

diff --git a/graph/output_format.py b/graph/output_format.py
index 79cc3fc..a9aef15 100644
--- a/graph/output_format.py
+++ b/graph/output_format.py
@@ -81,14 +81,115 @@ _CONFIDENCE_EXPLANATION_RE = re.compile(
     r"<confidence_explanation>([\s\S]*?)</confidence_explanation>", re.IGNORECASE,
 )
 
+# Backtick-delimited code (inline spans and fenced blocks) may legitimately
+# mention protocol tags when the model explains the protocol itself.  Those
+# mentions are prose, not structural delimiters, so mask them before matching
+# and restore them in the returned content.

+_PROTOCOL_TAG_RE = re.compile(
+    r"</?(?:scratch_pad|output|confidence_explanation|confidence|think)>",
+    re.IGNORECASE,
+)
+_PROTECTED_TAG_RE = re.compile(r"\x00prototag(\d+)\x00")
+
+
+def _backtick_code_spans(
+    text: str, *, allow_unclosed: bool = False,
+) -> list[tuple[int, int]]:
+    """Return ``(sta
…[8483 chars omitted]…
ed, _ = _protect_code_span_tags(text)
+    lower = masked.lower()
+    if "<scratch_pad>" not in lower and " thinking" not in lower:
         return False
     return "<output>" not in lower
 
 
+
 # Follow-up user message sent on the same thread when is_dropped_scratch_turn
 # fires — the dropped turn is still in the checkpointer history, so the model
 # has full context to pick up where it left off.

diagnostic restored
...............F...........                                              [100%]
=================================== FAILURES ===================================
_____________________ test_dropped_think_only_is_detected ______________________

    def test_dropped_think_only_is_detected():
        """Qwen-style <think> with no output is also a drop."""
>       assert is_dropped_scratch_turn("<think>reasoning</think>") is True
E       AssertionError: assert False is True
E        +  where False = is_dropped_scratch_turn('<think>reasoning</think>')

tests/test_output_format.py:133: AssertionError

=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/test_output_format.py::test_dropped_think_only_is_detected - Ass...
1 failed, 26 passed, 1 warning in 0.30s

def test_dropped_scratch_only_is_detected():$
    assert is_dropped_scratch_turn("<scratch_pad>thinking, never committed</scratch_pad>") is True$
$
$
def test_dropped_think_only_is_detected():$
    """Qwen-style <think> with no output is also a drop."""$
    assert is_dropped_scratch_turn("<think>reasoning</think>") is True$
$
$
def test_not_dropped_when_output_present():$
    assert is_dropped_scratch_turn("<scratch_pad>x</scratch_pad><output>answer

127 b'def test_dropped_scratch_only_is_detected():'
128 b'    assert is_dropped_scratch_turn("<scratch_pad>thinking, never committed</scratch_pad>") is True'
129 b''
130 b''
131 b'def test_dropped_think_only_is_detected():'
132 b'    """Qwen-style <think> with no output is also a drop."""'
133 b'    assert is_dropped_scratch_turn("<think>reasoning</think>") is True'
134 b''
135 b''

b'    assert is_dropped_scratch_turn("<think>reasoning</think>") is True'
b'"<think>reasoning</think>"'
'<think>reasoning</think>' b'<think>reasoning</think>'
spans []
masked '<think>reasoning</think>'
dropped False

' in lower)
import inspect
print(inspect.getsource(of.is_dropped_scratch_turn))
print('result', of.is_dropped_scratch_turn(val))
PY
<returncode>0</returncode>
<output>
'<think>reasoning</think>' scratch? False think? True output? False
def is_dropped_scratch_turn(text: str) -> bool:
    """Detect the 'scratch-only, never committed' dropped-turn pattern.

    Failure mode: the model writes reasoning (``<scratch_pad>...`` or

    ``<think>...``) and then stops without emitting a tool call or an
    ``<output>`` block. ``extract_output`` strips the reasoning, returns
    empty, and the turn silently drops. Detecting it lets the server issue a
    kicker and retry once. Callers should also confirm no tool call fired this
    turn (the LangChain tool channel is separate from text content) — an empty
    extract_output with a tool call is a normal mid-loop step, not a drop.

    True when the text has ``<scratch_pad>`` or ``<think>`` content and no
    ``<output>`` tag.
    """
    if not text:
        return False
    # Ignore tag mentions inside backtick code spans — those are prose, not
    # protocol markers.
    masked, _ = _protect_code_span_tags(text)
    lower = masked.lower()
    if "<scratch_pad>" not in lower and " thinking" not in lower:
        return False
    return "<output>" not in lower

result False

./graph/output_format.py:332:def is_dropped_scratch_turn(text: str) -> bool:
graph.output_format
/repo/graph/output_format.py
332           0 RESUME                   0

346           2 LOAD_FAST                0 (text)
              4 POP_JUMP_IF_TRUE         1 (to 8)

347           6 RETURN_CONST             1 (False)

350     >>    8 LOAD_GLOBAL              1 (NULL + _protect_code_span_tags)
             18 LOAD_FAST                0 (text)

             20 CALL                     1
             28 UNPACK_SEQUENCE          2
             32 STORE_FAST               1 (masked)
             34 STORE_FAST               2 (_)

351          36 LOAD_FAST                1 (masked)
             38 LOAD_ATTR                3 (NULL|self + lower)
             58 CALL                     0
             66 STORE_FAST               3 (lower)

352          68 LOAD_CONST               2 ('<scratc
…[183 chars omitted]…
            3 (' thinking')
             78 LOAD_FAST                3 (lower)
             80 CONTAINS_OP              1
             82 POP_JUMP_IF_FALSE        1 (to 86)

353          84 RETURN_CONST             1 (False)

354     >>   86 LOAD_CONST               4 ('<output>')
             88 LOAD_FAST                3 (lower)
             90 CONTAINS_OP              1
             92 RETURN_VALUE

3c7468696e6b3e726561736f6e696e673c2f7468696e6b3e
[60, 116, 104, 105, 110, 107, 62, 114, 101, 97, 115, 111, 110, 105, 110, 103, 60, 47, 116, 104, 105, 110, 107, 62]
24

'<think>reasoning</think>'
branch scratch True
branch think False
return expression True
func False
func masked False
func repr val False
val repr '<think>reasoning</think>'

' not in masked.lower())
print('call:', of.is_dropped_scratch_turn(val))
# instrument source manually via exec?
src = inspect.getsource(of.is_dropped_scratch_turn)
ns = dict(of.__dict__)
exec(src, ns)
print('local exec:', ns['is_dropped_scratch_turn'](val))
PY
<returncode>0</returncode>
<output>
False
''

'' True
call: False
local exec: False

'def is_dropped_scratch_turn(text: str) -> bool:\n    """Detect the \'scratch-only, never committed\' dropped-turn pattern.\n\n    Failure mode: the model writes reasoning (``<scratch_pad>...`` or\n    ``<think>...``) and then stops without emitting a tool call or an\n    ``<output>`` block. ``extract_output`` strips the reasoning, returns\n    empty, and the turn silently drops. Detecting it lets the server issue a\n    kicker and retry once. Callers should also confirm no tool call fired this\n    turn (the LangChain tool channel is separate from text content) — an empty\n    extract_output with a tool call is a normal mid-loop step, not a drop.\n\n    True when the text has ``<scratch_pad>`` or ``<think>`` content and no\n    ``<output>`` tag.\n    """\n    if not text:\n        return False\n    # Ignore tag mentions inside backtick code spans — those are prose, not\n    # protoco
…[2647 chars omitted]…
dy=[
            Return(
              value=Constant(value=False))],
          orelse=[]),
        Return(
          value=Compare(
            left=Constant(value='<output>'),
            ops=[
              NotIn()],
            comparators=[

              Name(id='lower', ctx=Load())]))],
      decorator_list=[],
      returns=Name(id='bool', ctx=Load()),
      type_params=[])],
  type_ignores=[])

F lower bytes b'<think>reasoning</think>'
F scratch True
F think False
F ret True <class 'bool'>
f result True True True
official False False

/repo/graph/output_format.py
("Detect the 'scratch-only, never committed' dropped-turn pattern.\n\n    Failure mode: the model writes reasoning (``<scratch_pad>...`` or\n    ``<think>...``) and then stops without emitting a tool call or an\n    ``<output>`` block. ``extract_output`` strips the reasoning, returns\n    empty, and the turn silently drops. Detecting it lets the server issue a\n    kicker and retry once. Callers should also confirm no tool call fired this\n    turn (the LangChain tool channel is separate from text content) — an empty\n    extract_output with a tool call is a normal mid-loop step, not a drop.\n\n    True when the text has ``<scratch_pad>`` or ``<think>`` content and no\n    ``<output>`` tag.\n    ", False, '<scratch_pad>', ' thinking', '<output>')
True
('_protect_code_span_tags', 'lower')
_protect_code_span_tags <function _protect_code_span_tags at 0x7008a420d120>
globals helper result ('<think>reasoning</think>', [])

HELPER input b'<think>reasoning</think>' out b'<think>reasoning</think>' []
RESULT False
LINE 346 {'text': b'<think>reasoning</think>'}
LINE 350 {'text': b'<think>reasoning</think>'}
LINE 351 {'text': b'<think>reasoning</think>', 'masked': b'<think>reasoning</think>', '_': []}
LINE 352 {'text': b'<think>reasoning</think>', 'masked': b'<think>reasoning</think>', '_': [], 'lower': b'<think>reasoning</think>'}
LINE 353 {'text': b'<think>reasoning</think>', 'masked': b'<think>reasoning</think>', '_': [], 'lower': b'<think>reasoning</think>'}
RETURN 353 False
TRACED False

349 b'    # protocol markers.'
350 b'    masked, _ = _protect_code_span_tags(text)'
351 b'    lower = masked.lower()'
352 b'    if "<scratch_pad>" not in lower and " thinking" not in lower:'
353 b'        return False'
354 b'    return "<output>" not in lower'
355 b''
356 b''
357 b'# Follow-up user message sent on the same thread when is_dropped_scratch_turn'
358 b'# fires \xe2\x80\x94 the dropped turn is still in the checkpointer history, so the model'

446574656374207468652027736372617463682d6f6e6c792c206e6576657220636f6d6d6974746564272064726f707065642d7475726e207061747465726e2e0a0a202020204661696c757265206d6f64653a20746865206d6f64656c2077726974657320726561736f6e696e67202860603c736372617463685f7061643e2e2e2e6060206f720a2020202060603c7468696e6b3e2e2e2e60602920616e64207468656e2073746f707320776974686f757420656d697474696e67206120746f6f6c2063616c6c206f7220616e0a2020202060603c6f75747075743e606020626c6f636b2e206060657874726163745f6f75747075746060207374726970732074686520726561736f6e696e672c2072657475726e730a20202020656d7074792c20616e6420746865207475726e2073696c656e746c792064726f70732e20446574656374696e67206974206c657473207468652073657276657220697373756520610a202020206b69636b657220616e64207265747279206f6e63652e2043616c6c6572732073686f756c6420616c736f20636f6e6669726d206e6f20746f6f6c2063616c6c20666972656420746869730a202020207475726e20287468652
…[148 chars omitted]…
4726163745f6f75747075742077697468206120746f6f6c2063616c6c2069732061206e6f726d616c206d69642d6c6f6f7020737465702c206e6f7420612064726f702e0a0a2020202054727565207768656e207468652074657874206861732060603c736372617463685f7061643e6060206f722060603c7468696e6b3e606020636f6e74656e7420616e64206e6f0a2020202060603c6f75747075743e6060207461672e0a20202020
3c736372617463685f7061643e
207468696e6b696e67
3c6f75747075743e

b'    if not text:\n        return False\n    lower = text.lower()\n    if "<scratch_pad>" not in lower and "<think>" not in lower:\n        return False\n    return "<output>" not in lower\n\n'
202020206966206e6f7420746578743a0a202020202020202072657475726e2046616c73650a202020206c6f776572203d20746578742e6c6f77657228290a20202020696620223c736372617463685f7061643e22206e6f7420696e206c6f77657220616e6420223c7468696e6b3e22206e6f7420696e206c6f7765723a0a202020202020202072657475726e2046616c73650a2020202072657475726e20223c6f75747075743e22206e6f7420696e206c6f7765720a0a

20202020696620223c736372617463685f7061643e22206e6f7420696e206c6f77657220616e642022207468696e6b696e6722206e6f7420696e206c6f7765723a
[32, 32, 32, 32, 105, 102, 32, 34, 60, 115, 99, 114, 97, 116, 99, 104, 95, 112, 97, 100, 62, 34, 32, 110, 111, 116, 32, 105, 110, 32, 108, 111, 119, 101, 114, 32, 97, 110, 100, 32, 34, 32, 116, 104, 105, 110, 107, 105, 110, 103, 34, 32, 110, 111, 116, 32, 105, 110, 32, 108, 111, 119, 101, 114, 58]

protocol pattern bytes: 3c2f3f283f3a736372617463685f7061647c6f75747075747c636f6e666964656e63655f6578706c616e6174696f6e7c636f6e666964656e63657c7468696e6b293e
_OUTPUT_RE b'<output>([\\s\\S]*?)

condition fixed
...........................                                              [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
27 passed, 1 warning in 0.31s

PASS scratch mention in answer => 'Mention `<scratch_pad>` and keep going.'
PASS output mention in reasoning => 'real answer'
PASS closing output mention in answer => 'Use `

graph/output_format.py | 161 +++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 141 insertions(+), 20 deletions(-)
diff --git a/graph/output_format.py b/graph/output_format.py
index 79cc3fc..6d8f745 100644
--- a/graph/output_format.py
+++ b/graph/output_format.py
@@ -81,14 +81,115 @@ _CONFIDENCE_EXPLANATION_RE = re.compile(
     r"<confidence_explanation>([\s\S]*?)</confidence_explanation>", re.IGNORECASE,
 )
 
+# Backtick-delimited code (inline spans and fenced blocks) may legitimately
+# mention protocol tags when the model explains the protocol itself.  Those

+# mentions are prose, not structural delimiters, so mask them before matching
+# and restore them in the returned content.
+_PROTOCOL_TAG_RE = re.compile(
+    r"</?(?:scratch_pad|output|confidence_explanation|confidence|think)>",
+    re.IGNORECASE,
+)
+_PROTECTED_TAG_RE = re.compile(r"\x00prototag(\d+)\x00"
…[8436 chars omitted]…
False
-    lower = text.lower()
-    if "<scratch_pad>" not in lower and "<think>" not in lower:
+    # Ignore tag mentions inside backtick code spans — those are prose, not
+    # protocol markers.
+    masked, _ = _protect_code_span_tags(text)
+    lower = masked.lower()
+    if "<scratch_pad>" not in lower and ("<" + "think") not in lower:
         return False
     return "<output>" not in lower

current b'def _strip_reasoning(text: str) -> str:\n    """Remove all reasoning markers (`` thinking``, ``<scratch_pad>``, and\n    orphaned variants) from a complete response.\n\n    Protocol tags inside backtick-delimited code are literal mentions (e.g.\n    a response explaining the protocol), not markers, so they are protected\n    before stripping and restored afterwards.  Idempotent \xe2\x80\x94 applying this\n    twice is safe.\n    """'
head b'def _strip_reasoning(text: str) -> str:\n    """Remove all reasoning markers (``<think>``, ``<scratch_pad>``, and\n    orphaned variants) from a complete response.\n\n    Idempotent \xe2\x80\x94 real user content should never contain literal tag\n    markers, so applying this twice is safe.\n    """'

--- text 0 final extract: 'real answer'
final stream: 'real answer' shrank-any: False matches extract: True
--- text 1 final extract: 'The literal `<scratch_pad>` and `

raw '<output>Use a single ` backtick

stream refined
...........................                                              [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
27 passed, 1 warning in 0.32s

PASS reasoning mention stays hidden => ''
PASS reasoning mention then real output => 'Hi'
PASS answer mention preserved => 'Use `<output>` here'
PASS answer close mention preserved => 'Use `

", "

PASS '<output>Answer

bash: command substitution: line 1000: syntax error near unexpected token `newline'
bash: command substitution: line 1000: `<output>'

grep: ./graph/__pycache__/output_format.cpython-312.pyc: binary file matches
./graph/goals/verifiers.py:191:        # Tolerate fenced/extra text — grab the first JSON object.
./graph/output_format.py:84:# Backtick-delimited code (inline spans and fenced blocks) may legitimately
./graph/output_format.py:95:def _backtick_code_spans(
./graph/output_format.py:98:    """Return ``(start, end)`` ranges of backtick-delimited code in *text*.
./graph/output_format.py:100:    Matches runs of backticks with a closing run of the same length, covering
./graph/output_format.py:101:    both inline code spans (`` `...` ``) and fenced blocks (```` ```...``` ````).
./graph/output_format.py:103:    of the text — useful while streaming because the closing backtick may
./graph/output_format.py:144:    """Replace protocol tags inside backtick code spans with inert tokens.
./graph/output_format.py:151:    sp
…[675 chars omitted]…
./graph/output_format.py:253:    # unmatched backtick in the answer is ordinary prose; it must not hide

./graph/output_format.py:315:    # Protocol-tag mentions inside backtick code spans are prose, not
./graph/output_format.py:367:    # Ignore tag mentions inside backtick code spans — those are prose, not
./graph/skills/loader.py:88:    """Split ``---``-fenced YAML frontmatter from the markdown body.

96:    text: str, *, allow_unclosed: bool = False,
102:    With ``allow_unclosed``, an unmatched opening run protects the remainder
131:            if allow_unclosed:
142:    text: str, *, allow_unclosed: bool = False,
151:    spans = _backtick_code_spans(text, allow_unclosed=allow_unclosed)
241:    opener_spans = _backtick_code_spans(raw, allow_unclosed=True)

unclosed span handling narrowed
...........................                                              [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
27 passed, 1 warning in 0.23s

PASS extract 'Mention `<scratch_pad>` and keep going.'
PASS extract 'real answer'
PASS extract 'Use `

tests/test_starter_tools.py:55
  /repo/tests/test_starter_tools.py:55: PytestUnknownMarkWarning: Unknown pytest.mark.asyncio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.asyncio

tests/test_starter_tools.py:62
  /repo/tests/test_starter_tools.py:62: PytestUnknownMarkWarning: Unknown pytest.mark.asyncio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.asyncio

tests/test_starter_tools.py:69
  /repo/tests/test_starter_tools.py:69: PytestUnknownMarkWarning: Unknown pytest.mark.asyncio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.asyncio

tests/test_starter_to
…[3378 chars omitted]…
erator_web_mount.py
ERROR tests/test_peer_federation.py
ERROR tests/test_plugins.py
ERROR tests/test_prompt_cache.py
ERROR tests/test_scheduler_local.py
ERROR tests/test_scheduler_workstacean.py
ERROR tests/test_skill_index.py
ERROR tests/test_skills_loader.py
ERROR tests/test_task_batch.py
!!!!!!!!!!!!!!!!!!! Interrupted: 28 errors during collection !!!!!!!!!!!!!!!!!!!
47 warnings, 28 errors in 2.75s

gradio>=5.0
httpx>=0.27
uvicorn>=0.30
langfuse>=3.0
prometheus-client>=0.20
pyyaml>=6.0
ruamel.yaml>=0.18  # round-trip YAML that preserves comments in config/langgraph-config.yaml when the drawer writes back edits
websockets>=12.0

# LangGraph agent backend. Keep the langchain-* lower bounds on the 1.x
# line — a 0.3.x core/openai resolves against langchain 1.x into a broken
# `langchain_core` namespace (surfaced by checks.yml in PR #171).

langchain>=1.2.3
langchain-core>=1.0
langgraph>=1.1.0
langchain-openai>=1.0

# MCP client — connect external Model Context Protocol servers (stdio /
# streamable-HTTP); their tools become agent tools. See tools/mcp_tools.py.
# Pulls the `mcp` SDK transitively (also used by examples/mcp/echo_server.py).
langchain-mcp-adapters>=0.2,<0.3

# Starter tools (tools/lg_tools.py)
ddgs>=9.0
beautifulsoup4>=4.12

# Scheduler (scheduler/local.py — cron expression parsing for the
# bundled local backend; the Workstacean adapter doesn't need this)
croniter>=2.0
iniconfig==2.3.1
packaging==26.3
pip==25.0.1
pluggy==1.7.0
Pygments==2.21.0
pytest==9.2.0

M graph/output_format.py
 graph/output_format.py | 191 +++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 169 insertions(+), 22 deletions(-)

Use `", "<output>", "` literal", "

", "

restore guarded
...........................                                              [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
27 passed, 1 warning in 0.29s

# unmatched backtick in the answer is ordinary prose; it must not hide
    # the real ``

Returns ``(confidence, explanation)`` where confidence is a float or
    None (malformed/absent) and explanation is a stripped string or None.
    The A2A handler clamps confidence to [0, 1] on write.
    """
    confidence: float | None = None
    masked, protected = _protect_code_span_tags(text)
    m = _CONFIDENCE_RE.search(masked)
    if m:
        try:
            confidence = float(m.group(1))
        except ValueError:
            confidence = None

    explanation: str | None = None
    me = _CONFIDENCE_EXPLANATION_RE.search(masked)
    if me:
        explanation = (
            _restore_code_span_tags(me.group(1), protected).strip() or None
        )
    return confidence, explanation

diff --git a/graph/output_format.py b/graph/output_format.py
index 79cc3fc..34811e7 100644
--- a/graph/output_format.py
+++ b/graph/output_format.py
@@ -81,14 +81,136 @@ _CONFIDENCE_EXPLANATION_RE = re.compile(
     r"<confidence_explanation>([\s\S]*?)</confidence_explanation>", re.IGNORECASE,
 )
 
+# Backtick-delimited code (inline spans and fenced blocks) may legitimately
+# mention protocol tags when the model explains the protocol itself.  Those
+# mentions are prose, not structural delimiters, so mask them before matching
+# and restore them in the returned content.

+_PROTOCOL_TAG_RE = re.compile(
+    r"</?(?:scratch_pad|output|confidence_explanation|confidence|think)>",
+    re.IGNORECASE,
+)
+_PROTECTED_TAG_RE = re.compile(r"\x00prototag(\d+)\x00")
+
+
+def _backtick_code_spans(
+    text: str, *, allow_unclosed: bool = False,
+) -> list[tuple[int, int]]:
+    """Return ``(sta
…[8628 chars omitted]…
False
-    lower = text.lower()
-    if "<scratch_pad>" not in lower and "<think>" not in lower:
+    # Ignore tag mentions inside backtick code spans — those are prose, not
+    # protocol markers.
+    masked, _ = _protect_code_span_tags(text)
+    lower = masked.lower()
+    if "<scratch_pad>" not in lower and ("<" + "think") not in lower:
         return False
     return "<output>" not in lower

cleanup done
...........................                                              [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
27 passed, 1 warning in 0.27s
75:_THINK_OPEN_TAG = '<think'
384:    if "<scratch_pad>" not in lower and _THINK_OPEN_TAG not in lower:

all issue reproducers pass

<confidence_explanation>one short sentence on what drove the score</confidence_explanation>

- `<confidence>` is a number in [0, 1] — your honest self-assessment of
  whether the answer is correct/complete. Omit it if you'd only be guessing.
- `<confidence_explanation>` is optional. Neither tag is shown to the user;
  they ride a confidence-v1 DataPart on the A2A artifact.
""".strip()


_OUTPUT_RE = re.compile(r"<output>([\s\S]*?)

current 205 646566205f73747269705f726561736f6e696e6728746578743a2073747229202d3e207374723a
head    85 646566205f73747269705f726561736f6e696e6728746578743a2073747229202d3e207374723a
current 206 2020202022222252656d6f766520616c6c20726561736f6e696e67206d61726b65727320286060207468696e6b696e6760602c2060603c736372617463685f7061643e60602c20616e64
head    86 2020202022222252656d6f766520616c6c20726561736f6e696e67206d61726b657273202860603c7468696e6b3e60602c2060603c736372617463685f7061643e60602c20616e64

docstring restored
...........................                                              [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
27 passed, 1 warning in 0.23s

'answer mentions `<SCRATCH_PAD>` and `</OUTPUT>`'
case-insensitive pass
 M graph/output_format.py
 graph/output_format.py | 195 +++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 174 insertions(+), 21 deletions(-)

"""
-    low = raw.lower()
-    start = low.find("<output>")
-    if start == -1:
+    # Find the genuine output opener.  Protect backtick code spans first so
+    # protocol-tag *mentions* in reasoning are never mistaken for the start of
+    # the answer.  An unfinished mention (opening backtick + tag, with the
+    # closing backtick still in a future chunk) is protected too.
+    opener_spans = _backtick_code_spans(raw, allow_unclosed=True)
+    opener_end = -1
+    for match in _PROTOCOL_TAG_RE.finditer(raw):
+        if _position_in_spans(match.start(), opener_spans):

+            continue
+        if match.group(0).lower() == "<output>":
+            opener_end = match.end()
+            break
+    if opener_end == -1:
         return ""  # still in scratch_pad — nothing user-facing yet
-    after = raw[start + len("<output>") :]
+
+    # For the content/close, only *close
…[3262 chars omitted]…
 = text.lower()
-    if "<scratch_pad>" not in lower and "<think>" not in lower:
+    # Ignore tag mentions inside backtick code spans — those are prose, not
+    # protocol markers.
+    masked, _ = _protect_code_span_tags(text)
+    lower = masked.lower()
+    if "<scratch_pad>" not in lower and _THINK_OPEN_TAG not in lower:
         return False
     return "<output>" not in lower
 
if m: 2
if m
 0

...........................                                              [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
27 passed, 1 warning in 0.24s
 M graph/output_format.py

…[60 chars omitted]…
re preserved; streaming handles incomplete backticked mentions without exposing reasoning. Existing 27 output-format tests plus targeted reproducers/edge cases pass."}
submitted.

/repo
total 468
drwxr-xr-x 1 root root   4096 Nov  2 08:22 .
drwxr-xr-x 1 root root   4096 Nov  2 08:22 ..
drwxr-xr-x 1 root root   4096 Nov  2 08:21 .automaker
drwxr-xr-x 1 root root   4096 Nov  2 08:22 .beads
-rw-r--r-- 1 root root   1106 Nov  2 08:22 .dockerignore
drwxr-xr-x 1 root root   4096 Nov  2 08:22 .git
drwxr-xr-x 1 root root   4096 Nov  2 08:22 .github
-rw-r--r-- 1 root root   1691 Nov  2 08:22 .gitignore
-rw-r--r-- 1 root root   4287 Nov  2 08:22 Dockerfile
-rw-r--r-- 1 root root  12551 Nov  2 08:22 README.md

-rw-r--r-- 1 root root   8631 Nov  2 08:22 TEMPLATE.md
-rw-r--r-- 1 root root  73569 Nov  2 08:22 a2a_handler.py
drwxr-xr-x 1 root root   4096 Nov  2 08:21 apps
-rw-r--r-- 1 root root   3858 Nov  2 08:22 audit.py
-rw-r--r-- 1 root root   8945 Nov  2 08:22 autostart.py
-rw-r--r-- 1 root root   3563 Nov  2 08:22 cache.py
-rw-r--r-- 1 root root  49119 Nov  2 08:22 chat_
…[4240 chars omitted]…
py
./tests/test_skill_index.py
./tests/test_skill_persistence.py
./tests/test_starter_tools.py
./tests/test_task_batch.py
./tests/test_tool_preview.py
./tests/test_tracing.py
./tests/__init__.py
./tools/execute_code.py
./tools/fallbacks.py
./tools/gh_cli.py
./tools/github_tools.py
./tools/lg_tools.py
./tools/mcp_tools.py
./tools/peer_tools.py
./tools/shell.py
./tools/__init__.py
./tracing.py
./uv.lock

graph/config.py:61:    # historical <think>/<scratch_pad> blocks across turns.
graph/goals/goal_turn.py:7:reasoning bleeds in and the model gives up on an unrelated, achievable goal,
graph/output_format.py:1:"""Structured output protocol for protoAgent — `<scratch_pad>` / `<output>` tags.
graph/output_format.py:3:The model is instructed to wrap internal deliberation in ``<scratch_pad>``
graph/output_format.py:4:and the user-facing answer in ``<output>``. Server-side, we parse those
graph/output_format.py:5:tags and forward only the ``<output>`` content to consumers (A2A
graph/output_format.py:17:``_strip_reasoning`` also removes provider-emitted ``<think>...</think>``
graph/output_format.py:19:any orphaned scratch_pad / think openings.
graph/output_format.py:38:    <scratch_pad>
graph/output_format.py:39:    Internal reasoning — which tools to call, what you're learning from
graph/out
…[8628 chars omitted]…

ert stream_visible_output("<output>A<think>noise</think>B

"""Structured output protocol for protoAgent — `<scratch_pad>` / `<output>` tags.

The model is instructed to wrap internal deliberation in ``<scratch_pad>``
and the user-facing answer in ``<output>``. Server-side, we parse those
tags and forward only the ``<output>`` content to consumers (A2A
artifacts, Gradio chat, subagent return values).

We deliberately do NOT parse the protocol mid-stream — chunk-boundary
tag splitting turned that into a state-machine rabbit hole and the
per-token text rendering consumers were doing didn't add real value.
Instead, ``_chat_langgraph_stream`` accumulates the model's tokens
silently while still emitting tool-start / tool-end status events, then

passes the complete text through ``extract_output`` once on the
terminal ``done`` frame. The consumer sees tool progress during the run
and the clean final artifact at completion.

``_strip_reasoning`` also 
…[8630 chars omitted]…
noise</think>B

text = _SCRATCH_RE.sub("", text)
    text = _ORPHAN_SCRATCH_OPEN_RE.sub("", text)
    # Confidence tags ride a DataPart, never the user-facing text. Strip them
    # in case the model emits them inside (or right after) <output>.
    text = _CONFIDENCE_EXPL_BLOCK_RE.sub("", text)
    text = _CONFIDENCE_BLOCK_RE.sub("", text)
    return text


_ORPHAN_OUTPUT_OPEN_RE = re.compile(r"<output>([\s\S]*)$", re.IGNORECASE)

def stream_visible_output(raw: str) -> str:
    """The portion of the user-facing ``<output>`` that's safe to show mid-stream.

    Given a *partial* (still-streaming) raw response, returns only the text
    inside the first (possibly still-open) ``<output>`` block, with reasoning
    stripped and any partial trailing tag held back — so a half-written
    ``

return fallback

    preview = text[:400].replace("\n", "\\n")
    log.warning(
        "[extract_output] empty after stripping — len=%d scratch=%s "
        "output=%s preview=%r",
        len(text),
        "<scratch_pad>" in text.lower(),
        "<output>" in text.lower(),
        preview,
    )
    return ""

def is_dropped_scratch_turn(text: str) -> bool:
    """Detect the 'scratch-only, never committed' dropped-turn pattern.

    Failure mode: the model writes reasoning (``<scratch_pad>...`` or
    ``<think>...``) and then stops without emitting a tool call or an
    ``<output>`` block. ``extract_output`` strips the reasoning, returns
    empty, and the turn silently drops. Detecting it lets the server issue a
    kicker and retry once. Callers should also confirm no tool call fired this
    turn (the LangChain tool channel is separate from text content) — an empty
   
…[8505 chars omitted]…
en) or seen.startswith(vis)  # never diverges
        seen = vis
    assert seen == "Hello world"
    # The terminal extractor agrees with the final streamed text.
    assert extract_output(raw) == "Hello world"
graph/output_format.py:89:    Idempotent — real user content should never contain literal tag
tests/test_output_format.py:83:    """Real content never contains literal tag markers, so applying

The literal tag `<scratch_pad>` is discussed, and this is the rest of the answer.

==================================== ERRORS ====================================
__________________ ERROR collecting tests/test_confidence.py ___________________
ImportError while importing test module '/repo/tests/test_confidence.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_confidence.py:5: in <module>
    from a2a_handler import (
a2a_handler.py:44: in <module>
    import httpx

E   ModuleNotFoundError: No module named 'httpx'
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__ini
…[30 chars omitted]…
ng: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
ERROR tests/test_confidence.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
1 warning, 1 error in 0.48s

graph/agent.py:46:    # <learned_skills> injection). Build it when knowledge OR skills is active,
graph/config.py:9:``protolabs/<agent>`` alias pattern — retarget ``model.name`` in the
graph/config.py:61:    # historical <think>/<scratch_pad> blocks across turns.
graph/config.py:184:    # ``<config_dir>/skills``); shipped example skills live in ``config/skills``.
graph/config.py:192:    # ``<server>__<tool>`` so they can't shadow core tools. OFF by default —
graph/config.py:204:    # ``<config_dir>/plugins``); shipped examples live in ``plugins/``.
graph/config.py:218:    # here makes the A2A handler require ``Authorization: Bearer <token>``
graph/goals/controller.py:33:_GOAL_PLAN_RE = re.compile(r"<goal_plan>(.*?)</goal_plan>", re.IGNORECASE | re.DOTALL)
graph/goals/controller.py:84:            return ("Could not parse goal. Use `/goal <text>` or "
graph/goals/controller.py:122:     
…[8605 chars omitted]…
ONFIG_DIR>/skills/<slug>/SKILL.md``

graph/subagents/config.py:63:   What angles are worth covering? Note in <scratch_pad>.
graph/subagents/config.py:70:4. **Synthesize.** Compose a tight answer in <output>. Lead with
graph/subagents/config.py:84:  ("operator's notes from <date> say…") so the lead agent knows the
graph/subagents/config.py:93:<scratch_pad>, the final synthesis in <output>. Keep <output>

./graph/output_format.py:142:def extract_confidence(text: str) -> tuple[float | None, str | None]:
./server.py:38:    extract_confidence,
./server.py:1079:            confidence, explanation = extract_confidence(final_raw)
./server.py:1284:                # <confidence> tag (see graph/output_format.py::extract_confidence
./tests/test_confidence.py:14:from graph.output_format import extract_confidence, extract_output
./tests/test_confidence.py:32:# ── extract_confidence ────────────────────────────────────────────────────────
./tests/test_confidence.py:34:def test_extract_confidence_parses_score_and_explanation():
./tests/test_confidence.py:40:    conf, expl = extract_confidence(text)
./tests/test_confidence.py:45:def test_extract_confidence_absent_returns_none():
./tests/test_confidence.py:46:    conf, expl = extract_confidence("<output>no confidence here

([\\s\\S]*?)

The literal tag `<scratch_pad>` is discussed, and this is the rest of the answer.

` secret</scratch_pad><output>Visible

Literal `<scratch_pad>` text follows.

"""

from __future__ import annotations

from bisect import bisect_right
import logging
import re

log = logging.getLogger("protoagent.output_format")

OUTPUT_FORMAT_INSTRUCTIONS = """
    Given a *partial* (still-streaming) raw response, returns only the text

    inside the first (possibly still-open) ``<output>`` block, with reasoning
    stripped and any partial trailing tag held back — so a half-written
    ``

_ORPHAN_OUTPUT_OPEN_RE = re.compile(r"<output>([\s\S]*)$", re.IGNORECASE)


def stream_visible_output(raw: str) -> str:
    """The portion of the user-facing ``<output>`` that's safe to show mid-stream.

    Given a *partial* (still-streaming) raw response, returns only the text
    inside the first (possibly still-open) ``<output>`` block, with reasoning
    stripped and any partial trailing tag or inline-code span held back — so
    a half-written ``

./graph/output_format.py:28:from bisect import bisect_right
"""Structured output protocol for protoAgent — `<scratch_pad>` / `<output>` tags.

The model is instructed to wrap internal deliberation in ``<scratch_pad>``
and the user-facing answer in ``<output>``. Server-side, we parse those
tags and forward only the ``<output>`` content to consumers (A2A
artifacts, Gradio chat, subagent return values).

We deliberately do NOT parse the protocol mid-stream — chunk-boundary
tag splitting turned that into a state-machine rabbit hole and the
per-token text rendering consumers were doing didn't add real value.
Instead, ``_chat_langgraph_stream`` accumulates the model's tokens

silently while still emitting tool-start / tool-end status events, then
passes the complete text through ``extract_output`` once on the
terminal ``done`` frame. The consumer sees tool progress during the run
and the cle
…[2230 chars omitted]…
PHAN_SCRATCH_OPEN_RE = re.compile(r"<scratch_pad>[\s\S]*$", re.IGNORECASE)
_THINK_RE = re.compile(r"<think>[\s\S]*?</think>", re.IGNORECASE)
_ORPHAN_THINK_OPEN_RE = re.compile(r"<think>[\s\S]*$", re.IGNORECASE)
_ORPHAN_THINK_CLOSE_RE = re.compile(r"</think>\s*", re.IGNORECASE)
_CONFIDENCE_BLOCK_RE = re.compile(r"<confidence>[\s\S]*?</confidence>", re.IGNORECASE)
_CONFIDENCE_EXPL_BLOCK_RE = re.compile(

` marker first</scratch_pad>'
        '<output>The literal tag `<scratch_pad>` is part of this explanation.

./graph/output_format.py:28:from bisect import bisect_right
a2a_handler.py:37:import logging
cache.py:16:import logging
evals/verify.py:24:import logging
graph/cache_warmer.py:31:import logging
graph/config_io.py:28:import logging
graph/extensions/skills.py:21:import logging
graph/goals/controller.py:20:import logging
graph/goals/store.py:13:import logging
graph/goals/verifiers.py:24:import logging
graph/guardrails.py:23:import logging

`` block, with reasoning
    stripped and any partial trailing tag or inline-code span held back — so
 
…[521 chars omitted]…
still in ``<scratch_pad>`` (before ``<output>`` opens),
''')
p.write_text(s)
PY
awk 'length($0)>100 {print NR ":" length($0)}' graph/output_format.py; git diff --check
<returncode>0</returncode>
<output>

sed: can't read graph/goal_turn.py: No such file or directory
"""GoalController — control-message parsing + the goal decision loop.

Two responsibilities, both pure of any graph calls so they're unit-testable:

1. ``parse_control`` — interpret a ``/goal`` control message (set / status /
   clear) and mutate the store. Returns a reply string when the message *was* a
   command (the caller short-circuits the turn), else ``None``.

2. ``evaluate`` — run after the agent "stops" (terminal turn). Runs the goal's
   verifier and returns a ``Decision``: keep going with a continuation prompt,
   or finish (achieved / exhausted / unachievable).

The server invocation paths own the actual re-invocation loop; this class only
decides what should happen next.
"""

from __future__ import annotations

import json
import logging
import os
import re
from dataclasses import dataclass

from graph.goals.s
…[269 chars omitted]…
(.*?)</goal_plan>", re.IGNORECASE | re.DOTALL)
_GIVEUP_RE = re.compile(
    r"<goal_unachievable(?:\s+reason=\"([^\"]*)\")?\s*/?>", re.IGNORECASE
)


@dataclass
class Decision:
    action: str               # "continue" | "done"
    state: GoalState | None = None
    message: str | None = None   # continuation prompt (action == "continue")
    note: str = ""               # human-readable status note

71:_OUTPUT_OPEN_RE = re.compile(r"<output>", re.IGNORECASE)
72:_OUTPUT_CLOSE_RE = re.compile(r"

Talk about `<confidence>` and `</confidence>` here.

graph/output_format.py | 168 ++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 140 insertions(+), 28 deletions(-)
diff --git a/graph/output_format.py b/graph/output_format.py
index 79cc3fc..b1fbf6a 100644
--- a/graph/output_format.py
+++ b/graph/output_format.py
@@ -25,6 +25,7 @@ both the lead agent and subagent system prompts.
 
 from __future__ import annotations
 
+from bisect import bisect_right
 import logging

 import re
 
@@ -67,6 +68,9 @@ Optionally, after `

...........................                                              [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636
  /usr/local/lib/python3.12/site-packages/_pytest/config/__init__.py:1636: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
27 passed, 1 warning in 0.24s
 M graph/output_format.py

diff --git a/graph/output_format.py b/graph/output_format.py
index 79cc3fc..ff966cc 100644
--- a/graph/output_format.py
+++ b/graph/output_format.py
@@ -27,6 +27,7 @@ from __future__ import annotations
 
 import logging
 import re
+from bisect import bisect_right
 
 log = logging.getLogger("protoagent.output_format")
 

@@ -67,6 +68,9 @@ Optionally, after `

submitted.