fix(projekt-matching): break infinite alert loop (issue 2)

Root cause chain (evidence: 25+ failed traces on nproj/3022414):
straight quote in page text ends the guided-decoding JSON string
early -> grammar deadlock -> whitespace loop until context/token
limit -> chat_json fails 3x (~35 min) -> IMAP conn idles out ->
move_to_trash raises -> mail never trashed -> reprocessed forever.

- ingest/mailer: MailBox.ensure_alive() reconnect before finalize;
  move_to_trash failure falls back to flag_checked, run continues
- stages: page_text() feeds only div.content minus skill-tag cloud
  to the LLM and maps quote chars to apostrophes (the actual
  degeneration trigger, live-verified: extract now OK in 21 s)
- llm: requirements maxItems 40; max_tokens 8192 caps degenerate
  runs at ~100 s instead of ~10 min

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
tlg
2026-07-14 10:33:52 +02:00
parent f40a5bb0f8
commit 36eac66abc
8 changed files with 208 additions and 13 deletions

View File

@@ -1,7 +1,9 @@
"""vLLM (OpenAI-compatible) calls with response_format json_schema + German prompts.
max_tokens stays UNSET so nothing can truncate the final answer (65k
context window). Thinking is DISABLED via chat_template_kwargs: with the
max_tokens is capped at 8192: every valid answer fits comfortably, and an
uncapped run lets a degenerate generation grind for ~10 min before failing
(observed live: 100+-requirement JSON, broken at ~char 1725, 3 retries =
~35 min per cycle). Thinking is DISABLED via chat_template_kwargs: with the
vLLM qwen3 reasoning parser active, json_schema guided decoding plus
thinking degenerates — the run burns the entire context window
(finish_reason "length", ~63k completion tokens) and returns empty or
@@ -34,6 +36,7 @@ EXTRACT_SCHEMA = {
"contactPerson": {"type": ["string", "null"]},
"requirements": {
"type": "array",
"maxItems": 40,
"items": {
"type": "object",
"properties": {
@@ -139,6 +142,7 @@ z. B. vLLM, TGI, Triton" ab).
def _post(base, model, messages, schema):
body = {"model": model, "messages": messages, "temperature": 0.1,
"max_tokens": 8192,
"chat_template_kwargs": {"enable_thinking": False},
"response_format": {"type": "json_schema", "json_schema": {
"name": "out", "schema": schema}}}