Compare commits
4 Commits
72a3b178cc
...
main
| Author | SHA256 | Date | |
|---|---|---|---|
| 58b64c6d5f | |||
| 36eac66abc | |||
| f40a5bb0f8 | |||
| 49aed3b243 |
358
docs/projekt-matching-design.md
Normal file
358
docs/projekt-matching-design.md
Normal file
@@ -0,0 +1,358 @@
|
|||||||
|
# Projekt-Matching — Design Description & Experiences
|
||||||
|
|
||||||
|
*Companion to the [user manual](projekt-matching-user-manual.md). Built
|
||||||
|
2026-07-09 via spec-driven, subagent-executed development; merged to `main`
|
||||||
|
as `72a3b17`. Approved spec:
|
||||||
|
`docs/superpowers/specs/2026-07-09-langflow-projekt-matching-design.md`;
|
||||||
|
implementation plan (with deviations preamble):
|
||||||
|
`docs/superpowers/plans/2026-07-09-langflow-projekt-matching.md`.
|
||||||
|
Updated 2026-07-14 after the poison-mail incident (§4.6, commit `36eac66`).*
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Goal and constraints
|
||||||
|
|
||||||
|
Convert the interactive Claude skill `projekt-anlegen` (evaluate freelance
|
||||||
|
project postings against Dr.-Ing. Thomas Langer's CV, write results into
|
||||||
|
EspoCRM) into a fully automated Langflow workflow with these hard
|
||||||
|
constraints:
|
||||||
|
|
||||||
|
- **Local AI only** — vLLM endpoint on this host, model
|
||||||
|
`AxionML/Qwen3.5-9B-NVFP4`; no cloud LLM.
|
||||||
|
- **No human review pause** — the skill's mandatory chat review was replaced
|
||||||
|
by defaults (ambiguous offer type → `Projekt`, ambiguous buyer → agency)
|
||||||
|
and by after-the-fact quality monitoring in Langfuse.
|
||||||
|
- **Numbers are never LLM output** — match calculation, Misc rules
|
||||||
|
(start-date window, distance via Nominatim/Haversine, security-clearance
|
||||||
|
K.-o.) and the exact CRM description markdown are deterministic Python.
|
||||||
|
- **Gate:** Must-have-Match **> 85 %** (strictly greater) → CRM + mail;
|
||||||
|
otherwise reject silently (trace only).
|
||||||
|
- **Terminal-state rule:** every project ends in exactly one of
|
||||||
|
`created / rejected / duplicate / failed` on the **first** attempt; the
|
||||||
|
trigger mail then always moves to Trash; failures produce one aggregated
|
||||||
|
alert per trigger mail. No cross-cycle retries (only bounded in-run HTTP
|
||||||
|
retries) — deliberate protection against retry loops and inbox congestion.
|
||||||
|
- **Duplicate key:** canonical project URL (redirects followed, query
|
||||||
|
stripped) stored in and matched against `cProjektlink`.
|
||||||
|
|
||||||
|
## 2. Architecture
|
||||||
|
|
||||||
|
### 2.1 The central decision: logic in a package, Langflow as the runtime
|
||||||
|
|
||||||
|
All behavior lives in a plain-Python package **`projektmatch`** that is
|
||||||
|
unit-tested on the host (63 pytest tests) and copied unchanged into the
|
||||||
|
Langflow container. The seven Langflow custom components are ~30-line
|
||||||
|
wrappers that read configuration from Langflow global variables and call one
|
||||||
|
package function each. This kept the visual pipeline Langflow promises while
|
||||||
|
making every business rule testable without a running container — by far the
|
||||||
|
highest-leverage choice in the project: 8 of 16 plan tasks were pure TDD on
|
||||||
|
the package, and almost every live-integration bug was fixable by editing
|
||||||
|
tested package code and re-running one deploy script.
|
||||||
|
|
||||||
|
### 2.2 Runtime topology
|
||||||
|
|
||||||
|
```
|
||||||
|
systemd user timer (5 min) ── curl POST ──▶ /api/v1/webhook/pm-ingest
|
||||||
|
│
|
||||||
|
┌── Flow 1 "PM Ingest" ────────────────────────▼────────┐
|
||||||
|
│ Webhook → PMIngest → TextOutput │
|
||||||
|
│ PMIngest = ingest.run_ingest(): │
|
||||||
|
│ flock lock → IMAP poll (UNKEYWORD $ProjektChecked) │
|
||||||
|
│ → own-mail/non-project filter → split projects │
|
||||||
|
│ → canonical URL → CRM dedup → per NEW project: │
|
||||||
|
│ POST /api/v1/run/<flow2> (x-api-key) ───────────┼──┐
|
||||||
|
│ → aggregate → ONE alert mail on failures │ │
|
||||||
|
│ → move trigger mail to Trash → summary │ │
|
||||||
|
└───────────────────────────────────────────────────────┘ │
|
||||||
|
┌── Flow 2 "PM Projekt bewerten" ──────────────────────────▼┐
|
||||||
|
│ TextInput → PMFetch → PMExtract → PMMatch → PMRules │
|
||||||
|
│ → PMCrm → PMNotify → TextOutput │
|
||||||
|
│ ctx dict flows through; status ∈ ok/failed/rejected/ │
|
||||||
|
│ created; run_stage() catches exceptions into failed and │
|
||||||
|
│ short-circuits later stages; PMNotify runs UNCONDITIONALLY│
|
||||||
|
│ (sends mail only on created, posts Langfuse trace always) │
|
||||||
|
└───────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
Flow 1 is a single custom component rather than the spec sketch's three,
|
||||||
|
because IMAP connections and per-mail state cannot cross Langflow component
|
||||||
|
boundaries; `run_ingest()` is fully unit-tested instead. Flow 2 keeps the
|
||||||
|
six visible stages.
|
||||||
|
|
||||||
|
### 2.3 Module inventory
|
||||||
|
|
||||||
|
| Module | Responsibility |
|
||||||
|
|---|---|
|
||||||
|
| `rules.py` | Misc rules, half-up match calc, exact description markdown |
|
||||||
|
| `mailparse.py` | own-mail detection, project splitting (longest-anchor-title dedup by URL path), canonical URL |
|
||||||
|
| `espocrm.py` | CRM client: dedup searches, team lookup (never guessed), ensure-account/contact, unique naming, retries, `X-Status-Reason` surfacing |
|
||||||
|
| `llm.py` | vLLM calls (`response_format json_schema`, thinking disabled, `max_tokens` 8192), German prompts + JSON schemas (requirements capped at 40), retry-on-empty, match completeness retry |
|
||||||
|
| `mailer.py` | IMAP (custom keyword, Trash detection, MOVE+fallback, `ensure_alive()` reconnect), SMTP (STARTTLS, Date/Message-ID, Sent copy) |
|
||||||
|
| `stages.py` | Flow-2 stage functions + ctx state machine + CRM verify read-back; `page_text()` (project card minus tag cloud, quote sanitization) |
|
||||||
|
| `pm_trace.py` | one consolidated Langfuse trace per project via the ingestion API (never raises) |
|
||||||
|
| `ingest.py` | Flow-1 orchestration: lock, loop, dispatch, alert aggregation, trash with flag-checked fallback |
|
||||||
|
| `deploy/` | `deploy_files.sh` (rsync + safe restart), `build_flows.py` (variables, API key, programmatic flow construction), `setup_langfuse.py`, systemd units, webhook trigger |
|
||||||
|
| `tests/` | 63 unit tests + 5 e2e helper scripts |
|
||||||
|
|
||||||
|
### 2.4 Trust boundaries and secrets
|
||||||
|
|
||||||
|
IMAP/SMTP password and EspoCRM API key live in Langflow global variables
|
||||||
|
(encrypted in Postgres) and chmod-600 files under `~/.config/projekt-matching/`;
|
||||||
|
they never entered git (verified across all 31 commits in the final review).
|
||||||
|
Langfuse keys follow the pod script's pre-existing convention (tracked, local
|
||||||
|
loopback services only — an accepted, user-confirmed trade-off). The CV and
|
||||||
|
the CRM secrets file are owned by `tlg` and readable by `lwc` via
|
||||||
|
`setfacl -m u:lwc:r`.
|
||||||
|
|
||||||
|
## 3. Key design decisions and their rationale
|
||||||
|
|
||||||
|
1. **Canonical-URL dedup** (not name-based): deterministic, immune to title
|
||||||
|
edits; the same job at two agencies intentionally stays two opportunities.
|
||||||
|
2. **Delete-mail-always + alert** (revised mid-design from
|
||||||
|
"retry until success"): deterministic failures would otherwise loop every
|
||||||
|
5 minutes forever. The alert transfers responsibility to the human;
|
||||||
|
in-run HTTP retries (2–3 attempts) absorb transient blips.
|
||||||
|
3. **Deterministic scoring layer**: the LLM only classifies and rates;
|
||||||
|
Python computes. This survived the model's quality variance — when
|
||||||
|
ratings flickered, the numbers stayed honest and auditable.
|
||||||
|
4. **One custom Langfuse trace per project** (via the ingestion REST API, not
|
||||||
|
the SDK): stable contract, independent of Langflow's own auto-tracing,
|
||||||
|
filterable by outcome tags, and the object your annotation queue feeds on.
|
||||||
|
Posted *last* in `stage_notify`, unconditionally — even for failures.
|
||||||
|
5. **Programmatic flow construction** (`build_flows.py` assembles node/edge
|
||||||
|
JSON via `POST /api/v1/custom_component` templates): flows are
|
||||||
|
reproducible from git; the Langflow UI is never the source of truth.
|
||||||
|
6. **Everything idempotent**: variable upserts, flow delete-and-recreate,
|
||||||
|
API-key delete-and-recreate, file rsync — any deploy step can be re-run.
|
||||||
|
|
||||||
|
## 4. Experiences — what the plan got wrong and what we learned
|
||||||
|
|
||||||
|
The plan deliberately marked three "known unknowns" with verify-and-adapt
|
||||||
|
steps; live integration surfaced several more. Each item below is baked into
|
||||||
|
the code and worth knowing before touching the system.
|
||||||
|
|
||||||
|
### 4.1 vLLM / Qwen3.5 structured output
|
||||||
|
|
||||||
|
- **`guided_json` is silently ignored** by vLLM 0.22.1 — HTTP 200,
|
||||||
|
unconstrained prose output. The plan's guided_json-primary +
|
||||||
|
400-triggered-fallback design could therefore never fall back. Fixed by
|
||||||
|
using **`response_format {"type": "json_schema"}`** as the sole mechanism
|
||||||
|
(verified enforced: exact schema keys returned).
|
||||||
|
- **Reasoning + schema enforcement degenerate**: with the qwen3 reasoning
|
||||||
|
parser active, `json_schema` decoding sent the model into thinking loops
|
||||||
|
that consumed the full 63k context (`finish_reason: length`, empty
|
||||||
|
content). Root-cause fix: `chat_template_kwargs {"enable_thinking": false}`
|
||||||
|
for structured calls — extraction dropped from minutes/never to ~35 s.
|
||||||
|
A test locks this parameter in place.
|
||||||
|
- **Empty completions under load** still occur occasionally (shared GPU,
|
||||||
|
concurrent runs): `chat_json` retries empty/unparseable content up to 3×
|
||||||
|
with backoff; non-200 fails fast.
|
||||||
|
- **A straight `"` in the INPUT text breaks schema-guided decoding** (found
|
||||||
|
2026-07-14, §4.6): the model copies the quote verbatim into a JSON string
|
||||||
|
value, the grammar reads it as end-of-string, the model cannot continue
|
||||||
|
its sentence — and escapes into an **infinite whitespace loop** (whitespace
|
||||||
|
is always grammar-legal) until the token limit, `finish_reason: length`.
|
||||||
|
vLLM's request-level `structured_outputs.disable_any_whitespace` does NOT
|
||||||
|
help: the loop simply moves inside a string literal, where spaces are
|
||||||
|
legal. The robust fix is input-side: `stages.page_text()` maps all quote
|
||||||
|
characters (`„ “ ” " « » ‹ ›`) to `'` before prompting.
|
||||||
|
- The old belief "set max_tokens ≥ 1024 for reasoning models" first evolved
|
||||||
|
into "set no max_tokens at all and disable thinking where you enforce
|
||||||
|
schemas" — and then, after the whitespace-loop incident, into
|
||||||
|
**"cap max_tokens at 8192"**: every valid answer fits comfortably, and a
|
||||||
|
degenerate run now burns ~100 s instead of ~10 min before the retry.
|
||||||
|
|
||||||
|
### 4.2 Langflow 1.10 internals
|
||||||
|
|
||||||
|
- **`Component.ctx` is a reserved property** — a `DataInput(name="ctx")`
|
||||||
|
silently shadows the wired edge payload. All data inputs are named
|
||||||
|
`ctx_in`.
|
||||||
|
- **The custom-component loader resolves imports statically** — in-file
|
||||||
|
`sys.path.insert()` hacks don't work. First fix (a `.pth` file in the
|
||||||
|
container venv) turned out non-durable because the systemd `--new` unit
|
||||||
|
**recreates the container on every restart**, wiping ephemeral writes. The
|
||||||
|
durable fix bakes `PYTHONPATH=/app/langflow` into the container definition
|
||||||
|
in `create_pod_langflow.sh`.
|
||||||
|
- **Never `podman restart` a systemd-managed `--rm/--new` container** — it
|
||||||
|
can end up removed with the unit inactive. Always
|
||||||
|
`systemctl --user restart container-langflow_ctr.service`
|
||||||
|
(`deploy_files.sh` does this and waits for health).
|
||||||
|
- Webhook lives in the `input_output` category of `GET /api/v1/all` (not
|
||||||
|
`data`); `Data` outputs wire as type `JSON`; edge handles use the
|
||||||
|
`œ`-escaped JSON encoding. `build_flows.py` derives types from live
|
||||||
|
templates instead of hardcoding — the plan's explicit "verify against a
|
||||||
|
dumped existing flow first" step paid off.
|
||||||
|
- `POST /api/v1/api_key` accumulates same-named keys; the builder now
|
||||||
|
deletes existing `projekt-matching` keys before minting (list shape:
|
||||||
|
`{"api_keys": [...]}`).
|
||||||
|
- **`podman logs langflow_ctr` is useless** (a structlog recursion loop
|
||||||
|
floods it); the monitor/builds API is the working observability path.
|
||||||
|
- Auth split (from earlier experiments, confirmed): flow CRUD/variables need
|
||||||
|
the Bearer JWT from `/api/v1/auto_login`; `run` and webhook need
|
||||||
|
`x-api-key`; responses are gzip (`curl --compressed`).
|
||||||
|
- Global variables with `load_from_db=True` resolve **at run time** — the
|
||||||
|
Gate-3 recipient flip needed no restart.
|
||||||
|
|
||||||
|
### 4.3 Langfuse 3.195 (OSS)
|
||||||
|
|
||||||
|
- The **Organization Management API is EE-gated** (routes exist, return 403).
|
||||||
|
`setup_langfuse.py` falls back to a NextAuth session + tRPC calls to create
|
||||||
|
org/project/keys — fragile by nature, acceptable for one-shot setup, kept
|
||||||
|
behind an admin-API-first attempt.
|
||||||
|
- Score configs can be created via the public API with project keys;
|
||||||
|
**annotation queues are UI-only** (manual step, done).
|
||||||
|
- Score-config creation is not idempotent (archive-only API) — documented in
|
||||||
|
the script.
|
||||||
|
|
||||||
|
### 4.4 E-mail
|
||||||
|
|
||||||
|
- Outgoing `EmailMessage`s **need explicit `Date` and `Message-ID` headers**;
|
||||||
|
without them clients display 01/01/1970 (user-reported, fixed, and the fix
|
||||||
|
had to wait for a container restart — a long-lived process keeps the old
|
||||||
|
module in memory).
|
||||||
|
- The mail server (Dovecot) supports custom IMAP keywords (`PERMANENTFLAGS \*`)
|
||||||
|
and UID MOVE; the Trash folder is literally `Trash`. The `$ProjektChecked`
|
||||||
|
keyword implements "inspect non-project mail exactly once, touch nothing".
|
||||||
|
- A malformed FETCH must not kill the poll loop: hardened so an unreadable
|
||||||
|
mail is flagged and skipped (`summary.unreadable`), the connection is
|
||||||
|
closed in `finally` — otherwise one poison mail would silently stall the
|
||||||
|
pipeline every 5 minutes with no alert.
|
||||||
|
- **The IMAP server drops idle connections** during long Flow-2 dispatches
|
||||||
|
(30+ min in the degenerate case, §4.6). `run_ingest` therefore calls
|
||||||
|
`MailBox.ensure_alive()` (NOOP probe + transparent reconnect) before the
|
||||||
|
per-mail finalization, and a failing `move_to_trash` falls back to
|
||||||
|
`flag_checked` (`summary.trashFailed`) so the run continues — the
|
||||||
|
terminal-state rule survives a dead connection.
|
||||||
|
|
||||||
|
### 4.5 Prompt engineering for a 9B model
|
||||||
|
|
||||||
|
The extraction/matching prompts embed the skill's German calibration rules
|
||||||
|
verbatim (strictness: PoC ≠ production, "mehrjährig" literal, "wie z. B."
|
||||||
|
alternatives count). Two tuning iterations were needed against the reference
|
||||||
|
project (skill result: 86 % Must):
|
||||||
|
|
||||||
|
- **Signal words must bind to their own line only** — trailing unmarked lines
|
||||||
|
after a "von Vorteil" line were bleeding into Nice; as Must the reference
|
||||||
|
computes 18✅+3❌ of 21 = 85.71 → 86, exactly reproducing the skill.
|
||||||
|
- …with an **explicit exception for a literal "Nice-to-haves" heading**
|
||||||
|
(heading precedence stays above line-level signals) — added after review
|
||||||
|
caught the over-generalization.
|
||||||
|
- Residual variance: Must-match stable (86 in all post-fix runs); Nice-match
|
||||||
|
flips a few points run-to-run; Misc rows occasionally missing. The
|
||||||
|
deterministic layer plus the >85 gate make this tolerable; the annotation
|
||||||
|
queue exists to watch it.
|
||||||
|
|
||||||
|
### 4.6 Post-launch incident: the poison-mail loop (2026-07-13/14)
|
||||||
|
|
||||||
|
Four days after go-live the pipeline stalled for ~26 hours on one mail,
|
||||||
|
sending an identical `[Projekt-Match-Fehler]` alert every ~35 minutes. The
|
||||||
|
failure chain — each link necessary, none sufficient alone — is instructive:
|
||||||
|
|
||||||
|
1. **Trigger:** one posting (`nproj/3022414`) contained
|
||||||
|
`„ISTQB Certified Tester …"` — a typographic opening quote closed by a
|
||||||
|
straight ASCII `"`. Copied verbatim into a JSON string by the model, the
|
||||||
|
`"` ended the string for the guided-decoding grammar mid-sentence; the
|
||||||
|
model escaped into the whitespace loop (§4.1) and burned the full context
|
||||||
|
on every attempt (~10 min × 3 retries ≈ 35 min per cycle).
|
||||||
|
2. **Timeout mismatch:** the ingest dispatch timeout (1800 s) fired before
|
||||||
|
Flow 2 finished, producing a premature `Read timed out` failure.
|
||||||
|
3. **Trap:** during the long wait the IMAP connection idled out server-side.
|
||||||
|
The alert still went out (fresh SMTP connection), but `move_to_trash` on
|
||||||
|
the dead connection raised and aborted `run_ingest` **before** the mail
|
||||||
|
reached its terminal state — so the next tick reprocessed the same mail,
|
||||||
|
forever. 66 mails queued up behind it.
|
||||||
|
|
||||||
|
Fix (commit `36eac66`, TDD, live-verified): quote sanitization + feeding only
|
||||||
|
the project card minus the skill-tag cloud to the LLM (`page_text()`),
|
||||||
|
`maxItems: 40` on the requirements schema, `max_tokens: 8192`, and the
|
||||||
|
finalization hardening from §4.4. The poison extraction went from 25+
|
||||||
|
consecutive failures to a clean 19-requirement result in 21 s; the backlog
|
||||||
|
(67 mails) drained in 36 minutes with zero failure alerts and 9 new
|
||||||
|
opportunities.
|
||||||
|
|
||||||
|
Lessons: **terminal-state guarantees must hold on every path, including
|
||||||
|
"connection died mid-run"** — the guarantee is only as strong as its weakest
|
||||||
|
finalization step; and **schema-guided decoding turns bad input characters
|
||||||
|
into decoding pathologies**, so sanitize model input against the output
|
||||||
|
grammar, not just for size.
|
||||||
|
|
||||||
|
### 4.7 Process experiences
|
||||||
|
|
||||||
|
- **Subagent-driven development with per-task adversarial review** caught
|
||||||
|
real defects at every altitude: committed build artifacts, undeclared
|
||||||
|
dependencies, a read-back that verified presence instead of equality,
|
||||||
|
alert-send blocking the trash guarantee, the prompt over-generalization.
|
||||||
|
Two verification-only gates were audited as *evidence* (timestamps, IDs,
|
||||||
|
counts, alternative explanations) rather than trusted — one audit forced a
|
||||||
|
trace-level proof that an unexpected notification during Gate 2 was
|
||||||
|
genuine production traffic (it was: a real project, Must 100 %, evaluated
|
||||||
|
and filed while we were testing).
|
||||||
|
- **Live systems interleave with tests.** Real project mails arrived and
|
||||||
|
were processed during every e2e window. Distinguishing test artifacts from
|
||||||
|
production artifacts by canonical-URL tokens — and writing that discipline
|
||||||
|
into the helper scripts — was essential.
|
||||||
|
- The **progress ledger** (`.superpowers/sdd/progress.md`, git-ignored)
|
||||||
|
carried environment facts between tasks; most of its content graduated
|
||||||
|
into this document and the memory notes.
|
||||||
|
|
||||||
|
## 5. Verification summary
|
||||||
|
|
||||||
|
- **63 unit tests** (pure package, no network); TDD throughout (51 at
|
||||||
|
go-live, +1 Sent-copy fix, +11 poison-mail fix).
|
||||||
|
- **Component gates:** Flow 2 driven directly via API — consider project
|
||||||
|
stable at 86 % across repeated runs with full CRM field verification
|
||||||
|
(team, agency linking, contact, description byte-format); reject project
|
||||||
|
at 0 %, no writes.
|
||||||
|
- **Gate 1 (e2e consider):** re-sent original example-1 mail (5 projects) →
|
||||||
|
timer → project 4 created (86 %), notification with valid Date, trigger
|
||||||
|
mail in Trash, 5 traces (1 created / 4 rejected), zero unwanted entries.
|
||||||
|
- **Gate 2 (e2e reject):** example-2 mail → full table in trace, Must 0 %,
|
||||||
|
no CRM entry, no mail, mail in Trash.
|
||||||
|
- **Dedup:** re-sent Gate-1 mail → ingest summary
|
||||||
|
`{projects: 5, created: 0, rejected: 4, duplicate: 1}`, notification count
|
||||||
|
unchanged, no ` (2)` entry.
|
||||||
|
- **Gate 3 (production recipient):** variables flipped live (no restart),
|
||||||
|
opportunity re-created at 86 %, no mail in the design-phase inbox, SMTP
|
||||||
|
acceptance proven via trace error-absence, **receipt at
|
||||||
|
Thomas.Langer@destengs.com confirmed by Thomas**.
|
||||||
|
|
||||||
|
## 6. Known limitations & future work
|
||||||
|
|
||||||
|
- freelancermap-only detection (extension point: `mailparse.PROJECT_URL_RE`
|
||||||
|
+ splitting logic).
|
||||||
|
- Login-walled postings fail with an alert by design (no snippet fallback).
|
||||||
|
- Created-but-notification-failed edge case is recorded in the trace, not
|
||||||
|
re-sent (spec-accepted).
|
||||||
|
- Worst-case in-run LLM latency is bounded since `36eac66`
|
||||||
|
(`max_tokens: 8192` → a fully degenerate extraction costs ~100 s × 3
|
||||||
|
retries ≈ 6 min, comfortably below the 1800 s dispatch timeout). Should
|
||||||
|
the timeout ever fire anyway, the ingest side alerts while Flow 2 may
|
||||||
|
still finish server-side; dedup prevents damage, and the finalization
|
||||||
|
fallback (§4.4) keeps the trigger mail terminal.
|
||||||
|
- `parse_run_result` anchors on "any embedded JSON with a `status` key" —
|
||||||
|
robust today, but a Langflow upgrade that serializes intermediate ctx
|
||||||
|
objects into the run response would need a stricter anchor.
|
||||||
|
- niceMatch variance / Misc extraction variance: monitor via the
|
||||||
|
`projekt-matching-review` annotation queue; if quality drifts, tuning
|
||||||
|
lives in `llm.py`'s prompts and is regression-tested by
|
||||||
|
`tests/e2e/run_flow2.py` against the two reference projects.
|
||||||
|
|
||||||
|
## 7. Restore from a fresh clone (runbook)
|
||||||
|
|
||||||
|
1. `bash create_pod_langflow.sh` — pod, DBs, Langflow, Langfuse (bootstrap
|
||||||
|
admin + tracing keys baked in).
|
||||||
|
2. Ensure ACLs: `setfacl -m u:lwc:r` on the CV and
|
||||||
|
`.secrets/espocrm-api.md` (as `tlg`); create
|
||||||
|
`~/.config/projekt-matching/secrets.env` (0600) with `PM_IMAP_PASSWORD`.
|
||||||
|
3. `projekt-matching/deploy/deploy_files.sh` — package + vorgaben into the
|
||||||
|
data dir.
|
||||||
|
4. `set -a; source projekt-matching/deploy/secrets.local.env; set +a` then
|
||||||
|
`.venv/bin/python projekt-matching/deploy/build_flows.py` — variables,
|
||||||
|
API key, both flows, `~/.config/projekt-matching/env`.
|
||||||
|
5. `.venv/bin/python projekt-matching/deploy/setup_langfuse.py` — org,
|
||||||
|
project, keys, score configs (annotation queue: manual UI step); wire the
|
||||||
|
printed keys into the pod script and rerun it.
|
||||||
|
6. `cp projekt-matching/deploy/projekt-matching.{service,timer} ~/.config/systemd/user/`
|
||||||
|
then `systemctl --user daemon-reload && systemctl --user enable --now projekt-matching.timer`.
|
||||||
|
7. Optional clean start: `.venv/bin/python projekt-matching/tests/e2e/flag_inbox.py`
|
||||||
|
to baseline the current INBOX before the first tick.
|
||||||
232
docs/projekt-matching-user-manual.md
Normal file
232
docs/projekt-matching-user-manual.md
Normal file
@@ -0,0 +1,232 @@
|
|||||||
|
# Projekt-Matching — User Manual
|
||||||
|
|
||||||
|
*The automated pipeline that turns freelancermap project e-mails into evaluated
|
||||||
|
EspoCRM opportunities. Version of 2026-07-09 (all acceptance gates passed);
|
||||||
|
updated 2026-07-14 (robustness fixes: alert-loop bug resolved, outgoing mails
|
||||||
|
now also appear in the Sent folder).*
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. What the system does
|
||||||
|
|
||||||
|
Every 5 minutes, the system checks the mailbox `chancen@destengs.com` for new
|
||||||
|
e-mails containing freelancermap project descriptions. For every project it
|
||||||
|
finds, it:
|
||||||
|
|
||||||
|
1. resolves the project's canonical URL and checks whether it already exists
|
||||||
|
in the CRM (if yes: nothing happens — no duplicates),
|
||||||
|
2. fetches the full project description from freelancermap,
|
||||||
|
3. lets the **local AI** (vLLM, `AxionML/Qwen3.5-9B-NVFP4` — nothing leaves
|
||||||
|
your machine) extract the requirements and classify them as
|
||||||
|
**Must / Nice / Misc**,
|
||||||
|
4. lets the AI rate every Must/Nice requirement against your CV
|
||||||
|
(✅ covered / ❌ not covered / ❔ partial evidence),
|
||||||
|
5. computes the match values **deterministically in Python** (the AI never
|
||||||
|
invents the numbers): ✅ = 100, ❔ = 25, ❌ = 0, averaged per category,
|
||||||
|
6. decides: **Must-have-Match > 85 %** → the project is *considered*;
|
||||||
|
otherwise it is *rejected*,
|
||||||
|
7. for considered projects: creates the company (Account), contact person
|
||||||
|
(Contact) and the opportunity in EspoCRM — with the same description table
|
||||||
|
the old Claude skill produced — and sends you a notification e-mail,
|
||||||
|
8. records every project evaluation as a trace in Langfuse for your review,
|
||||||
|
9. moves the processed trigger e-mail to the server's **Trash** folder.
|
||||||
|
|
||||||
|
```
|
||||||
|
e-mail arrives every 5 min (systemd timer)
|
||||||
|
│ │
|
||||||
|
▼ ▼
|
||||||
|
INBOX (chancen@) ◀──── poll ──── Langflow "PM Ingest"
|
||||||
|
│ │ per project:
|
||||||
|
│ non-project mail: ▼
|
||||||
|
│ left untouched Langflow "PM Projekt bewerten"
|
||||||
|
│ fetch → AI extract → AI match CV
|
||||||
|
│ → deterministic match calc
|
||||||
|
▼ → gate: Must > 85 %?
|
||||||
|
processed mail │yes │no
|
||||||
|
→ Trash ▼ ▼
|
||||||
|
EspoCRM entry (nothing;
|
||||||
|
+ notification trace only)
|
||||||
|
e-mail
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Langfuse trace (always)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2. What you receive
|
||||||
|
|
||||||
|
### Notification (a project matched)
|
||||||
|
|
||||||
|
- **To:** `Thomas.Langer@destengs.com`
|
||||||
|
- **Subject:** `[Projekt-Match] Freelancermap — <Must> % — <Projektname>`
|
||||||
|
- **Body:** both match values and the direct link to the CRM opportunity
|
||||||
|
(`https://crm.creature-go.com/#Opportunity/view/<id>`).
|
||||||
|
|
||||||
|
### Alert (something went wrong)
|
||||||
|
|
||||||
|
- **To:** `Thomas.Langer@destengs.com`
|
||||||
|
- **Subject:** `[Projekt-Match-Fehler] Freelancermap — <n> Projekt(e)`
|
||||||
|
- **Body:** one entry per failed project with title, URL and the error reason.
|
||||||
|
|
||||||
|
**Important:** a failed project is **not retried**. The alert e-mail is your
|
||||||
|
record — if the project matters, evaluate it manually (the URL is in the
|
||||||
|
alert). This is deliberate: it prevents endless retry loops and inbox
|
||||||
|
congestion.
|
||||||
|
|
||||||
|
### What you will NOT see
|
||||||
|
|
||||||
|
- Rejected projects produce no e-mail and no CRM entry. Their full evaluation
|
||||||
|
(table + match values) is in the Langfuse trace.
|
||||||
|
- Non-project e-mails in the inbox are inspected once, invisibly marked with
|
||||||
|
an IMAP keyword (`$ProjektChecked`), and left untouched forever after.
|
||||||
|
- The system never processes its own notification/alert mails (recognized by
|
||||||
|
the subject prefix).
|
||||||
|
|
||||||
|
## 3. The CRM entry
|
||||||
|
|
||||||
|
Considered projects get the full structure the Claude skill used to create:
|
||||||
|
|
||||||
|
| Element | Rule |
|
||||||
|
|---|---|
|
||||||
|
| Opportunity name | project title; duplicate names get ` (2)`, ` (3)` … |
|
||||||
|
| Description | exact skill format: match line + `\| Nr. \| Kat. \| ❔ \| Anforderung \|` table, Must → Nice → Misc |
|
||||||
|
| cProjektlink | canonical project URL (also the duplicate-detection key) |
|
||||||
|
| Team | Projekt → `DesTEngS` · Arbeitnehmer-Angebot → `Arbeitnehmer` · ANÜ → `ANÜ` |
|
||||||
|
| Account | deduplicated by name; agency → `Reseller` + linked via *Über Agentur* (`cAccount1`), direct client → `Customer` + linked via `account` |
|
||||||
|
| Contact | created and linked when the posting names a person and a company |
|
||||||
|
|
||||||
|
Ambiguities the skill used to ask you about are resolved by defaults:
|
||||||
|
unclear offer type → `Projekt`; unclear buyer type → agency.
|
||||||
|
|
||||||
|
## 4. Reviewing quality in Langfuse
|
||||||
|
|
||||||
|
Open **http://127.0.0.1:8091** (login `admin@example.com` /
|
||||||
|
`langfuse-admin-pw` unless you changed it), project **projekt-matching**.
|
||||||
|
|
||||||
|
- Every evaluated project is one trace named **`projekt-match`**, tagged with
|
||||||
|
its outcome (`created` / `rejected` / `failed`). The trace shows the
|
||||||
|
canonical URL, the full requirements table, both match values and — for
|
||||||
|
created projects — the CRM link.
|
||||||
|
- **Your evaluation workflow:** open the annotation queue
|
||||||
|
**`projekt-matching-review`** and score each trace on the two dimensions:
|
||||||
|
- `extraction-correct` — were Must/Nice/Misc correctly derived from the
|
||||||
|
posting? (correct / partially-correct / wrong)
|
||||||
|
- `matching-correct` — are the ✅/❌/❔ ratings against your CV right?
|
||||||
|
- The scores are your quality record. Nothing acts on them automatically —
|
||||||
|
they tell you whether the local 9B model stays good enough over time.
|
||||||
|
|
||||||
|
Langflow's own execution traces (component level, LLM prompts/outputs) land
|
||||||
|
in the same project and are useful for debugging.
|
||||||
|
|
||||||
|
## 5. Everyday operations
|
||||||
|
|
||||||
|
All commands run as user `lwc` on this host.
|
||||||
|
|
||||||
|
### Is it running?
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl --user list-timers | grep projekt-matching # next/last tick
|
||||||
|
curl -s http://127.0.0.1:8090/health # Langflow: {"status":"ok"}
|
||||||
|
journalctl --user -u projekt-matching.service -n 5 # last trigger output
|
||||||
|
```
|
||||||
|
|
||||||
|
To see what recent runs did, use the Langfuse traces (section 4). Do **not**
|
||||||
|
use `podman logs langflow_ctr` — it is flooded by a known harmless logging
|
||||||
|
loop and tells you nothing.
|
||||||
|
|
||||||
|
### Pause / resume the automation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl --user stop projekt-matching.timer # pause (mail queues up safely in INBOX)
|
||||||
|
systemctl --user start projekt-matching.timer # resume; next tick processes the backlog
|
||||||
|
```
|
||||||
|
|
||||||
|
### Restart Langflow (e.g. after a hang)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl --user restart container-langflow_ctr.service
|
||||||
|
```
|
||||||
|
|
||||||
|
**Never** use `podman restart langflow_ctr` — with the systemd-managed
|
||||||
|
container this can leave the container removed and the pipeline dead.
|
||||||
|
|
||||||
|
### Update your CV or the Rahmenbedingungen
|
||||||
|
|
||||||
|
Edit the originals in `/home/tlg/mkt/bewerb/vorgaben/`, then:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
/home/lwc/bin/projekt-matching/deploy/deploy_files.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
The script copies the files, redeploys the code, restarts Langflow safely and
|
||||||
|
waits until it is healthy again (≈ 1 minute).
|
||||||
|
|
||||||
|
### Change the notification recipient or the 85 % threshold
|
||||||
|
|
||||||
|
Both live in Langflow **global variables** (UI: http://127.0.0.1:8090 →
|
||||||
|
Settings → Global Variables, or via API):
|
||||||
|
`PM_NOTIFY_TO`, `PM_ALERT_TO`, `PM_THRESHOLD` (also: `PM_VLLM_BASE`,
|
||||||
|
`PM_VLLM_MODEL`, `PM_ESPO_BASE`). Changes take effect on the next run — no
|
||||||
|
restart needed.
|
||||||
|
|
||||||
|
### After a reboot
|
||||||
|
|
||||||
|
Everything is systemd-managed and starts automatically (`pod-langflow_pod`
|
||||||
|
for the services, `projekt-matching.timer` for the polling). Nothing to do.
|
||||||
|
|
||||||
|
## 6. Troubleshooting
|
||||||
|
|
||||||
|
| Symptom | Likely cause / what to do |
|
||||||
|
|---|---|
|
||||||
|
| No mails processed, INBOX grows | `systemctl --user list-timers` — timer active? Langflow healthy (`curl …/health`)? Restart per section 5. Mails are never lost: the next successful tick processes the backlog. |
|
||||||
|
| Alert mail: `Abruf fehlgeschlagen` / `Seitentext zu kurz` | freelancermap page unreachable or behind a login wall. Evaluate that project manually via the URL in the alert. |
|
||||||
|
| Alert mail: `vLLM HTTP …` / `LLM lieferte kein JSON` / `Read timed out` | the local AI was down or overloaded. Occasional single failures are absorbed by built-in retries; if alerts pile up, check the vLLM service on port 8081. |
|
||||||
|
| The **same** alert arrives again and again for the same project | this was a bug (fixed 2026-07-14): one poisonous posting could block the pipeline and repeat its alert every ~35 min. A failed project is now reliably final — if you ever see identical repeating alerts again, restart Langflow (section 5) and report it. |
|
||||||
|
| Alert mail: `Team … nicht gefunden` | the CRM user `cowork-api` lost read access to Teams, or a team was renamed. Fix in EspoCRM admin. |
|
||||||
|
| A project you expected is missing | check Langfuse: if its trace says `rejected`, the Must-match was ≤ 85 % — the table shows why. If there is no trace at all, check the alert mails. |
|
||||||
|
| Duplicate opportunity suspected | duplicates are keyed on the canonical URL (`cProjektlink`). The same project posted by two different agencies has two different URLs and is intentionally kept as two opportunities. |
|
||||||
|
| Everything looks dead after an experiment | `systemctl --user start container-langflow_ctr.service`, then check health. |
|
||||||
|
|
||||||
|
## 7. Test utilities (use with care)
|
||||||
|
|
||||||
|
In `/home/lwc/bin/projekt-matching/tests/e2e/` (run with the project venv,
|
||||||
|
secrets sourced: `set -a && source deploy/secrets.local.env && set +a`):
|
||||||
|
|
||||||
|
- `check_state.py [token]` — shows CRM hits for a URL token plus the last
|
||||||
|
INBOX/Trash subjects. Safe, read-only.
|
||||||
|
- `run_flow2.py <canonical-url> <title>` — evaluates one project directly
|
||||||
|
(bypasses e-mail). Writes to the CRM if it matches!
|
||||||
|
- `send_test_mail.py "<subject>" [fallback-url]` — re-sends a stored example
|
||||||
|
mail to yourself to trigger the full pipeline.
|
||||||
|
- `cleanup_crm.py [tokens…]` — **deletes** opportunities whose cProjektlink
|
||||||
|
matches the tokens. Default tokens are the two historical test projects.
|
||||||
|
Never run it with a token that matches real opportunities.
|
||||||
|
- `flag_inbox.py` — marks every current INBOX mail as "already checked"
|
||||||
|
(used once at go-live; rarely needed again).
|
||||||
|
|
||||||
|
## 8. Where things live
|
||||||
|
|
||||||
|
| What | Where |
|
||||||
|
|---|---|
|
||||||
|
| Code + tests | `/home/lwc/bin/projekt-matching/` (git) |
|
||||||
|
| Flows (source of truth) | built by `deploy/build_flows.py` into Langflow ("PM Ingest", "PM Projekt bewerten") |
|
||||||
|
| Deployed runtime copy | `~/.local/share/langflow_pod/langflow-data/` (`/app/langflow` in-container) |
|
||||||
|
| Secrets | `~/.config/projekt-matching/` (0600) + Langflow global variables (encrypted); EspoCRM key sourced from `/home/tlg/mkt/bewerb/.secrets/espocrm-api.md` |
|
||||||
|
| systemd units | `~/.config/systemd/user/projekt-matching.{service,timer}` |
|
||||||
|
| Services | Langflow http://127.0.0.1:8090 · Langfuse http://127.0.0.1:8091 · vLLM http://127.0.0.1:8081 · pod script `/home/lwc/bin/create_pod_langflow.sh` |
|
||||||
|
| Design details | `docs/projekt-matching-design.md`, spec/plan under `docs/superpowers/` |
|
||||||
|
|
||||||
|
## 9. Known limitations
|
||||||
|
|
||||||
|
- **freelancermap only.** Mails from other portals are ignored (left in the
|
||||||
|
inbox untouched). Extending detection is a small code change in
|
||||||
|
`projektmatch/mailparse.py`.
|
||||||
|
- **Login-protected postings fail** with an alert (no page text, no snippet
|
||||||
|
fallback — a snippet cannot support honest matching).
|
||||||
|
- **Nice-to-have-Match varies a few points** between runs of the same project
|
||||||
|
(single ❔/✅ flips by the 9B model). The Must-match — the gate — has been
|
||||||
|
stable in all verification runs. Watch this in the annotation queue.
|
||||||
|
- **Rare edge case:** if the CRM write succeeds but the notification mail
|
||||||
|
fails to send, the opportunity exists but no mail arrives and none is
|
||||||
|
re-sent. The Langfuse trace records it (`error: notify: …`).
|
||||||
|
- Misc rows (start date, workload, …) are occasionally missing from the
|
||||||
|
extraction; they never influence the match values.
|
||||||
@@ -119,6 +119,10 @@ def run_ingest(cfg, mailbox=None, dispatch=None, espo=None, resolve=None):
|
|||||||
or f"unbekannter Status: {result.get('status')!r}")
|
or f"unbekannter Status: {result.get('status')!r}")
|
||||||
results.append(result)
|
results.append(result)
|
||||||
summary[result["status"]] += 1
|
summary[result["status"]] += 1
|
||||||
|
try:
|
||||||
|
box.ensure_alive() # Flow-2-Dispatches können >30 min dauern — Server-Idle-Timeout
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass # Alert/Trash unten haben eigene Fallbacks
|
||||||
failed = [r for r in results if r["status"] == "failed"]
|
failed = [r for r in results if r["status"] == "failed"]
|
||||||
if failed:
|
if failed:
|
||||||
lines = "\n\n".join(
|
lines = "\n\n".join(
|
||||||
@@ -134,7 +138,14 @@ def run_ingest(cfg, mailbox=None, dispatch=None, espo=None, resolve=None):
|
|||||||
"werden:\n\n" + lines + "\n")
|
"werden:\n\n" + lines + "\n")
|
||||||
except Exception: # noqa: BLE001
|
except Exception: # noqa: BLE001
|
||||||
summary["alertFailed"] = summary.get("alertFailed", 0) + 1
|
summary["alertFailed"] = summary.get("alertFailed", 0) + 1
|
||||||
|
try:
|
||||||
box.move_to_trash(uid)
|
box.move_to_trash(uid)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
summary["trashFailed"] = summary.get("trashFailed", 0) + 1
|
||||||
|
try:
|
||||||
|
box.flag_checked(uid) # nie erneut verarbeiten — Terminal-Zustand
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass # nächster Lauf räumt auf
|
||||||
return summary
|
return summary
|
||||||
finally:
|
finally:
|
||||||
box.close()
|
box.close()
|
||||||
|
|||||||
@@ -1,7 +1,9 @@
|
|||||||
"""vLLM (OpenAI-compatible) calls with response_format json_schema + German prompts.
|
"""vLLM (OpenAI-compatible) calls with response_format json_schema + German prompts.
|
||||||
|
|
||||||
max_tokens stays UNSET so nothing can truncate the final answer (65k
|
max_tokens is capped at 8192: every valid answer fits comfortably, and an
|
||||||
context window). Thinking is DISABLED via chat_template_kwargs: with the
|
uncapped run lets a degenerate generation grind for ~10 min before failing
|
||||||
|
(observed live: 100+-requirement JSON, broken at ~char 1725, 3 retries =
|
||||||
|
~35 min per cycle). Thinking is DISABLED via chat_template_kwargs: with the
|
||||||
vLLM qwen3 reasoning parser active, json_schema guided decoding plus
|
vLLM qwen3 reasoning parser active, json_schema guided decoding plus
|
||||||
thinking degenerates — the run burns the entire context window
|
thinking degenerates — the run burns the entire context window
|
||||||
(finish_reason "length", ~63k completion tokens) and returns empty or
|
(finish_reason "length", ~63k completion tokens) and returns empty or
|
||||||
@@ -34,6 +36,7 @@ EXTRACT_SCHEMA = {
|
|||||||
"contactPerson": {"type": ["string", "null"]},
|
"contactPerson": {"type": ["string", "null"]},
|
||||||
"requirements": {
|
"requirements": {
|
||||||
"type": "array",
|
"type": "array",
|
||||||
|
"maxItems": 40,
|
||||||
"items": {
|
"items": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
@@ -139,6 +142,7 @@ z. B. vLLM, TGI, Triton" ab).
|
|||||||
|
|
||||||
def _post(base, model, messages, schema):
|
def _post(base, model, messages, schema):
|
||||||
body = {"model": model, "messages": messages, "temperature": 0.1,
|
body = {"model": model, "messages": messages, "temperature": 0.1,
|
||||||
|
"max_tokens": 8192,
|
||||||
"chat_template_kwargs": {"enable_thinking": False},
|
"chat_template_kwargs": {"enable_thinking": False},
|
||||||
"response_format": {"type": "json_schema", "json_schema": {
|
"response_format": {"type": "json_schema", "json_schema": {
|
||||||
"name": "out", "schema": schema}}}
|
"name": "out", "schema": schema}}}
|
||||||
|
|||||||
@@ -16,12 +16,31 @@ KEYWORD = "$ProjektChecked"
|
|||||||
|
|
||||||
class MailBox:
|
class MailBox:
|
||||||
def __init__(self, user, password, conn=None):
|
def __init__(self, user, password, conn=None):
|
||||||
self.conn = conn or imaplib.IMAP4_SSL(IMAP_HOST, IMAP_PORT)
|
self.user, self.password = user, password
|
||||||
if conn is None:
|
self.conn = conn or self._connect()
|
||||||
self.conn.login(user, password)
|
|
||||||
self.conn.select("INBOX")
|
self.conn.select("INBOX")
|
||||||
self._trash = None
|
self._trash = None
|
||||||
|
|
||||||
|
def _connect(self):
|
||||||
|
conn = imaplib.IMAP4_SSL(IMAP_HOST, IMAP_PORT)
|
||||||
|
conn.login(self.user, self.password)
|
||||||
|
return conn
|
||||||
|
|
||||||
|
def ensure_alive(self):
|
||||||
|
"""Reconnect if the server dropped the connection (idle timeout)."""
|
||||||
|
try:
|
||||||
|
typ, _ = self.conn.noop()
|
||||||
|
if typ == "OK":
|
||||||
|
return
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
self.conn.logout()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass
|
||||||
|
self.conn = self._connect()
|
||||||
|
self.conn.select("INBOX")
|
||||||
|
|
||||||
def unchecked_uids(self):
|
def unchecked_uids(self):
|
||||||
typ, data = self.conn.uid("SEARCH", None, f"UNKEYWORD {KEYWORD}")
|
typ, data = self.conn.uid("SEARCH", None, f"UNKEYWORD {KEYWORD}")
|
||||||
if typ != "OK" or not data or not data[0]:
|
if typ != "OK" or not data or not data[0]:
|
||||||
@@ -71,6 +90,39 @@ class MailBox:
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _sent_folder(conn):
|
||||||
|
"""Find the server's Sent folder via special-use flag, fallback names."""
|
||||||
|
typ, data = conn.list()
|
||||||
|
candidates = []
|
||||||
|
for raw in data or []:
|
||||||
|
line = raw.decode() if isinstance(raw, bytes) else str(raw)
|
||||||
|
name = line.rsplit(" ", 1)[-1].strip('"')
|
||||||
|
if "\\Sent" in line.split(")")[0]:
|
||||||
|
return name
|
||||||
|
candidates.append(name)
|
||||||
|
for cand in ("Sent", "INBOX.Sent"):
|
||||||
|
if cand in candidates:
|
||||||
|
return cand
|
||||||
|
return "Sent"
|
||||||
|
|
||||||
|
|
||||||
|
def append_sent(user, password, msg, conn=None):
|
||||||
|
"""Store a copy of an outgoing message in the Sent folder (best effort)."""
|
||||||
|
own = conn is None
|
||||||
|
conn = conn or imaplib.IMAP4_SSL(IMAP_HOST, IMAP_PORT)
|
||||||
|
try:
|
||||||
|
if own:
|
||||||
|
conn.login(user, password)
|
||||||
|
folder = _sent_folder(conn)
|
||||||
|
conn.append(folder, "(\\Seen)", None, msg.as_bytes())
|
||||||
|
finally:
|
||||||
|
if own:
|
||||||
|
try:
|
||||||
|
conn.logout()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
def send_mail(user, password, to, subject, body):
|
def send_mail(user, password, to, subject, body):
|
||||||
msg = EmailMessage()
|
msg = EmailMessage()
|
||||||
msg["From"], msg["To"], msg["Subject"] = user, to, subject
|
msg["From"], msg["To"], msg["Subject"] = user, to, subject
|
||||||
@@ -81,3 +133,7 @@ def send_mail(user, password, to, subject, body):
|
|||||||
server.starttls(context=ssl.create_default_context())
|
server.starttls(context=ssl.create_default_context())
|
||||||
server.login(user, password)
|
server.login(user, password)
|
||||||
server.send_message(msg)
|
server.send_message(msg)
|
||||||
|
try:
|
||||||
|
append_sent(user, password, msg)
|
||||||
|
except Exception: # noqa: BLE001 — copy is best effort; the mail went out
|
||||||
|
pass
|
||||||
|
|||||||
@@ -20,6 +20,7 @@ TEAM_BY_OFFER = {"Projekt": "DesTEngS",
|
|||||||
"Arbeitnehmer-Angebot": "Arbeitnehmer",
|
"Arbeitnehmer-Angebot": "Arbeitnehmer",
|
||||||
"ANÜ": "ANÜ"}
|
"ANÜ": "ANÜ"}
|
||||||
MIN_PAGE_CHARS = 200
|
MIN_PAGE_CHARS = 200
|
||||||
|
QUOTES = str.maketrans({c: "'" for c in "„“”\"«»‹›"})
|
||||||
|
|
||||||
|
|
||||||
def run_stage(name, fn, ctx, cfg, **kw):
|
def run_stage(name, fn, ctx, cfg, **kw):
|
||||||
@@ -32,6 +33,25 @@ def run_stage(name, fn, ctx, cfg, **kw):
|
|||||||
return ctx
|
return ctx
|
||||||
|
|
||||||
|
|
||||||
|
def page_text(html):
|
||||||
|
"""Project text for the LLM: header + description, WITHOUT the skill-tag
|
||||||
|
cloud (div.project-body-badges, ~100 entries on some pages — blows the
|
||||||
|
requirements list up until the model emits broken JSON) and without
|
||||||
|
site chrome. Falls back to whole-page text if div.content is missing."""
|
||||||
|
soup = BeautifulSoup(html, "html.parser")
|
||||||
|
for tag in soup(["script", "style", "noscript"]):
|
||||||
|
tag.decompose()
|
||||||
|
root = soup.find("div", class_="content") or soup
|
||||||
|
for tag in root.find_all(class_="project-body-badges"):
|
||||||
|
tag.decompose()
|
||||||
|
# Anführungszeichen -> Apostroph: ein wörtlich übernommenes " beendet den
|
||||||
|
# JSON-String vorzeitig und treibt guided decoding in eine Whitespace-
|
||||||
|
# Endlosschleife (finish_reason "length", live verifiziert an nproj/3022414)
|
||||||
|
return "\n".join(line for line in
|
||||||
|
root.get_text("\n", strip=True).splitlines()
|
||||||
|
if line.strip()).translate(QUOTES)
|
||||||
|
|
||||||
|
|
||||||
def stage_fetch(ctx, cfg):
|
def stage_fetch(ctx, cfg):
|
||||||
last = None
|
last = None
|
||||||
for attempt in range(3):
|
for attempt in range(3):
|
||||||
@@ -39,12 +59,7 @@ def stage_fetch(ctx, cfg):
|
|||||||
resp = requests.get(ctx["canonical"], timeout=30,
|
resp = requests.get(ctx["canonical"], timeout=30,
|
||||||
headers={"User-Agent": BROWSER_UA})
|
headers={"User-Agent": BROWSER_UA})
|
||||||
resp.raise_for_status()
|
resp.raise_for_status()
|
||||||
soup = BeautifulSoup(resp.text, "html.parser")
|
text = page_text(resp.text)
|
||||||
for tag in soup(["script", "style", "noscript"]):
|
|
||||||
tag.decompose()
|
|
||||||
text = "\n".join(line for line in
|
|
||||||
soup.get_text("\n", strip=True).splitlines()
|
|
||||||
if line.strip())
|
|
||||||
if len(text) < MIN_PAGE_CHARS:
|
if len(text) < MIN_PAGE_CHARS:
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
f"Seitentext zu kurz ({len(text)} Zeichen) — "
|
f"Seitentext zu kurz ({len(text)} Zeichen) — "
|
||||||
|
|||||||
@@ -156,6 +156,63 @@ def test_close_called_even_if_unchecked_uids_raises(tmp_path):
|
|||||||
box.close.assert_called_once()
|
box.close.assert_called_once()
|
||||||
|
|
||||||
|
|
||||||
|
def test_ensure_alive_called_before_finalize(tmp_path):
|
||||||
|
mails = {"1": mail("Mail", PROJECT_HTML)}
|
||||||
|
summary, box, sent = run(tmp_path, mails,
|
||||||
|
dispatch_result={"status": "failed",
|
||||||
|
"error": "Read timed out"})
|
||||||
|
names = [c[0] for c in box.method_calls]
|
||||||
|
assert "ensure_alive" in names
|
||||||
|
assert names.index("ensure_alive") < names.index("move_to_trash")
|
||||||
|
box.move_to_trash.assert_called_once_with("1")
|
||||||
|
assert len(sent) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_trash_failure_falls_back_to_flag_and_continues(tmp_path):
|
||||||
|
mails = {"1": mail("Mail A", PROJECT_HTML),
|
||||||
|
"2": mail("Mail B", PROJECT_HTML)}
|
||||||
|
box = mock.Mock()
|
||||||
|
box.unchecked_uids.return_value = ["1", "2"]
|
||||||
|
box.fetch.side_effect = lambda uid: mails[uid]
|
||||||
|
box.move_to_trash.side_effect = [OSError("socket error: EOF"), None]
|
||||||
|
espo = mock.Mock()
|
||||||
|
espo.find_opportunity_by_link.return_value = None
|
||||||
|
with mock.patch("projektmatch.ingest.mailparse.bodies",
|
||||||
|
side_effect=lambda m: (m["html"], "")), \
|
||||||
|
mock.patch("projektmatch.ingest.mailer.send_mail", lambda *a: None):
|
||||||
|
summary = ingest.run_ingest(
|
||||||
|
cfg(tmp_path), mailbox=box, espo=espo,
|
||||||
|
resolve=lambda url: "https://www.freelancermap.de/projekt/eins",
|
||||||
|
dispatch=lambda item: {"status": "created"})
|
||||||
|
assert summary["trashFailed"] == 1
|
||||||
|
assert summary["created"] == 2
|
||||||
|
box.flag_checked.assert_called_once_with("1")
|
||||||
|
assert box.move_to_trash.call_count == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_dead_connection_everywhere_still_finishes_run(tmp_path):
|
||||||
|
mails = {"1": mail("Mail A", PROJECT_HTML),
|
||||||
|
"2": mail("Mail B", PROJECT_HTML)}
|
||||||
|
box = mock.Mock()
|
||||||
|
box.unchecked_uids.return_value = ["1", "2"]
|
||||||
|
box.fetch.side_effect = lambda uid: mails[uid]
|
||||||
|
box.ensure_alive.side_effect = OSError("server unreachable")
|
||||||
|
box.move_to_trash.side_effect = OSError("socket error: EOF")
|
||||||
|
box.flag_checked.side_effect = OSError("socket error: EOF")
|
||||||
|
espo = mock.Mock()
|
||||||
|
espo.find_opportunity_by_link.return_value = None
|
||||||
|
with mock.patch("projektmatch.ingest.mailparse.bodies",
|
||||||
|
side_effect=lambda m: (m["html"], "")), \
|
||||||
|
mock.patch("projektmatch.ingest.mailer.send_mail", lambda *a: None):
|
||||||
|
summary = ingest.run_ingest(
|
||||||
|
cfg(tmp_path), mailbox=box, espo=espo,
|
||||||
|
resolve=lambda url: "https://www.freelancermap.de/projekt/eins",
|
||||||
|
dispatch=lambda item: {"status": "created"})
|
||||||
|
assert summary["trashFailed"] == 2
|
||||||
|
assert summary["created"] == 2
|
||||||
|
box.close.assert_called_once()
|
||||||
|
|
||||||
|
|
||||||
def test_parse_run_result_finds_status_json():
|
def test_parse_run_result_finds_status_json():
|
||||||
inner = json.dumps({"status": "created", "mustMatch": 90})
|
inner = json.dumps({"status": "created", "mustMatch": 90})
|
||||||
payload = {"outputs": [{"outputs": [{"results": {"text": {
|
payload = {"outputs": [{"outputs": [{"results": {"text": {
|
||||||
|
|||||||
@@ -24,11 +24,16 @@ def test_chat_json_strips_think_and_parses():
|
|||||||
{"type": "object"})
|
{"type": "object"})
|
||||||
assert out == {"a": 1}
|
assert out == {"a": 1}
|
||||||
body = p.call_args.kwargs["json"]
|
body = p.call_args.kwargs["json"]
|
||||||
assert "max_tokens" not in body
|
assert body["max_tokens"] == 8192
|
||||||
assert body["chat_template_kwargs"] == {"enable_thinking": False}
|
assert body["chat_template_kwargs"] == {"enable_thinking": False}
|
||||||
assert body["response_format"]["json_schema"]["schema"] == {"type": "object"}
|
assert body["response_format"]["json_schema"]["schema"] == {"type": "object"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_extract_schema_caps_requirements_list():
|
||||||
|
reqs = llm.EXTRACT_SCHEMA["properties"]["requirements"]
|
||||||
|
assert reqs["maxItems"] == 40
|
||||||
|
|
||||||
|
|
||||||
def test_chat_json_strips_markdown_fences():
|
def test_chat_json_strips_markdown_fences():
|
||||||
content = "```json\n{\"a\": 2}\n```"
|
content = "```json\n{\"a\": 2}\n```"
|
||||||
with mock.patch("projektmatch.llm.requests.post", fake_post([content])):
|
with mock.patch("projektmatch.llm.requests.post", fake_post([content])):
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import email
|
import email
|
||||||
|
from email.message import EmailMessage
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
from unittest import mock
|
from unittest import mock
|
||||||
@@ -62,7 +63,8 @@ def test_fetch_raises_on_malformed_ok_response():
|
|||||||
|
|
||||||
|
|
||||||
def test_send_mail_starttls():
|
def test_send_mail_starttls():
|
||||||
with mock.patch("projektmatch.mailer.smtplib.SMTP") as smtp:
|
with mock.patch("projektmatch.mailer.smtplib.SMTP") as smtp, \
|
||||||
|
mock.patch("projektmatch.mailer.append_sent") as append_sent:
|
||||||
server = smtp.return_value.__enter__.return_value
|
server = smtp.return_value.__enter__.return_value
|
||||||
mailer.send_mail("u@x", "pw", "to@x", "Subj", "Body")
|
mailer.send_mail("u@x", "pw", "to@x", "Subj", "Body")
|
||||||
server.starttls.assert_called_once()
|
server.starttls.assert_called_once()
|
||||||
@@ -71,3 +73,52 @@ def test_send_mail_starttls():
|
|||||||
assert msg["Subject"] == "Subj" and msg["To"] == "to@x"
|
assert msg["Subject"] == "Subj" and msg["To"] == "to@x"
|
||||||
assert msg["Date"]
|
assert msg["Date"]
|
||||||
assert msg["Message-ID"].endswith("@destengs.com>")
|
assert msg["Message-ID"].endswith("@destengs.com>")
|
||||||
|
append_sent.assert_called_once_with("u@x", "pw", msg)
|
||||||
|
|
||||||
|
|
||||||
|
def test_send_mail_survives_append_failure():
|
||||||
|
with mock.patch("projektmatch.mailer.smtplib.SMTP") as smtp, \
|
||||||
|
mock.patch("projektmatch.mailer.append_sent",
|
||||||
|
side_effect=OSError("imap down")):
|
||||||
|
mailer.send_mail("u@x", "pw", "to@x", "Subj", "Body")
|
||||||
|
|
||||||
|
|
||||||
|
def test_ensure_alive_keeps_healthy_connection():
|
||||||
|
conn = fake_conn()
|
||||||
|
conn.noop.return_value = ("OK", [b""])
|
||||||
|
box = mailer.MailBox("u", "p", conn=conn)
|
||||||
|
box.ensure_alive()
|
||||||
|
assert box.conn is conn
|
||||||
|
conn.noop.assert_called_once()
|
||||||
|
|
||||||
|
|
||||||
|
def test_ensure_alive_reconnects_on_dead_connection():
|
||||||
|
conn = fake_conn()
|
||||||
|
conn.noop.side_effect = OSError("connection closed")
|
||||||
|
box = mailer.MailBox("u", "p", conn=conn)
|
||||||
|
fresh = fake_conn()
|
||||||
|
with mock.patch("projektmatch.mailer.imaplib.IMAP4_SSL",
|
||||||
|
return_value=fresh):
|
||||||
|
box.ensure_alive()
|
||||||
|
assert box.conn is fresh
|
||||||
|
fresh.login.assert_called_once_with("u", "p")
|
||||||
|
fresh.select.assert_called_once_with("INBOX")
|
||||||
|
|
||||||
|
|
||||||
|
def test_append_sent_uses_special_use_folder():
|
||||||
|
conn = mock.Mock()
|
||||||
|
conn.list.return_value = ("OK", [
|
||||||
|
b'(\\HasNoChildren) "." "INBOX"',
|
||||||
|
b'(\\HasNoChildren \\Sent) "." "Sent"'])
|
||||||
|
msg = EmailMessage()
|
||||||
|
msg["Subject"] = "Subj"
|
||||||
|
msg.set_content("Body")
|
||||||
|
|
||||||
|
mailer.append_sent("u@x", "pw", msg, conn=conn)
|
||||||
|
|
||||||
|
args = conn.append.call_args.args
|
||||||
|
assert args[0] == "Sent"
|
||||||
|
assert "\\Seen" in args[1]
|
||||||
|
assert args[3] == msg.as_bytes()
|
||||||
|
conn.login.assert_not_called()
|
||||||
|
conn.logout.assert_not_called()
|
||||||
|
|||||||
@@ -33,6 +33,68 @@ def ctx_after_match(ratings):
|
|||||||
return ctx
|
return ctx
|
||||||
|
|
||||||
|
|
||||||
|
PROJECT_PAGE = """
|
||||||
|
<html><body>
|
||||||
|
<nav>Login Registrieren</nav>
|
||||||
|
<div class="content">
|
||||||
|
<div class="project-header">
|
||||||
|
<h1>ISTQB Testautomatisierung (m/w/d)</h1>
|
||||||
|
<span>iSAX Consulting GmbH</span>
|
||||||
|
<span>Start 8/2026</span>
|
||||||
|
</div>
|
||||||
|
<div class="project-body">
|
||||||
|
<div class="project-body-badges badges-container">
|
||||||
|
<a href="#">ISTQB</a><a href="#">Testsuite</a><a href="#">IBM Aix</a>
|
||||||
|
</div>
|
||||||
|
<div class="project-body-description">
|
||||||
|
<p>Beschreibung: Muss-Anforderungen: 3 Jahre Testautomatisierung.
|
||||||
|
Erstellung, Beratung und methodische Unterstuetzung der Projektteams
|
||||||
|
bei Testkonzept und Teststrategie. Organisation der Testkoordination
|
||||||
|
fuer einen uebergreifenden Test. Durchfuehrung von manuellen und
|
||||||
|
automatisierten Tests sowie Last- und Performancetests.</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<footer>Impressum Datenschutz</footer>
|
||||||
|
</body></html>
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def test_page_text_keeps_header_and_description_drops_tag_cloud():
|
||||||
|
text = stages.page_text(PROJECT_PAGE)
|
||||||
|
assert "ISTQB Testautomatisierung (m/w/d)" in text
|
||||||
|
assert "Start 8/2026" in text
|
||||||
|
assert "Muss-Anforderungen: 3 Jahre Testautomatisierung" in text
|
||||||
|
assert "IBM Aix" not in text
|
||||||
|
assert "Testsuite" not in text
|
||||||
|
assert "Impressum" not in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_page_text_replaces_quotes_that_break_json_strings():
|
||||||
|
html = ('<div class="content">Zertifizierung „ISTQB Certified Tester" '
|
||||||
|
'sowie "Scrum" und “Kanban”.</div>')
|
||||||
|
text = stages.page_text(html)
|
||||||
|
assert '"' not in text
|
||||||
|
assert "„" not in text and "“" not in text and "”" not in text
|
||||||
|
assert "'ISTQB Certified Tester'" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_page_text_falls_back_to_whole_page():
|
||||||
|
html = "<html><body><p>Nur Text ohne content-Div.</p></body></html>"
|
||||||
|
assert "Nur Text ohne content-Div." in stages.page_text(html)
|
||||||
|
|
||||||
|
|
||||||
|
def test_stage_fetch_uses_project_section():
|
||||||
|
resp = mock.Mock(status_code=200, text=PROJECT_PAGE)
|
||||||
|
resp.raise_for_status = lambda: None
|
||||||
|
ctx = {"canonical": "https://www.freelancermap.de/projekt/x", "title": "t"}
|
||||||
|
with mock.patch("projektmatch.stages.requests.get", return_value=resp):
|
||||||
|
out = stages.stage_fetch(dict(ctx), CFG)
|
||||||
|
assert out["status"] == "ok"
|
||||||
|
assert "IBM Aix" not in out["page_text"]
|
||||||
|
assert "Muss-Anforderungen" in out["page_text"]
|
||||||
|
|
||||||
|
|
||||||
def test_run_stage_catches_and_skips():
|
def test_run_stage_catches_and_skips():
|
||||||
def boom(ctx, cfg):
|
def boom(ctx, cfg):
|
||||||
raise ValueError("kaputt")
|
raise ValueError("kaputt")
|
||||||
|
|||||||
Reference in New Issue
Block a user