Add user manual and design description for projekt-matching
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
229
docs/projekt-matching-user-manual.md
Normal file
229
docs/projekt-matching-user-manual.md
Normal file
@@ -0,0 +1,229 @@
|
||||
# Projekt-Matching — User Manual
|
||||
|
||||
*The automated pipeline that turns freelancermap project e-mails into evaluated
|
||||
EspoCRM opportunities. Version of 2026-07-09 (all acceptance gates passed).*
|
||||
|
||||
---
|
||||
|
||||
## 1. What the system does
|
||||
|
||||
Every 5 minutes, the system checks the mailbox `chancen@destengs.com` for new
|
||||
e-mails containing freelancermap project descriptions. For every project it
|
||||
finds, it:
|
||||
|
||||
1. resolves the project's canonical URL and checks whether it already exists
|
||||
in the CRM (if yes: nothing happens — no duplicates),
|
||||
2. fetches the full project description from freelancermap,
|
||||
3. lets the **local AI** (vLLM, `AxionML/Qwen3.5-9B-NVFP4` — nothing leaves
|
||||
your machine) extract the requirements and classify them as
|
||||
**Must / Nice / Misc**,
|
||||
4. lets the AI rate every Must/Nice requirement against your CV
|
||||
(✅ covered / ❌ not covered / ❔ partial evidence),
|
||||
5. computes the match values **deterministically in Python** (the AI never
|
||||
invents the numbers): ✅ = 100, ❔ = 25, ❌ = 0, averaged per category,
|
||||
6. decides: **Must-have-Match > 85 %** → the project is *considered*;
|
||||
otherwise it is *rejected*,
|
||||
7. for considered projects: creates the company (Account), contact person
|
||||
(Contact) and the opportunity in EspoCRM — with the same description table
|
||||
the old Claude skill produced — and sends you a notification e-mail,
|
||||
8. records every project evaluation as a trace in Langfuse for your review,
|
||||
9. moves the processed trigger e-mail to the server's **Trash** folder.
|
||||
|
||||
```
|
||||
e-mail arrives every 5 min (systemd timer)
|
||||
│ │
|
||||
▼ ▼
|
||||
INBOX (chancen@) ◀──── poll ──── Langflow "PM Ingest"
|
||||
│ │ per project:
|
||||
│ non-project mail: ▼
|
||||
│ left untouched Langflow "PM Projekt bewerten"
|
||||
│ fetch → AI extract → AI match CV
|
||||
│ → deterministic match calc
|
||||
▼ → gate: Must > 85 %?
|
||||
processed mail │yes │no
|
||||
→ Trash ▼ ▼
|
||||
EspoCRM entry (nothing;
|
||||
+ notification trace only)
|
||||
e-mail
|
||||
│
|
||||
▼
|
||||
Langfuse trace (always)
|
||||
```
|
||||
|
||||
## 2. What you receive
|
||||
|
||||
### Notification (a project matched)
|
||||
|
||||
- **To:** `Thomas.Langer@destengs.com`
|
||||
- **Subject:** `[Projekt-Match] Freelancermap — <Must> % — <Projektname>`
|
||||
- **Body:** both match values and the direct link to the CRM opportunity
|
||||
(`https://crm.creature-go.com/#Opportunity/view/<id>`).
|
||||
|
||||
### Alert (something went wrong)
|
||||
|
||||
- **To:** `Thomas.Langer@destengs.com`
|
||||
- **Subject:** `[Projekt-Match-Fehler] Freelancermap — <n> Projekt(e)`
|
||||
- **Body:** one entry per failed project with title, URL and the error reason.
|
||||
|
||||
**Important:** a failed project is **not retried**. The alert e-mail is your
|
||||
record — if the project matters, evaluate it manually (the URL is in the
|
||||
alert). This is deliberate: it prevents endless retry loops and inbox
|
||||
congestion.
|
||||
|
||||
### What you will NOT see
|
||||
|
||||
- Rejected projects produce no e-mail and no CRM entry. Their full evaluation
|
||||
(table + match values) is in the Langfuse trace.
|
||||
- Non-project e-mails in the inbox are inspected once, invisibly marked with
|
||||
an IMAP keyword (`$ProjektChecked`), and left untouched forever after.
|
||||
- The system never processes its own notification/alert mails (recognized by
|
||||
the subject prefix).
|
||||
|
||||
## 3. The CRM entry
|
||||
|
||||
Considered projects get the full structure the Claude skill used to create:
|
||||
|
||||
| Element | Rule |
|
||||
|---|---|
|
||||
| Opportunity name | project title; duplicate names get ` (2)`, ` (3)` … |
|
||||
| Description | exact skill format: match line + `\| Nr. \| Kat. \| ❔ \| Anforderung \|` table, Must → Nice → Misc |
|
||||
| cProjektlink | canonical project URL (also the duplicate-detection key) |
|
||||
| Team | Projekt → `DesTEngS` · Arbeitnehmer-Angebot → `Arbeitnehmer` · ANÜ → `ANÜ` |
|
||||
| Account | deduplicated by name; agency → `Reseller` + linked via *Über Agentur* (`cAccount1`), direct client → `Customer` + linked via `account` |
|
||||
| Contact | created and linked when the posting names a person and a company |
|
||||
|
||||
Ambiguities the skill used to ask you about are resolved by defaults:
|
||||
unclear offer type → `Projekt`; unclear buyer type → agency.
|
||||
|
||||
## 4. Reviewing quality in Langfuse
|
||||
|
||||
Open **http://127.0.0.1:8091** (login `admin@example.com` /
|
||||
`langfuse-admin-pw` unless you changed it), project **projekt-matching**.
|
||||
|
||||
- Every evaluated project is one trace named **`projekt-match`**, tagged with
|
||||
its outcome (`created` / `rejected` / `failed`). The trace shows the
|
||||
canonical URL, the full requirements table, both match values and — for
|
||||
created projects — the CRM link.
|
||||
- **Your evaluation workflow:** open the annotation queue
|
||||
**`projekt-matching-review`** and score each trace on the two dimensions:
|
||||
- `extraction-correct` — were Must/Nice/Misc correctly derived from the
|
||||
posting? (correct / partially-correct / wrong)
|
||||
- `matching-correct` — are the ✅/❌/❔ ratings against your CV right?
|
||||
- The scores are your quality record. Nothing acts on them automatically —
|
||||
they tell you whether the local 9B model stays good enough over time.
|
||||
|
||||
Langflow's own execution traces (component level, LLM prompts/outputs) land
|
||||
in the same project and are useful for debugging.
|
||||
|
||||
## 5. Everyday operations
|
||||
|
||||
All commands run as user `lwc` on this host.
|
||||
|
||||
### Is it running?
|
||||
|
||||
```bash
|
||||
systemctl --user list-timers | grep projekt-matching # next/last tick
|
||||
curl -s http://127.0.0.1:8090/health # Langflow: {"status":"ok"}
|
||||
journalctl --user -u projekt-matching.service -n 5 # last trigger output
|
||||
```
|
||||
|
||||
To see what recent runs did, use the Langfuse traces (section 4). Do **not**
|
||||
use `podman logs langflow_ctr` — it is flooded by a known harmless logging
|
||||
loop and tells you nothing.
|
||||
|
||||
### Pause / resume the automation
|
||||
|
||||
```bash
|
||||
systemctl --user stop projekt-matching.timer # pause (mail queues up safely in INBOX)
|
||||
systemctl --user start projekt-matching.timer # resume; next tick processes the backlog
|
||||
```
|
||||
|
||||
### Restart Langflow (e.g. after a hang)
|
||||
|
||||
```bash
|
||||
systemctl --user restart container-langflow_ctr.service
|
||||
```
|
||||
|
||||
**Never** use `podman restart langflow_ctr` — with the systemd-managed
|
||||
container this can leave the container removed and the pipeline dead.
|
||||
|
||||
### Update your CV or the Rahmenbedingungen
|
||||
|
||||
Edit the originals in `/home/tlg/mkt/bewerb/vorgaben/`, then:
|
||||
|
||||
```bash
|
||||
/home/lwc/bin/projekt-matching/deploy/deploy_files.sh
|
||||
```
|
||||
|
||||
The script copies the files, redeploys the code, restarts Langflow safely and
|
||||
waits until it is healthy again (≈ 1 minute).
|
||||
|
||||
### Change the notification recipient or the 85 % threshold
|
||||
|
||||
Both live in Langflow **global variables** (UI: http://127.0.0.1:8090 →
|
||||
Settings → Global Variables, or via API):
|
||||
`PM_NOTIFY_TO`, `PM_ALERT_TO`, `PM_THRESHOLD` (also: `PM_VLLM_BASE`,
|
||||
`PM_VLLM_MODEL`, `PM_ESPO_BASE`). Changes take effect on the next run — no
|
||||
restart needed.
|
||||
|
||||
### After a reboot
|
||||
|
||||
Everything is systemd-managed and starts automatically (`pod-langflow_pod`
|
||||
for the services, `projekt-matching.timer` for the polling). Nothing to do.
|
||||
|
||||
## 6. Troubleshooting
|
||||
|
||||
| Symptom | Likely cause / what to do |
|
||||
|---|---|
|
||||
| No mails processed, INBOX grows | `systemctl --user list-timers` — timer active? Langflow healthy (`curl …/health`)? Restart per section 5. Mails are never lost: the next successful tick processes the backlog. |
|
||||
| Alert mail: `Abruf fehlgeschlagen` / `Seitentext zu kurz` | freelancermap page unreachable or behind a login wall. Evaluate that project manually via the URL in the alert. |
|
||||
| Alert mail: `vLLM HTTP …` / `LLM lieferte kein JSON` / `Read timed out` | the local AI was down or overloaded. Occasional single failures are absorbed by built-in retries; if alerts pile up, check the vLLM service on port 8081. |
|
||||
| Alert mail: `Team … nicht gefunden` | the CRM user `cowork-api` lost read access to Teams, or a team was renamed. Fix in EspoCRM admin. |
|
||||
| A project you expected is missing | check Langfuse: if its trace says `rejected`, the Must-match was ≤ 85 % — the table shows why. If there is no trace at all, check the alert mails. |
|
||||
| Duplicate opportunity suspected | duplicates are keyed on the canonical URL (`cProjektlink`). The same project posted by two different agencies has two different URLs and is intentionally kept as two opportunities. |
|
||||
| Everything looks dead after an experiment | `systemctl --user start container-langflow_ctr.service`, then check health. |
|
||||
|
||||
## 7. Test utilities (use with care)
|
||||
|
||||
In `/home/lwc/bin/projekt-matching/tests/e2e/` (run with the project venv,
|
||||
secrets sourced: `set -a && source deploy/secrets.local.env && set +a`):
|
||||
|
||||
- `check_state.py [token]` — shows CRM hits for a URL token plus the last
|
||||
INBOX/Trash subjects. Safe, read-only.
|
||||
- `run_flow2.py <canonical-url> <title>` — evaluates one project directly
|
||||
(bypasses e-mail). Writes to the CRM if it matches!
|
||||
- `send_test_mail.py "<subject>" [fallback-url]` — re-sends a stored example
|
||||
mail to yourself to trigger the full pipeline.
|
||||
- `cleanup_crm.py [tokens…]` — **deletes** opportunities whose cProjektlink
|
||||
matches the tokens. Default tokens are the two historical test projects.
|
||||
Never run it with a token that matches real opportunities.
|
||||
- `flag_inbox.py` — marks every current INBOX mail as "already checked"
|
||||
(used once at go-live; rarely needed again).
|
||||
|
||||
## 8. Where things live
|
||||
|
||||
| What | Where |
|
||||
|---|---|
|
||||
| Code + tests | `/home/lwc/bin/projekt-matching/` (git) |
|
||||
| Flows (source of truth) | built by `deploy/build_flows.py` into Langflow ("PM Ingest", "PM Projekt bewerten") |
|
||||
| Deployed runtime copy | `~/.local/share/langflow_pod/langflow-data/` (`/app/langflow` in-container) |
|
||||
| Secrets | `~/.config/projekt-matching/` (0600) + Langflow global variables (encrypted); EspoCRM key sourced from `/home/tlg/mkt/bewerb/.secrets/espocrm-api.md` |
|
||||
| systemd units | `~/.config/systemd/user/projekt-matching.{service,timer}` |
|
||||
| Services | Langflow http://127.0.0.1:8090 · Langfuse http://127.0.0.1:8091 · vLLM http://127.0.0.1:8081 · pod script `/home/lwc/bin/create_pod_langflow.sh` |
|
||||
| Design details | `docs/projekt-matching-design.md`, spec/plan under `docs/superpowers/` |
|
||||
|
||||
## 9. Known limitations
|
||||
|
||||
- **freelancermap only.** Mails from other portals are ignored (left in the
|
||||
inbox untouched). Extending detection is a small code change in
|
||||
`projektmatch/mailparse.py`.
|
||||
- **Login-protected postings fail** with an alert (no page text, no snippet
|
||||
fallback — a snippet cannot support honest matching).
|
||||
- **Nice-to-have-Match varies a few points** between runs of the same project
|
||||
(single ❔/✅ flips by the 9B model). The Must-match — the gate — has been
|
||||
stable in all verification runs. Watch this in the annotation queue.
|
||||
- **Rare edge case:** if the CRM write succeeds but the notification mail
|
||||
fails to send, the opportunity exists but no mail arrives and none is
|
||||
re-sent. The Langfuse trace records it (`error: notify: …`).
|
||||
- Misc rows (start date, workload, …) are occasionally missing from the
|
||||
extraction; they never influence the match values.
|
||||
Reference in New Issue
Block a user