Logbuch-Pipeline / Werkbank

Input is the Phase-1 scraper shape: thread and post IDs, source URL, system category, raw German bodies including platform clutter. Run the bundled sample or paste your own export.

{
  "source": "phpbb-thread-export",
  "thread_id": 4821,
  "url": "https://forum.hebetechnik-beispiel.de/viewtopic.php?t=4821",
  "title_de": "Verschleiss Trapezgewindespindel SHG-050",
  "system_category": "spindelhubgetriebe/hubtische",
  "macro_domain": "antriebstechnik",
  "posts": [
    { "post_id": 118203, "author": "Markus_HTM", "body_de": "Hallo zusammen, ... Axialspiel auf 0,8 mm erhoeht ... Anzugsmoment ... (M12, DIN 931) ... 25 kN ... 1500 U/min ... Gesendet von meinem SM-G991B mit Tapatalk / Nach oben / Zitieren" },
    { "post_id": 118206, "author": "Dreher_Kalle", "body_de": "... Vogel & Brandt Antriebstechnik GmbH ... SHG-050-TR40x7-E2 (Werkstoff C45, Mutter aus RG7) ... 85 Nm ..." },
    { "post_id": 118211, "author": "Markus_HTM", "body_de": "Vielen Dank, hat funktioniert!" },
    { "post_id": 118240, "author": "ElektroFuzzi", "body_de": "... Schaltplan der Hubtisch-Steuerung als Anhang ...", "attachments": ["schaltplan_hubtisch.png", "massblatt_shg050.png"] }
  ]
}

Rule-based line classifier. Red lines are stripped (signatures, navigation, mobile footers, thanks-only chat); green lines are kept with a stated reason. Supplier and fabricator contact details are explicitly preserved, never treated as noise.

Part numbers, torque specs and measurements are extracted BEFORE translation and replaced with placeholders, so the translator never touches them. After restoration, the gate re-extracts every token from the source and verifies it survived, one row per token. Flip the switch to see what an unshielded model does to the same text, and what the gate catches.

The logbook document. YAML front-matter carries URL, thread ID, category, translation path, gate verdict and the part-number index. Under every heading sits a one-line metadata echo (the green ctx lines): that redundancy is deliberate, and station 05 shows why.

The same recursive splitter (headings first, then paragraphs, packed to the token budget) runs against two versions of the document. With metadata echoes, every chunk still knows its source URL, thread and category. The naive version (front-matter only at the top) orphans everything after the first cut.

Dual-track ingestion. Track A renames and catalogs binaries against thread and post IDs. Track B first triages images (entropy and sharpness computed live with sharp on this request), then routes only high-value schematics and drawings to the VLM with a structured extraction prompt.

SchaltplanMassblatt

Run VLM

Structured promptGeneric prompt (counterexample)

SYSTEM
You are an engineering documentation extractor for a technical
knowledge base about lifting systems (screw jacks, lifting
platforms, drives).

You receive one image from a German engineering forum thread.
Extract SPECIFIC, verifiable engineering metadata. Never write
generic visual descriptions.

Rules:
1. Transcribe every readable designator, part number, dimension
   and rating EXACTLY as printed (OCR fidelity over fluency).
   If unreadable, write "unreadable", never guess.
2. Classify doc_type from a fixed vocabulary (electrical
   schematic, dimension drawing, photo of damage, ...).
3. Output ONLY valid JSON matching the schema.
4. Set confidence per field: "read" or "inferred".

USER
Extract to this JSON schema:
{
  "doc_type": "...",
  "title_block": { "present": true, "text": "..." },
  "components": [ { "ref": "Q1", "name": "...", "rating": "...",
                    "confidence": "read|inferred" } ],
  "part_numbers": [ "..." ],
  "dimensions": [ { "name": "...", "value": "...",
                    "confidence": "read|inferred" } ],
  "connections_summary": "one sentence, only if traceable",
  "function_summary": "one sentence, engineering register",
  "relevance_for_kb": "why this image matters"
}
+ [image]

Extracted metadata (schaltplan_hubtisch.png)

loading...

The extracted JSON is rendered to a text block and appended inline beneath the matching post in the station 04 logbook (see "Attachment enrichment" sections). PDFs follow the same track with layout-preserving extraction (Docling / Unstructured) before any VLM call.