What the harness proves — offline, no API key
Guarded topics route to a human, the right one, before the model is ever consulted — all 8 guard-rail tickets.
Answerable questions clear the gate AND retrieve the right document — 26 tickets across shipping, returns, warranty, orders, payments, accounts, collection and product care.
Out-of-scope questions escalate — kayaks, job applications, wholesale, sponsorship, vagueness and keyboard mash: 6 tickets.
Swap in your own tickets as you build your KB; the harness is the quality gate the going-live checklist is built around.
In the download
- Full source — 61 files, 4,407 lines of dependency-light Node (imapflow, nodemailer, pdf-parse — that's the whole list)
- Ingestion for Markdown/HTML/text/PDF with heading-aware chunking (headings become the citations)
- Deterministic retrieval: BM25 + query synonyms + the grounding signal, all offline-testable
- Citation-enforced answering via Anthropic or OpenAI — plus a keyless demo provider for trying the UI
- Embeddable chat widget + token-gated review inbox with approve/dismiss
- Email channel: IMAP worker, shadow mode (draft-only), live mode with escalation notifications
- 40-ticket eval harness + 12-doc sample company (Northbeam Gear Co.)
- 81 unit tests · Docker compose · README + architecture, escalation playbook, going-live checklist
Honestly not included: an LLM subscription/API key, hosting, or Zendesk/Intercom migration tooling. The demo provider answers extractively so you can evaluate everything before adding a key.
The safety posture, in one column
- Hard guard rails: refunds, legal, security, cancellations — always human, not configurable off
- Grounded or silent: poorly-grounded questions escalate with the numbers shown
- Citations validated in code: uncited drafts are rejected and never sent — the draft is attached for the human
- Shadow mode: email drafts sit in the inbox until approved; a one-line rollback from live
- Demo provider fenced twice: config refuses live email with it; the SMTP sender refuses its drafts regardless
The whole pipeline is in src/lib/answer/engine.js — five stages, one screen of code, read it before you buy the claim.