17 September 2026/10 min read/Consulting workflows
Email and ticket triage on-prem: running a local LLM over the inbox without sending customer data out
TL;DR
Email and ticket triage — classify, extract intent and entities, route, suggest a draft — scores about 9 of 12 on the local-vs-cloud axes: high volume, a fixed taxonomy, and personal data in every message.
A 7–12B open-weight instruct model on one 24 GB GPU is enough; multilingual capability is a hard requirement in DACH and the Nordics, and you test it on your own samples, not the model card.
At 1,500 input and 300 output tokens per email, 50,000 emails a month cost about €50 on GPT-4.1 Mini, €248 on GPT-4.1 and €414 on Claude Sonnet 5 at list prices — under a €600 reserved GPU line, so triage alone rarely crosses on price; sensitivity and a shared GPU make the case.
Build the labeled set first: about 500 historical emails, two labelers, per-class precision and recall, a confidence threshold per class, and a human triage view below it.
Three guardrails are non-negotiable: the system never auto-sends, PII is pseudonymized before any exception reaches a frontier API, and every message gets an audit log entry with model version, confidence and overrides.
Questions people ask
Can a local LLM handle email triage?
Yes. Classifying a message into 8–15 classes, extracting identifiers such as order or invoice numbers, choosing a queue and drafting a short reply is a narrow, repetitive task. Open-weight instruct models in the 7–12B range, quantized to fit a 24 GB GPU, handle it with schema-enforced JSON output and a prompt that defines each class with a few examples. Fine-tuning is rarely needed at the start.
How much does it cost to triage 50,000 emails a month with an API versus a local model?
Assuming 1,500 input and 300 output tokens per email and list prices at the time of writing, 50,000 emails cost about €50 on GPT-4.1 Mini, €248 on GPT-4.1 and €414 on Claude Sonnet 5. A reserved EU GPU is about €600 a month, so triage alone does not cross on price at that volume; it does when the GPU is shared with document extraction or when the data cannot leave the network.
Why run ticket classification on-premise instead of on a cloud API?
Because the inbox is personal data: customer contact details, supplier bank details, contract terms and HR correspondence that can include special-category data under GDPR Art. 9. Sending every message to a US-headquartered API is a transfer under Art. 44–49 that needs a mechanism and a processor agreement. A local model reads the message inside your network, keeps working when an API is down, and leaves no third-party retention to audit.
How do you evaluate an email triage model?
Label about 500 historical emails with two independent labelers, stratified so every class has at least 25 examples, and hold out 100. Report precision and recall per class: precision where a wrong route is costly, recall where a missed class is costly, such as complaints and cancellations. Confusion between adjacent classes usually means the taxonomy needs fixing, not the model. Set a confidence threshold per class.
Should AI triage send replies automatically?
No. The system should route, tag, prioritize and draft; a person sends. An auto-sent wrong reply to a complaint costs more than the automation saves, and a routing suggestion is a different thing from an automated decision with legal effect under GDPR Art. 22. Keep a human in the decision for refunds, cancellations and terminations regardless of the model’s confidence.
How do you integrate a local triage model with a helpdesk or mailbox?
Through the mailbox API (Microsoft Graph, the Gmail API or IMAP) or the helpdesk’s ticket-created webhook. A small service reads the message, calls the local model behind an OpenAI-compatible endpoint, and writes the class, entities, queue and draft back as fields and an internal note. Run it in shadow mode first, with suggestions in hidden fields, and compare against human routing before enabling auto-routing.
Want this worked out on your documents?
We will price a real sample against OpenAI or Anthropic and tell you whether Local AI consulting or a local model on your hardware is the cheaper first step.
Email and ticket triage — classify the message, pull out the intent and the entities, route it to the right queue, and suggest a reply — is one of the two or three best workflows to move onto a local model. It is high-volume, narrow, and full of personal data. A 7–12B open-weight instruct model on a single GPU handles it. At 50,000 emails a month the price case alone is not decisive: GPT-4.1 costs about €248 a month at list price against a €600 reserved GPU line. The case is made by sensitivity — customer PII, contracts, HR mail — and by sharing the card with the document workflows that already justify it.
This is the triage chapter of a local AI consulting engagement, written down.
What triage does: four steps and one JSON object
Triage is not a chatbot. Nobody talks to it. Each inbound message goes through four steps, and the output is one structured record attached to the ticket:
Classify. Assign one of 8–15 classes from a fixed taxonomy: password reset, invoice query, delivery status, complaint, cancellation, contract question, HR request, spam. Fewer than eight and the classes are too broad to route on; more than fifteen and the labelers stop agreeing.
Extract intent and entities. The intent within the class (“wants a refund” vs “wants a replacement”) and the identifiers a system can act on: order number, invoice number, customer ID, contract reference, deadline, language, urgency.
Route. Map class and entities to a queue, a priority, and a set of tags. This step saves the most time: a message that lands in the right queue with the order number already parsed skips the first human touch.
Suggest a draft. A short reply the agent can edit, grounded in the class and the extracted facts. It is a suggestion in the compose window. It is never sent by the system.
The model returns all four as one JSON object with a confidence per field. Everything downstream — the helpdesk rule, the audit log, the threshold — reads that object, not free text.
Why triage is a strong local candidate
On the four-axis method in which workflows should run on local models, support and AP triage scores about 9 out of 12: Volume 3, Sensitivity 2, Stability 2, Availability 2. Each axis has a plain reason.
Volume. The inbox never stops. A mid-sized European support or accounts-payable mailbox sees thousands of messages a week, every week, with a predictable floor. That is the shape a fixed-cost GPU wants.
Sensitivity. An inbox is a database of personal data that nobody designed. Customer names and addresses in support mail; supplier bank details and invoice attachments in AP mail; contract terms in the legal alias; sick notes, grievances, and union correspondence in the HR alias. Sending each message to a US-headquartered API is a transfer under GDPR Art. 44–49 that needs a mechanism and a processor agreement (Art. 28), and HR mail routinely contains special-category data under Art. 9. A local model reads the message inside your network, and the GDPR-compliant document AI checklist applies to triage unchanged.
Stability. The taxonomy is fixed. The emails are not — every customer writes differently — but the task is the same twelve labels every day, and a small model that has seen 500 examples of “delivery status” handles the 501st.
Availability. Triage sits in front of a queue with response-time targets. When an external API is rate-limited, down, or has retired your model version, triage stops and the queue fills. A local endpoint has predictable latency and no third-party roadmap.
The API stays right for the long reply to an unusual complaint or a language you have not sampled — the exceptions the hybrid AI stack routes on.
Which models are enough
Classification, entity extraction, and a three-sentence draft do not need a frontier model. Open-weight instruct models in the 7–12B parameter range — the Gemma 4, Qwen3, Mistral, and Llama families all publish weights in that band — handle the task with a well-written prompt and a JSON schema enforced by the serving stack. Quantized to 4 or 8 bits, a model of that size fits in a 24 GB card with room for batching. Fine-tuning is usually unnecessary at the start: one-line class definitions and two or three examples per class in the prompt get you to the evaluation.
Multilingual is not optional in DACH and the Nordics. A German support inbox mixes German and English; a Norwegian one mixes Norwegian, Swedish, Danish, and English. Check that the model card lists your languages, then ignore the card and test on your own samples — a model that is fluent in German prose can still mis-parse a Swedish order confirmation. Detect the language as one of the extracted fields; it decides which agent group the message routes to and which language the draft is written in.
Step up to a 30B-class local model when drafts must be polished or the class depends on reasoning across a long thread. The local document models page covers the serving options; the box that runs invoice extraction runs triage in the gaps between batches.
Build the labeled set before the model
The evaluation set is the project. Pull about 500 historical emails, stratified so that every class has at least 25 examples and the rare-but-costly classes (complaints, cancellations, legal) are over-represented. Two people label independently; where they disagree, a third decides and the taxonomy gets a note. If two humans agree less than nine times in ten on a class, the class is badly defined and no model will do better. Hold out 100 of the 500 for the final number, and never tune on those.
Precision and recall, per class
Report both, per class, and read them against the cost of each error. Precision matters where a wrong route is expensive: a complaint filed under “spam.” Recall matters where a missed class is expensive: a cancellation that does not get flagged is a churned customer and, in some sectors, a regulatory clock that has started. Confusion between two adjacent classes — “invoice query” and “payment reminder,” say — is almost always a taxonomy problem: merge them, or write down the rule that separates them and add examples.
Human in the loop, by design
Set the confidence threshold per class, not globally. Above it the message auto-routes; below it, it lands in a human triage view with the model’s suggestion visible. Every agent correction is logged against the message and becomes next month’s evaluation data. Review the below-threshold pile weekly; if a class sits there too often, either the threshold is wrong or the class is.
Vendor-neutral integration patterns
Two entry points cover almost every organization, and neither ties you to a helpdesk vendor.
Mailbox APIs. Microsoft Graph for Exchange Online and Microsoft 365 mailboxes, the Gmail API for Google Workspace, and IMAP for everything else. A small service subscribes to new-message notifications (or polls a folder), reads the message, calls the local model, and writes back a category, labels, a folder move, and a draft. Keep the model behind an OpenAI-compatible endpoint so the weights can change without the integration changing.
Helpdesk webhooks. Every mainstream helpdesk exposes a ticket-created event and an API to set fields, tags, and assignee. The same service receives the event, calls the model, and writes the JSON back into custom fields, with the draft as an internal note. Run it in shadow mode first: the suggestion goes into a hidden field, agents work exactly as before, and after two weeks you compare the model’s route with the human’s.
Three details decide whether it works in production. Strip quoted history and signatures before the model sees the thread, and cap input at a token budget. Make the service idempotent so a retried webhook does not create two classifications. And send attachments to the document pipeline, not the triage model: an invoice on an AP email is a bulk document processing job, and the extracted invoice number comes back as an entity on the ticket.
Token math for 50,000 emails a month
Assume 1,500 input tokens per email — a trimmed body plus the taxonomy, class definitions, and output schema in the prompt — and 300 output tokens for the JSON record and a short draft. List prices at the time of writing, converted at $1 = €0.92: GPT-4.1 at $2 / $8 per million tokens, GPT-4.1 Mini at $0.40 / $1.60, Claude Sonnet 5 at $3 / $15. For the local column, a 7–12B model served with batching on an EU on-demand A100-class card at €1.80 an hour, at a deliberately conservative 600 emails an hour.
Illustrative scenario, assumptions stated here: 1,500 input + 300 output tokens per email; list prices at the time of writing at $1 = €0.92; no batch discount or caching; local on-demand at €1.80/hour and 600 emails/hour; reserved GPU line €600/month. Crossover is the reserved line divided by the per-email API rate.
Option
Per email
50,000 emails / month
Crossover vs €600 GPU
GPT-4.1 Mini
€0.0010
€50
≈ 600,000 emails
GPT-4.1
€0.0050
€248
≈ 120,000 emails
Claude Sonnet 5
€0.0083
€414
≈ 72,000 emails
Local, on-demand rental (compute only)
≈ €0.0030
≈ €150
—
Local, reserved GPU
fixed
€600, shared with other workflows
—
Illustrative scenario, assumptions stated here: 1,500 input + 300 output tokens per email; list prices at the time of writing at $1 = €0.92; no batch discount or caching; local on-demand at €1.80/hour and 600 emails/hour; reserved GPU line €600/month. Crossover is the reserved line divided by the per-email API rate.
Read it honestly. At 50,000 emails a month, GPT-4.1 is under the €600 line and Mini is far under it. Apply the 1.3–2.0× multiplier that long threads, attachments, and retries add to every rate card and Sonnet is at or over the line (€540–€830) while GPT-4.1 still is not. Triage on its own rarely crosses on price at this volume. It crosses in three other ways: the marginal cost of triage on a GPU that invoice extraction already pays for is close to zero; the sensitivity axis removes the API from the options for HR, legal, and any special-category class; and a mailbox past about 120,000 messages a month is past the GPT-4.1 line on its own. The per-page version of this arithmetic is in the local LLM vs OpenAI cost crossover; the method for pulling your real token counts is in the workflow-first way to stop paying per invoice.
Guardrails: never auto-send, handle PII, log everything
Never auto-send. The system routes, tags, prioritizes, and drafts. A person sends. An auto-sent wrong reply to a complaint costs more than a month of triage saves, and a routing suggestion is a very different thing from an automated decision with legal effect under GDPR Art. 22. Keep the human in the decision for anything with a consequence — refunds, cancellations, terminations — even when the model is confident.
PII stays where it is. The model runs inside the network, so the message does not leave. If an exception is routed to a frontier API, pseudonymize first: names, email addresses, phone numbers, IBANs, and customer IDs replaced with placeholders, and a record of what left and why. Output stored on the ticket inherits the ticket’s retention; do not build a second copy of the inbox in a log store. HR and legal classes go to queues with restricted visibility, and the model’s output on those messages is not readable by general agents.
Audit log per message. Model version, prompt version, a hash of the input, the output JSON, the per-field confidence, the action taken, and any human override with who and when. That log is what a data protection impact assessment (Art. 35) and the Art. 30 record of processing point to, and it is the practical form of the transparency the EU AI Act expects even from a system that is not high-risk.
The rules above land in one table per inbox: ticket type, action, and the confidence rule that decides whether a human sees it first.
Illustrative routing table for a combined support and AP inbox. Thresholds are starting points to be set per class from your own labeled set; “auto-route” means the queue and tags are set without a human, never that a reply is sent.
Ticket type
Action
Confidence rule
Password reset / account access
Route to IT helpdesk; attach the standard-steps draft
Auto-route at 0.90+; below, general triage with the suggestion shown
Invoice or payment query
Route to finance; extract invoice number, amount, supplier; attachment to document extraction
Auto-route at 0.85+ and an invoice number found; otherwise finance triage view
Order or delivery status
Route to logistics; extract order number; draft with a lookup placeholder
Auto-route at 0.90+ and the order number matches a record
Complaint or escalation
Route to a senior agent; priority high; draft as suggestion only
Route at any confidence; never auto-close; recall over precision
Cancellation request
Route to retention queue; extract contract or subscription ID; flag the notice deadline
Route at 0.70+; below, human triage — a missed cancellation is the costly error
Contract or legal question
Route to legal queue with restricted visibility; no draft to the customer
Route at 0.80+; below, human review; a draft leaves only with legal sign-off
HR request (leave, payroll, grievance)
Route to HR queue with restricted visibility; output hidden from general agents
Route at any confidence to the restricted queue; treat as Art. 9 by default
Spam, newsletter, auto-reply
Archive
Auto-archive at 0.95+; sample 2% daily for a human check
Unclear or multiple intents
Human triage queue with the top two classes shown
Anything below the class threshold, or two classes within 0.15 of each other
Illustrative routing table for a combined support and AP inbox. Thresholds are starting points to be set per class from your own labeled set; “auto-route” means the queue and tags are set without a human, never that a reply is sent.
Shipping it in four weeks
Week 1 — Inventory and taxonomy. Export 90 days of mailbox or helpdesk history. Define 8–15 classes with the people who work the queue. Label 500 messages with two labelers, measure agreement, fix the classes where they disagree. Hold out 100.
Week 2 — Model and prompt. Deploy a 7–12B instruct model behind an OpenAI-compatible endpoint with JSON schema output. Write the prompt from the class definitions. Run the 400, read the per-class precision and recall, and fix the taxonomy before touching the model. Set a threshold per class.
Week 3 — Shadow mode. Connect the mailbox API or the helpdesk webhook. Write the model’s route and draft to hidden fields on every live ticket. Agents work as before. Compare the model’s route with the human’s at the end of the week and adjust thresholds.
Week 4 — Go live with guardrails. Turn on auto-routing for the classes that cleared their threshold in shadow mode. Show drafts as suggestions. Switch on the audit log and the restricted queues. Schedule the weekly review of the below-threshold pile, and re-run the held-out 100 monthly to catch drift.
Five numbers tell you whether it worked: routing accuracy on the held-out 100, the share of messages that auto-routed, the human-touch rate before and after, time to first response, and overrides per class per week. If the first two are high and the last is low, the model is doing the work an on-premise document AI stack was bought for. If you are not sure where your inbox sits, the labeled 500 is the first thing we build together.