Which workflows should run on local models? A four-axis scoring method for on-prem vs cloud AI
TL;DR
Score each recurring workflow 0–3 on four axes — volume and regularity, sensitivity and residency, stability, and latency and availability — for a total out of 12.
Eight or more goes local by default, four or less stays on a cloud API, and five to seven is the hybrid zone where a local model takes the routine share and an API or a person takes the exceptions.
Two overrides beat the total: special-category data under GDPR Art. 9 goes local regardless, and a rare, open-ended task stays on an API regardless.
On typical scores, invoice extraction lands at 10 of 12, email triage and internal search at 9, contract review at 6 with the sensitivity override, and sales drafting or ad-hoc research at 0–2.
The mapping produces three artifacts: a routing map, a hardware plan (one A100-class GPU at about 1,200 pages an hour covers most departments), and a cost line against your API export.
Questions people ask
Which workflows should run on a local LLM?
Workflows that are high-volume, repetitive, sensitive, and need to keep running without an external dependency: invoice and receipt extraction, claims and forms, support and AP inbox triage, internal knowledge search, and routine translation. Scored 0–3 on volume, sensitivity, stability and availability, these typically total 8 or more out of 12, which is the threshold for local by default.
When should I use a cloud API instead of on-prem AI?
When the task is rare, open-ended, low-sensitivity, or so small that a GPU would idle: proposal drafting, one-off research and analysis, and unusual customer correspondence. These score 4 or less on the four axes. A cloud API is also the right home for seasonal spikes beyond a local GPU’s floor, provided personal data is stripped and a processing agreement is in place.
Should contract review run on a local model or on OpenAI?
On the total score, contract clause review usually lands in the hybrid zone at about 6 of 12: low volume and high variability pull it toward an API, while sensitivity pulls it local. The sensitivity override decides: contracts run on a local long-context model, and a frontier API sees only a redacted excerpt when the local answer is uncertain — never the data room.
How much hardware does a local-model workflow need?
Less than most teams expect. A compact document model on an A100-class GPU processes about 1,200 pages an hour, so 60,000 pages a month is roughly 50 GPU-hours — a couple of hours a day on one card, with capacity left for triage and search. Most mid-sized organizations need one GPU, rented in the EU at about €600 a month or owned as an RTX 4090-class workstation.
What does a workflow routing map contain?
One row per recurring workflow with its execution tier (local model, mini API, frontier API or human), the model class, the confidence threshold, the fallback below that threshold, and the data-flow rule stating whether any data may leave the network and in what form. It is the document a DPIA under GDPR Art. 35 and a CFO’s cost line can both reference.
Can a local model handle a workflow that is difficult?
Yes, if the difficulty repeats. The scoring axis is stability, not difficulty: a dense multi-column customs form is hard, but it is hard the same way every day, and a local specialist model that has seen 500 of them handles the 501st. What local models handle poorly is novelty — a different task every time, or one that needs broad world knowledge and long reasoning.
Want this worked out on your documents?
We will price a real sample against OpenAI or Anthropic and tell you whether Local AI consulting or a local model on your hardware is the cheaper first step.
Run a workflow on a local model when it is high-volume, repetitive, sensitive, and has to keep working when an outside API does not. Keep it on a cloud API when it is rare, open-ended, low-sensitivity, or so small that a GPU would sit idle. We score every recurring workflow from 0 to 3 on four axes — volume and regularity, sensitivity and residency, stability, and latency and availability — for a total out of 12. Eight or more goes local by default. Four or less stays on an API. Five to seven is the hybrid zone, where a local model takes the routine share and a frontier API or a person takes the exceptions. Two overrides sit above the total: special-category data goes local regardless, and a rare, open-ended task stays on an API regardless.
Definition · Workflow routing map
A workflow routing map is a table that assigns each recurring document or text task in an organization to one execution tier — local model, mini API, frontier API, or human — based on its volume, sensitivity, stability and availability needs, together with the confidence threshold and fallback for each. It is the first deliverable of a local-versus-cloud AI assessment.
The question is per workflow, not per company
“Should we go on-prem or stay on OpenAI?” has no useful answer at company level. A finance department that extracts 40,000 invoice pages a month and a sales team that drafts 30 proposals are both “using AI,” and they belong on opposite tiers. The unit of decision is the workflow: a task that repeats, has an owner, has a measurable volume, and produces a result someone acts on. Most organizations have eight to fifteen of them. Scoring each one takes about ten minutes once the inventory exists, and the sum of those scores is the on-prem-versus-cloud decision.
This is the scoring step of our local AI consulting engagement. The method is public because the value is in running it on your numbers, not in knowing it exists. If you have not yet pulled a usage export or measured tokens per unit, start with the workflow-first method for reducing API costs; the scoring below assumes that inventory is on the table.
The four axes
Volume and regularity
Local models are a fixed cost. A reserved EU A100-class GPU runs about €600 a month whether it processes 500 pages or 100,000, and a compact document model handles roughly 1,200 pages an hour, so a single card absorbs most departmental volumes in a few hours a day. The axis rewards steady flow: a workflow with 30,000 units every month scores higher than one with 90,000 in December and none in July. Spikes are what an API is for.
Sensitivity and residency
Every unit sent to a US-headquartered API is a transfer of data, and GDPR wants a lawful basis for the processing (Art. 6), extra conditions for special-category data such as health information (Art. 9), a processor agreement (Art. 28), and a transfer mechanism if the data leaves the EEA (Art. 44–49). Standard contractual clauses and the EU–US Data Privacy Framework make transfers possible; they do not make them free of review, and Schrems II showed that a mechanism can fall. A workflow scores 3 here when the data is special-category, covered by a contractual bar on third-party processing, or when your DPO or works council has said it stays inside. It scores 0 when the input is public or synthetic.
Stability: variability and difficulty
Local specialist models — document vision models, small text classifiers, embedding models — are strongest when the task is narrow and the layouts repeat. Frontier APIs are strongest when every unit is different and the task needs broad world knowledge or long reasoning. So this axis scores stability, not difficulty: a fixed schema on a stable family of documents scores 3; open-ended analysis of something new each time scores 0. Difficulty alone does not push a task to the cloud. A dense multi-column customs form is hard, but it is hard in the same way every day, and a local model that has seen 500 of them handles the 501st.
Latency, availability and control
Some workflows tolerate an overnight batch. Others sit inside a process with a person waiting, or must keep running when an external API is rate-limited, down, or has retired the model version you tested against. Local inference has predictable latency and no third-party dependency; an API has elastic capacity and someone else’s roadmap. Score 3 when the workflow must run inside your network or without an external dependency, 0 when it is occasional and nobody is waiting.
The scoring table and the threshold rule
Score each recurring workflow 0–3 on each axis and sum to a total out of 12. Volume bands are typical for document and text workflows in a mid-sized European organization; adjust the bands to your own scale, but keep the ordering.
Axis
0
1
2
3
Volume and regularity
A few units a month, irregular
Hundreds a month
Thousands a month, steady
Tens of thousands a month, or continuous daily flow
Sensitivity and residency
Public or synthetic data
Internal business data, no personal data
Personal data (Art. 6), supplier bank details, customer records
Special-category data (Art. 9), trade secrets, contractual bar on transfer
Stability
Every unit different; open-ended reasoning
Several families, frequent novelty
A few stable families, occasional new layout
One narrow, repeatable task with a fixed schema
Latency, availability, control
Occasional; nobody waiting
Batch is fine; spiky load
Inside a daily process with a response-time expectation
Must run in-network or without any external dependency
Score each recurring workflow 0–3 on each axis and sum to a total out of 12. Volume bands are typical for document and text workflows in a mid-sized European organization; adjust the bands to your own scale, but keep the ordering.
Six workflows, scored
Illustrative scores for six common workflows, using the bands above for a mid-sized European organization. Your scores will differ — a contract desk with 5,000 documents a month scores Volume 3, not 1. Totals are out of 12.
Workflow
Volume
Sensitivity
Stability
Availability
Total
Route
Supplier invoice extraction
3
2
3
2
10
Local specialist; API or human on exceptions
Support and AP email triage
3
2
2
2
9
Local small LLM classifier; human below threshold
Internal knowledge search and Q&A
2
3
2
2
9
Local embeddings and local LLM
Contract clause review
1
3
1
1
6
Local long-context model (override); frontier API only on redacted excerpts
Sales and proposal drafting
1
1
0
0
2
Frontier API
Ad-hoc research and analysis
0
0
0
0
0
Frontier API
Illustrative scores for six common workflows, using the bands above for a mid-sized European organization. Your scores will differ — a contract desk with 5,000 documents a month scores Volume 3, not 1. Totals are out of 12.
Invoice extraction is the textbook local workflow: high volume, stable layouts, bank details and supplier contacts on every page, and an AP team that wants the queue cleared by 9:00. Email triage scores nearly as high because classification is a narrow task even when the emails are not. Internal search scores on sensitivity and availability more than volume: the corpus is the company’s own contracts, HR files and engineering notes, and nobody wants that indexed by a third party. Contract review lands in the hybrid zone on the total, but the sensitivity override pulls it local; the frontier API gets a redacted clause when the local model’s answer is uncertain, not the data room. Drafting and research score near zero on every axis and belong on an API. Buying hardware for them is the anti-pattern in the next section.
Sensitivity often decides earlier than the arithmetic. For invoices and claims in DACH, the Nordics and Benelux, the question a compliance lead asks first is where the page goes, not what it costs; GDPR-compliant document AI and the EU AI Act obligations for document processing cover what that review looks like.
Four anti-patterns
Local for rare, exotic tasks. A GPU reserved for 40 open-ended analyses a month costs €15 per task and produces worse answers than a frontier model. Volume 0, Stability 0: leave it on the API.
A US API for high-volume regulated documents. Tens of thousands of prescriptions, claims or payroll forms a month to a US endpoint with no transfer basis is not a cost problem; it is an Art. 9 and Art. 44 problem, and the volume makes it visible. Sensitivity 3 is an override for a reason.
One frontier model for everything. A single API key behind invoices, triage, search and drafts is the most common budget failure. At list prices at the time of writing, a routine page is €0.0116 on Claude Sonnet 5 and €0.0014 on GPT-4.1 Mini; the difference is paid on every unit of the highest-volume workflow.
Hardware for a seasonal spike. A card sized for the December backlog idles for eleven months. Size local for the steady floor; send the spike to an API with personal data stripped.
What the mapping produces
The scoring exercise ends with three artifacts, and each one is something a CFO or a DPO can sign.
A routing map
One row per workflow: the tier, the model class (document vision model, small text LLM, embeddings, frontier API), the confidence threshold, the fallback below it, and whether any data may leave the network and in what form. This is the document that turns “we use AI” into something a data protection impact assessment (GDPR Art. 35) can reference.
A hardware plan
Sum the monthly units of every workflow routed local, convert to pages or equivalent, and divide by throughput. At about 1,200 pages an hour for a compact document model on an A100-class card, 60,000 pages a month is roughly 50 GPU-hours — a couple of hours a day on one card, with room for triage and search on the same box. Most mid-sized organizations need one GPU, not a cluster, and an owned RTX 4090-class workstation amortized over three years is cheaper than rental at high utilization. Deployment options are on the local document models page.
A cost line
The reserved GPU line — about €600 a month rented, or the amortized owned box — set against the current API export for the workflows moving local, plus the API cost of the exceptions that still leave. The fixed-cost versus per-token comparison at 100,000 pages shows what that line looks like for a single large document workflow; the mapping does the same sum across the whole week.
Running the workshop in one day
Before the day. Export 30 days of OpenAI and Anthropic usage by key or project. Ask each department head for a list of tasks that repeat weekly, with a rough monthly count. Pull 50 sample units for each of the top five candidates.
09:00 — Inventory. Merge the lists into one table of workflows with owner, monthly units, current tool and the accuracy bar the business needs. Stop at fifteen; the long tail can wait.
10:00 — Score. Owners score their own workflows on the four axes with the bands in the table. Argue about 2 versus 3; do not argue about 0 versus 3.
11:30 — Apply the overrides. Flag every Sensitivity 3 and every Volume 0 + Stability 0. Those routes are fixed before any cost math.
13:00 — Rank. Multiply the total score by monthly units. The top two or three are the local candidates; everything under the API threshold is left where it is.
14:00 — Size and price. Sum the local candidates’ volumes, divide by 1,200 pages an hour, and pick the GPU. Set the €600 line against the export for those workflows.
15:30 — Write the routing map. Tier, model class, threshold, fallback and data-flow rule for each workflow. One page.
16:30 — Choose the pilot. Highest score × volume with a labeled sample available — usually invoice extraction or inbox triage. Define the success metrics: field accuracy on 200 labeled units, human-touch rate, cost per unit, throughput.
How the map feeds a hybrid stack
The routing map is not a one-time sorting into two bins. Every workflow that goes local carries a threshold, and the threshold is where the hybrid stack lives: a local model processes every unit, a confidence layer scores each result, and the units below the line go to a frontier API with personal data removed or to a review queue. For a document workflow that is usually about 95% accepted locally and 5% routed on. The map records that split per workflow, which is what lets you size the GPU for the 95% and budget the API for the 5% instead of paying frontier prices on all of it.
For the workflows that score local but where you would rather not run the hardware, bulk document processing delivers the same routing as a managed pipeline: local models for the volume, validation and review on the exceptions, output to CSV, JSON or an API. What is on-prem, what is managed and what stays on the API are all lines on the same map; the on-premise document AI primer explains the first of those in more detail.
To start this week: list your recurring workflows, score the top ten with the table above, and multiply each total by its monthly units. The workflow at the top of that list is your first local candidate, and the export you already have tells you what it costs today.