Lesson9 min

Combining local and cloud

The question is rarely local or cloud, but: which task goes where? A routing rule from two axes – confidentiality and difficulty – four patterns that hold up, and the places where data leaves unnoticed after all.

Boaz Lichtenstein

Part of the path Local AIstation 7 of 7level 3 Expert

Anyone who has set up a local model faces a new question: when do I use which? The answer “always local” fails on the hard tasks; “always cloud” makes the set-up pointless. What holds up is a rule.

Two axes

Every task has two properties that decide where it goes:

Confidentiality. May the data leave your premises? That is not a matter of taste but of contracts, data protection and professional rules.

Difficulty. Is a small model enough – or does it take long chains of reasoning, a lot of context, reliable tool calls?

	Easy	Hard
Confidential	local	local with compromises – or reduce, then decide afresh
Not confidential	local or an inexpensive cloud model	cloud

Three of the four fields are clear. The difficult one is top right – and there is a pattern for it.

Pattern 1: pre-processing locally

The local model does not have to solve the task. It is enough for it to prepare the data so that the rest may go out:

replace names, addresses, customer numbers with placeholders
pick the two relevant paragraphs out of a hundred pages
turn free text into a table with fixed fields

Those are tasks small models do well. What then goes to the cloud is not the file but an anonymous extract. Check the anonymisation on samples – a model occasionally overlooks a name in running text.

Pattern 2: local for volume

Classifying a thousand emails, checking ten thousand product descriptions for missing details, tagging an archive: many similar, simple steps. In the cloud you pay for each one; locally only for the electricity. From what volume that pays off is shown by the sample calculation in Local AI vs. cloud.

Pattern 3: cloud for the hard part, deliberately

Planning across many steps, large documents as a whole, agents with many tools – here the big cloud models are clearly better. For non-confidential tasks that is the right choice, without a bad conscience.

Pattern 4: local agent, narrow task

An agent with a local model works when the task is narrow: few tools, short flows, a clear result. The more tools and steps, the more often a small model calls the wrong tool or loses the thread. So keep local agents to one thing.

The fallback is not a technical question

Many tools offer to switch automatically to a cloud model when the local one gets stuck. For non-confidential tasks that is convenient. For confidential ones it is exactly the leak you wanted to prevent – triggered, of all things, by the difficult cases.

Therefore: for confidential tasks the fallback is off. If the local model fails, the run ends with a message and a human decides.

Where data leaves unnoticed

Even with a local model there are ways out:

Web search. Your question – together with pasted context – goes to a search engine.
Side models. Images, speech or search within documents sometimes run through a different, remote service.
Connectors. A connected cloud service receives what you send through it.
Telemetry and crash reports from the tool.

For confidential work a separate profile of the tool, with all of that switched off, is worth having.

The rule in three lines

Write it down – for yourself and for everyone who works with you:

Always local:  staff, contracts, customer data, anything under
               confidentiality. No fallback to the cloud.
Cloud allowed: public material, own texts, anonymised extracts.
In doubt:      local. Exceptions are decided by <name>.

Measure rather than assume

Whether the local model is “enough” for a task is a question that can be measured: the same ten cases, once locally, once in the cloud, the same scoring. How to build such a test set is the first station of the next path – Running agents.