Running a local model

A few years ago a model of your own was a tinkering project. Today it is a download. You need a program that runs models, and a model – a file of a few gigabytes.

Two tools

Ollama comes from the command line and is set up in a few commands. It is particularly suitable when other programs are to use the model.

LM Studio has a graphical interface: search for a model, load it, chat. The way in without a terminal. Its agent variant is covered in LM Studio Bionic.

Both can be used free of charge, both run on macOS, Windows and Linux. Start with whichever suits the way you work – the models behind them are the same.

The route with Ollama

After installing from the project’s website, one command is enough:

ollama run gemma4:e2b

Ollama downloads the model – for this example from the official guide about seven gigabytes – and opens a chat in the terminal. /bye ends it.

ollama pull <model>     download a model without chatting
ollama list             what is on this machine?
ollama run <model>      start a chat

Which model you take depends on your memory. How to work that out in advance is the subject of the next station. For the first attempt: rather a small model that runs smoothly than a large one that keeps you waiting. Ollama quickstart

The local interface

As soon as Ollama is running, it listens for requests on your machine at http://localhost:11434. That is the real value: any program on your machine can use the model.

The interface speaks the formats the big providers use as well. Many tools built for a cloud model can therefore be switched to the local one by changing the address. Ollama ships a shortcut for that: ollama launch starts a coding agent such as Claude Code, Codex or OpenCode directly against a model of your choice.

The trap: cloud inside the local tool

“Local tool” no longer automatically means “local processing”. After sign-in, Ollama also offers cloud models that compute on the provider’s servers. LM Studio Bionic lets you choose between local, a second device of your own and the cloud.

That is handy when your own hardware is not enough. It is a problem when you assume your draft contracts stay in-house.

Three checks:

1. The network test. Wifi off, ask a question. If an answer comes, computing is local.
2. The model list. Does the selected model carry a note about cloud or an account?
3. The side models. Some tools use a different model for web search, images or speech than for text. Check every setting individually.

What you can expect

A model that fits on a laptop is not the model from your cloud subscription. It is smaller, and it shows.

Good: summarising, rephrasing, translating, pulling data out of texts, sorting and classifying, simple drafts.

Weaker: long chains of reasoning, large documents as a whole, reliable tool calls across many steps – exactly what agents need.

That is not a defect but a division of labour. Many everyday tasks are of the first kind, and for those you pay nothing per request locally and send nothing away.

The first evening

Give the model five tasks from your real work, not from a benchmark: summarise an email, describe a table, shorten a paragraph. Note where it is enough and where it is not. That list is the basis for the question that closes the path: what stays local and what may go to the cloud.