Ollama explained: every PC buyer should know about it in 2026

Two years ago, running a capable large language model on your own PC meant wrestling with Python environments, CUDA versions, and loose weight files. In 2026, you double click an installer, pick a model from a menu, and start chatting inside a minute. That shift has a name: Ollama.

If you are buying a PC in 2026, even if you have never touched a chatbot, this is worth ten minutes.

What Ollama actually is

Ollama is a lightweight runtime that downloads, manages, and serves open weight language models on your own machine. Think of it as the equivalent of Docker for AI models: one command pulls a model, one command runs it, and everything sits behind a clean local API on port 11434.

It is not a model itself. It is the plumbing that makes models like Llama 3.3, Mistral, Qwen 2.5, Gemma 3, and Phi 4 behave the same way from any code that can talk to an OpenAI compatible endpoint. Any tool built for OpenAI can be pointed at Ollama with two lines of config, and it just works.

Before Ollama, local AI was a hobby. You needed to know what a quantisation format was, which fork of llama.cpp to compile, and how to route GPU drivers around three different Python stacks. Ollama collapsed that into a one line install. Small businesses can now run a private assistant that never sends customer messages to a third party, and developers can build features that would cost thousands per month in API bills for zero marginal cost. A machine that was overkill for browsing and email in 2024 is now the entry level for anyone who wants an AI assistant running in the background.

The model library, plain English

Ollama's library covers three broad tiers.

Small (1B to 4B parameters). Phi 4 Mini, Gemma 3 4B, Llama 3.2 3B. These fit in 4GB to 6GB of memory, run on integrated graphics, and handle summarising, tagging, and simple chat.

Mid (7B to 14B parameters). Llama 3.1 8B, Mistral 7B, Qwen 2.5 14B, DeepSeek R1 Distill 14B. Good enough for coding help, drafting, and structured data extraction. They want 8GB to 12GB of VRAM.

Large (30B to 70B parameters). Llama 3.3 70B, Qwen 2.5 32B, Mixtral 8x7B. These rival hosted GPT class models, but need 24GB of VRAM for the 32B tier, and 48GB or more for a 70B model.

For most home and small office use, the mid tier is the sweet spot.

Quantisation and GPU offload, without the jargon

Quantisation is compression for model weights. A raw 14B parameter model at full precision is roughly 28GB. Quantised to 4 bit (the Q4_K_M format Ollama uses by default), it drops to around 8GB with only a modest quality hit.

GPU offload is Ollama deciding how much of the model lives in fast video memory versus slower system RAM. If the whole model fits in VRAM, you get full GPU speed. If not, Ollama splits it across the GPU and CPU, and each layer stuck on the CPU roughly halves your tokens per second.

The practical rule: pick a model whose quantised size fits inside your VRAM with a couple of gigabytes to spare for context.

The hardware Ollama actually demands

For light use with small models, any modern PC with 16GB of RAM will do. For serious use:

• RAM: 32GB is the floor. 64GB is the comfortable answer if you also run browsers and an IDE alongside the model.
• GPU: VRAM is the single most important spec. An RTX 3060 12GB or RTX 4060 Ti 16GB handles most 7B to 14B models cleanly. A used RTX 3090 with 24GB remains the best second hand value for 32B class models.
• CPU: Any Ryzen 5 or Core i5 from the last four generations is enough. Ollama is GPU bound when a GPU is present.
• Storage: NVMe, not SATA. Model files are 4GB to 40GB each. 1TB minimum.
• Power supply: underrated. A 24GB GPU under sustained inference can pull 300W to 400W for minutes at a time.

Which Birmingham AV rig fits the job

Our mid tier gaming rig is built around exactly this profile. Ryzen 7 class CPU, 32GB of DDR4 or DDR5 depending on the variant, a discrete GPU with enough VRAM to hold a Q4 quantised 14B model in full, and an NVMe boot drive with room for a model library.

See the current mid tier build on eBay

Listing variations matter. Depending on the trim, the GPU ranges from an RTX 3060 12GB through to a 16GB card, and storage moves between 500GB and 2TB. If Ollama is on your shortlist, aim for the 12GB VRAM floor and 1TB of NVMe as a minimum. The 16GB variant opens the door to 14B models at longer context lengths.

FAQ

Is Ollama free?

Yes, Ollama itself is open source and free to download and use, including for commercial work. The models it runs are also free, though each model has its own licence. Most are permissive, a few restrict commercial resale of the model weights themselves.

Can I run Ollama without a GPU?

Yes, but only small models will feel usable. A 3B or 4B model on a modern CPU with 16GB of RAM will do simple chat and summarising at reading speed. Anything above 7B on CPU alone is slow enough that you will stop using it after a week.

How much VRAM do I need for a 14B model?

Roughly 10GB to 12GB of VRAM covers a 14B model at Q4 quantisation with a normal context window. Push the context out to 16k or 32k tokens and you need closer to 14GB. A 16GB card is the safe answer if you want headroom.

Does Ollama send my data anywhere?

No. Everything runs locally on your machine, and the model never phones home. That is the entire point of the tool, and it is the main reason legal, medical, and financial users have adopted it over hosted alternatives.

About Birmingham AV

We have sold more than 87,000 items on eBay since 2017, with 24,756 buyer feedbacks at 98.9% positive. Every system ships with a twelve month warranty, and we are based in Bromsgrove, Worcestershire. Companies House 12383651, VAT GB 348755066. One of the highest volume refurbished PC operations on eBay UK.