Air gapped AI workstation: the privacy first PC build
Running large language models on a machine that never touches the internet. A practical hardware, OS, and network guide for regulated sectors and privacy conscious builders.
The phrase "air gapped" used to belong to defence contractors and central banks. In 2026 it belongs to anyone who wants to run a capable AI model on premises without a packet leaking. Law firms, GP practices, accountants, and the merely paranoid all want a workstation that can host a 30B or 70B parameter model and physically cannot exfiltrate anything. This is a build guide.
Why air gap, and why now
Cloud AI is convenient. It is also a data processing agreement waiting to bite you. The Information Commissioner's Office has been vocal about controllers who bolt on ChatGPT without a lawful basis. For anything covered by UK GDPR special category rules (health, legal, biometric, financial), the safest posture is that the data never leaves the room.
Local models have caught up. Llama 3.3 70B, Qwen 2.5 72B, and the smaller Mistral and Gemma variants now match GPT 4 class output on many professional tasks. Quantised to 4 bit, a 70B model fits inside 48GB of VRAM. Quantised to 8 bit, a 30B model fits inside 24GB. Those numbers dictate the graphics card, and the graphics card dictates the rest of the machine.
The hardware target
For a serious privacy first workstation, aim at this specification band:
- CPU: Intel Xeon W or AMD Ryzen 9 / Threadripper class, 16 cores minimum, 32 preferred.
- RAM: 128GB ECC DDR5 as the sweet spot. 64GB works for 30B models, 256GB future proofs you.
- GPU: single RTX 4090 (24GB VRAM) at entry level, dual RTX 4090 or a single RTX 6000 Ada (48GB) for 70B work.
- Storage: 2TB NVMe Gen 4 for OS and weights, plus 4TB SATA SSD for datasets. Weights can hit 140GB per 70B checkpoint.
- PSU: 1200W platinum minimum for a single 4090, 1600W for dual GPU.
- Chassis: full tower with real airflow. Fractal Define 7 XL, Corsair 7000D, or a Lian Li O11 Dynamic XL.
A machine of that class from a mainstream integrator will land in the £4,500 to £7,500 range. Refurbished workstation grade hardware from a used Xeon or Threadripper platform cuts that by forty percent without losing ECC memory or PCIe lane count.
See the Birmingham AV workstation build on eBay
Choosing an operating system
Windows 11 Pro will run local inference stacks such as Ollama, but it also runs telemetry, Recall, and background phone home services. For a genuinely air gapped machine, Linux is the pragmatic choice.
Ubuntu 24.04 LTS is the path of least resistance because Nvidia's drivers, CUDA toolkit, and cuDNN packages target it directly. Fedora 41 is a strong second choice. For the truly cautious, Debian 12 with a hardened kernel and AppArmor profiles gives you an audit trail a compliance officer can read. On first install, unplug the ethernet cable, complete set up offline, then pull packages onto a USB stick from a separate machine.
The network setup that is actually air gapped
An air gap is not a firewall rule. It is the absence of a cable. But most users still want the machine to reach a printer or a local NAS. Three sensible topologies:
- True air gap. No network card enabled at all. Updates arrive on a write once USB stick scanned on a separate quarantine machine first.
- Isolated VLAN. The workstation sits on a VLAN with no route to the internet. Local file share reachable, upstream routers not. This is the pattern most law firms deploy.
- Egress deny by default. The machine has an internet capable NIC, but the router blocks all outbound traffic from its MAC address except to a named internal DNS resolver.
Whichever topology you choose, disable Wi Fi and Bluetooth in BIOS, not just in the OS. Physically removing the wireless card is proportionate for anything regulated.
Software stack, all local
Ollama for a friendly command line and a simple HTTP API on localhost. llama.cpp underneath for raw tuning. Open WebUI as the browser front end, running entirely on 127.0.0.1.
For document ingestion, run a local vector database (Qdrant or Chroma), a local embedding model (BAAI bge large en, or Nomic embed text), and a loader that never touches a cloud OCR service. Tesseract handles scanned PDFs. Whisper.cpp handles audio transcription. Back weights up to an offline drive in a fire safe.
What this actually costs to run
Idle power draw sits around 90W. Under sustained inference on a single 4090, expect 450W to 550W. At UK business tariffs of roughly 27p per kWh, running eight hours a day at 500W costs about £30 a month. Compare that with a mid tier ChatGPT Enterprise seat at around £50 per user per month, and the machine pays for itself inside eighteen months for a five person practice.
Frequently asked questions
Can I really match GPT 4 quality on a local model?
For writing, summarisation, extraction, and code assistance, a well quantised Llama 3.3 70B or Qwen 2.5 72B produces output that experts rate as indistinguishable in blind tests. Frontier reasoning still favours the cloud. For everyday knowledge work, the gap has closed.
Do I need two graphics cards?
Not for a 30B model at 8 bit precision. A single RTX 4090 with 24GB VRAM handles that comfortably. Two cards, or a single RTX 6000 Ada with 48GB, become necessary once you move to 70B class models at usable quantisation levels.
Is Linux really required?
No, but it is strongly preferred. Windows 11 Pro can be locked down and telemetry blocked at the firewall, but the burden of proof for a compliance audit is much higher. Linux gives you a smaller attack surface and free licences.
How do I update models without breaking the air gap?
Use a dedicated download station that fetches weights, verifies SHA256 hashes, and writes to a USB drive. The drive is scanned on a quarantine machine, then physically carried to the workstation. This is the same pattern used for classified networks.
About Birmingham AV
We have sold over 87,000 items on eBay since 2017, with 24,756 buyer feedbacks at 98.9% positive. Every machine ships with a twelve month warranty. Birmingham AV Ltd is registered at Companies House number 12383651, VAT registered as GB 348755066, and based in Bromsgrove, Worcestershire. One of the highest volume refurbished PC operations on eBay UK, we specialise in workstation grade hardware fit for exactly this kind of build.