Building a RAG workstation on a refurbished tower

Retrieval augmented generation has quietly become the most useful thing you can run locally. Point a small language model at a folder of PDFs, meeting notes, contracts and emails, and it will answer questions with citations to the source documents. No cloud upload, no monthly fee, no data leakage. The workstation matters more than most people expect. Not because the model is heavy (a 7 to 14 billion parameter model runs on modest hardware) but because retrieval is I/O intensive and embedding is CPU bound without a GPU to offload to.

What a RAG stack actually asks of the hardware

A RAG pipeline has three moving parts. The embedder turns each document chunk into a vector, typically 384 to 1024 dimensions. The vector database stores those vectors and returns nearest neighbours for a query. The language model reads the retrieved chunks and writes an answer. Only the last part is glamorous. The first two decide whether your machine feels responsive or sluggish.

Embedding a 10,000 document corpus with a modern model like BGE-small takes roughly 30 to 90 minutes on a decent CPU, and closer to 5 minutes on a mid-range consumer GPU. Once the index is built, live queries are cheap. The expensive part is the initial ingest. Plan the box around ingest, not query.

Why a 14th Gen i7 SFF is the sweet spot

The Birmingham AV i7 14th Gen small form factor tower sits in a useful pocket of the market. The i7-14700 has 20 cores (8 performance, 12 efficiency), which matters because embedding libraries like sentence-transformers parallelise across cores well. Configurations run from 16GB up to 64GB DDR5 RAM, with storage from a single 512GB NVMe up to dual drive builds with 2TB of fast storage. For a personal knowledge base of up to about 500,000 chunks, that is comfortable headroom.

The chassis sits under a desk without dominating it, runs quiet on the efficiency cores when idle, and pulls about 65W at the wall during light retrieval. Peak draw during a full ingest sits around 180W, well within the standard 260W supply.

View the current BAV i7 14th Gen SFF listing on eBay

Choosing a vector database

The three serious options for local use are Chroma, Qdrant and LanceDB.

Chroma is the easiest to start with: a single Python import, an in-memory or on-disk store, indexing inside 20 lines of code. It scales cleanly to a few hundred thousand vectors on a 32GB machine.

Qdrant runs as a proper service (a small Rust binary or a Docker container) and handles millions of vectors. It supports payload filtering, which you will want the moment you restrict retrieval to a specific project or date range. For a serious personal knowledge base, this is the pick.

LanceDB stores vectors in a columnar format on disk, so it can query datasets larger than RAM without paging misery. All three run happily on the i7 SFF spec.

Embedding model requirements

The embedding model is the piece most people over-specify. You do not need a 7B parameter embedder. The current sweet spot is BGE-small-en-v1.5 (384 dimensions, roughly 130MB on disk) or Nomic Embed Text v1.5 (768 dimensions, roughly 550MB). Both run on CPU at 200 to 400 chunks per second on a 14th Gen i7.

If you fit a discrete GPU (the chassis accepts a low-profile card up to about 200mm, so an RTX 4060 low-profile fits), embedding throughput jumps to around 3,000 chunks per second. For most personal knowledge bases, the small models are indistinguishable in practice.

At 768 dimensions and float32, one million chunks is 3GB of raw vectors, plus index overhead of roughly 40%. The 1TB NVMe on the mid-tier configuration handles this without noticing.

Software stack that just works

The combination that has proven most reliable is Ollama for the language model runtime, Qdrant for the vector store, and LangChain or LlamaIndex as the orchestration layer. Ollama pulls quantised GGUF models with a single command, exposes an OpenAI compatible API on localhost, and handles model swapping cleanly. A 14B Qwen or Llama 3.1 model in Q4 quantisation runs at 8 to 15 tokens per second on the i7 CPU, readable in real time.

Add a lightweight front end (Open WebUI or a custom Next.js page) and the whole stack lives on one machine, on your own network.

What the build costs

At current pricing, the BAV i7 14th Gen SFF sits between £449 and £749 depending on configuration. The 32GB RAM, 1TB NVMe spec at around £599 is the point where nothing feels compromised for a personal RAG workstation. All configurations ship with a twelve month warranty and Windows 11 Pro licensed. Linux (recommended for the software stack above) installs cleanly.

Compared to a new equivalent workstation at £1,400 to £1,800, the refurbished route pays back quickly.

FAQ

How much RAM do I actually need?

For a corpus under 100,000 chunks with a 7B model in Q4 quantisation, 16GB is workable but tight. 32GB is comfortable and leaves headroom for a browser and code editor alongside. Go to 64GB only for a 14B model with a large context window running against a heavy in-memory index.

Can I run this without a GPU?

Yes. CPU-only inference on a 14th Gen i7 gives readable output speeds on 7B models and workable speeds on 14B models. Embedding is slower on CPU but still finishes overnight for most personal corpora. A GPU speeds things up but is not required.

How do I stop the model hallucinating over my documents?

The retrieval layer is the answer. Increase top-k to 6 or 10 rather than the default 3, use a reranker like BGE-reranker-base on the retrieved set, and instruct the model to cite chunk IDs in its answer. If a claim has no citation, treat it as unreliable.

What if my knowledge base grows past a million documents?

Move Qdrant onto a dedicated volume, enable disk-based indexing rather than in-memory, and consider a hybrid retrieval approach (BM25 for keyword recall, dense vectors for semantic recall, then rerank). The i7 SFF still handles the load, but ingest grows to several hours.

About Birmingham AV

Birmingham AV Ltd has sold over 87,000 items on eBay since 2017, with 24,756 buyer feedbacks at a 98.9% positive rating. We are one of the highest volume refurbished PC operations on eBay UK, and every machine ships with a twelve month warranty. Companies House registration 12383651, VAT number GB 348755066, based in Bromsgrove, Worcestershire.