Series
Local AI Toolkit: Runtimes, GPU Backends, and Model Files
Three practical primers on choosing llama.cpp, vLLM, or LM Studio; understanding CUDA, ROCm, Vulkan, and Metal; and reading GGUF, quantization, dense-model, and MoE labels. A companion to the standalone model-architecture guide and Running AI Yourself.
3 parts · first published
- 01
llama.cpp, vLLM, or LM Studio: Which Local LLM Tool Does What?
Separate the desktop app, inference engine, model file, and GPU backend before choosing how to run a local language model.
·7 min read - 02
CUDA, ROCm, Vulkan, and Metal: Why the GPU Software Stack Matters for Local AI
CUDA is neither a gaming requirement nor an empty AI slogan. Understand GPU backends, AMD's ROCm, Vulkan compute, and Apple's Metal before comparing local inference hardware.
·6 min read - 03
GGUF, Quantization, Dense, and MoE: How to Read a Local Model Download
Decode model filenames and parameter counts without confusing a file format, numerical precision, expert routing, and the memory needed to run a conversation.
·6 min read