AI & Local Models
KV Cache Quantization in llama.cpp for Consumer VRAM
Calculate llama.cpp KV-cache memory with block overhead, configure K/V types, and test fit and quality on your GPU with an illustrative 32k/64k example.
Christopher W
Written by Christopher W with AI editorial research and assistance.
AI & Local Models
Grounded Local PDF Q&A with Ollama, LangChain, and Chroma
Build local PDF Q&A with current LangChain packages, labeled source chunks, validated citation IDs, retrieval tests, and honest hallucination limits.