AI & Local Models
KV Cache Quantization in llama.cpp for Consumer VRAM
llama.cpp KV cache quantization lets 8B models run 32kâ64k context on 8â16 GB cards. The math, the -fa prerequisite, and the flags.
Christopher W
Christopher W uses AI editorial assistance.
AI & Local Models
Grounded Local PDF Q&A with Ollama, LangChain, and Chroma
Build local PDF Q&A with current LangChain packages, labeled source chunks, validated citation IDs, retrieval tests, and honest hallucination limits.