Stop Paying for Copilot: Run Code-Llama Locally in VS Code

Stop paying for Copilot. A quantized 7B or 13B parameter Code-Llama model running on a mid-range GPU now produces completions that, in targeted use, rival what Copilot offers—and it never sends your keystrokes to a server. The integration path through Ollama and the Continue extension for VS Code is smooth enough to be a daily driver. This is not a magic bullet. It is a deliberate trade-off: you give up some inference speed and a slice of accuracy in exchange for privacy, offline access, and a zero-dollar price tag. For a growing number of programmers, that trade-off is worth it.

What You’ll Need

The hardware floor is lower than you might expect. A GPU with at least 6 GB of VRAM can run the 7B model at 4-bit quantization—that includes an RTX 2060, a GTX 1660 Ti, or an Apple M1 with 16 GB of unified memory. For the 13B model, 8 GB of VRAM is the practical minimum; an RTX 3060 12GB or an M2 Max handles it without swapping. If you lack a discrete GPU, the 7B model will run on CPU only, but completions take 2–5 seconds on a modern 8-core processor—usable for autocomplete, painful for chat. The software stack is three pieces: Ollama, which manages the model and exposes an OpenAI-compatible API; the Continue VS Code extension; and the model itself. You’ll also want 20–30 GB of free disk space. The 7B model file is about 4 GB and the 13B around 8 GB, but Ollama keeps cached versions. Everything runs locally. No network calls leave your machine after the initial model download.

Setting Up the Model

Start with Ollama. Download the installer from its site and run it. On macOS, it’s a standard .dmg; on Windows, an MSI; on Linux, a one-line curl pipe to bash. Once installed, the Ollama daemon runs in the background. Pull the Code-Llama model you want. The 7B instruct variant is the safest starting point:

ollama pull codellama:7b-instruct

That downloads the 4-bit quantized version. If your GPU can handle it, pull the 13B:

ollama pull codellama:13b-instruct

There are also specialized versions: codellama:7b-code for fill-in-the-middle tasks, and codellama:python fine-tuned on Python. The instruct variants are better for chat and explanation; the code variants are optimized for completion. For autocomplete inside VS Code, the code variant is theoretically faster, but in practice the 7B-instruct is more versatile and only marginally slower. Start with instruct and switch only if you find completions too generic.

Test the model from the terminal:

ollama run codellama:7b-instruct

Type a prompt like “Write a Python function that checks if a string is a palindrome.” The response should appear in under a second on a decent GPU. If it takes longer than 3 seconds, check that Ollama is using the GPU: run ollama ps and look for “gpu” in the output. On macOS with Apple Silicon, it will use the Neural Engine by default; on NVIDIA systems, you may need to ensure CUDA drivers are installed. CPU-only fallback is automatic but slow.

Integrating with VS Code

The bridge is the Continue extension. Install it from the VS Code marketplace. Open its settings (the gear icon in the Continue sidebar or ~/.continue/config.json). Add a model entry that points to Ollama’s local API:

{
  "models": [
    {
      "title": "Code-Llama 7B",
      "provider": "ollama",
      "model": "codellama:7b-instruct",
      "apiBase": "http://localhost:11434"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Code-Llama 7B Autocomplete",
    "provider": "ollama",
    "model": "codellama:7b-code",
    "apiBase": "http://localhost:11434"
  }
}

The models array controls the chat model. The tabAutocompleteModel object controls inline completions. You can point both to the same model, but using the code-specific variant for autocomplete often yields tighter completions. The apiBase must match Ollama’s default port, 11434, and the model name must exactly match the tag you pulled. After saving, restart VS Code or run the “Continue: Reload config” command.

Open any code file and start typing a function signature. After a delay of 300–800 ms, a ghost-text suggestion will appear. Press Tab to accept. The chat panel (Cmd+L / Ctrl+L) lets you highlight code and ask for explanations, refactors, or tests. The experience is not quite as polished as Copilot’s—completions sometimes trail off mid-line, and the chat can ramble. But it works, and it works entirely offline.

Performance and Limitations

Be blunt about what you lose. The 7B quantized model has roughly the reasoning capacity of a junior developer who skimmed the documentation. It will generate correct boilerplate, complete repetitive patterns, and write simple functions reliably. It will also hallucinate API methods, invent non-existent parameters, and suggest insecure patterns. The 13B model reduces these failures, but the error rate is still higher than Copilot’s underlying GPT-4o. For Python and JavaScript, the gap is narrow; for Rust, Go, or C++, the local model fumbles more often because its training data is thinner. The table below lays out the differences plainly.

Context window size matters more than you’d think. Copilot can see your entire file and multiple open tabs. Code-Llama’s default context window is 4,096 tokens—roughly 3,000 words of code. That means it often misses the broader structure of your project, leading to completions that don’t match your variable names or function signatures from earlier in the file. You can mitigate this by keeping files shorter and using the chat panel to feed relevant snippets, but it’s a real friction point.

Another limitation is fill-in-the-middle (FIM) quality. Copilot excels at reading the code before and after your cursor and generating a coherent middle. Code-Llama supports FIM, but the 7B version frequently produces syntactically broken insertions. The 13B code variant is better, but you’ll still see more garbage completions than with Copilot. The workaround is to use the chat panel for larger edits rather than relying on inline completions for anything non-trivial.

Comparison with Copilot: When the Local Model Wins

There are scenarios where the local model is genuinely superior. First, latency consistency. Copilot’s cloud latency varies wildly with load; during peak hours, completions can stall for 2–3 seconds. The local model’s latency is a flat line—predictable and unaffected by internet conditions. If you work on a train or in a coffee shop with spotty Wi-Fi, the local model keeps working. Second, privacy-sensitive codebases. Healthcare, finance, and defense projects often prohibit sending code to third-party servers. A local model eliminates the legal and compliance headache. Third, cost at scale. A team of 20 developers paying $19/month each spends $4,560 per year on Copilot. Running a local model on existing hardware costs nothing beyond the electricity. If your organization already provides developers with decent GPUs, the savings are real.

The local model also wins on customization. You can fine-tune Code-Llama on your own codebase using LoRA, creating a model that understands your internal libraries and naming conventions. That is not possible with Copilot’s black-box service. For a company with a monorepo full of internal DSLs, a fine-tuned 7B model can outperform Copilot on domain-specific tasks. The fine-tuning process is not trivial—it requires a few hundred megabytes of curated examples and an afternoon of GPU time—but it is a capability that simply does not exist in the SaaS offering.

Setting Up for Daily Use

To make the local model tolerable as a daily driver, tweak a few knobs. In Continue’s config, set maxTokens for completions to 150–200. The model tends to ramble; limiting output keeps it concise. Enable useCache to avoid re-processing identical prompts. For the chat model, set temperature to 0.2 for deterministic answers, or 0.7 if you want creative suggestions. A temperature of 0.2 reduces hallucination significantly without making responses robotic. If you have the VRAM, run both the 7B and 13B models simultaneously: the 7B for fast autocomplete, the 13B for chat. Ollama handles concurrent models gracefully. In the config, list both models and assign them accordingly. For a deeper look at what different quantizations do to model quality on consumer hardware, see the Llama 3.1 8B Quant Shootout.

One practical tip: use the chat panel as a pre-filter. Before accepting a suggested refactor, ask the chat model to explain it. If the explanation contains obvious nonsense, reject the edit. This adds a few seconds but catches the worst hallucinations. Over time, you develop an intuition for what the model handles well—regular expressions, SQL queries, unit test generation—and what it doesn’t—complex state machines, async error handling, security-critical logic. Treat it like a fast but careless intern: verify everything it produces.

The Privacy Argument in Concrete Terms

The privacy argument is not abstract paranoia. When you use Copilot, every keystroke, every file context, and every accepted suggestion flows through GitHub’s servers. That data is logged, used for model training by default (though you can opt out), and subject to legal requests. For a solo developer working on a side project, the risk is minimal. For anyone dealing with proprietary algorithms, customer data, or regulated code, the risk is unacceptable. A local model sidesteps the entire issue. No data leaves the machine. You can verify this by disconnecting your network and watching the model work identically. That auditability matters.

There is a subtler privacy benefit: the model’s training data is frozen. Copilot’s model updates continuously, which means its behavior can change without notice. A completion that worked yesterday might vanish today because the underlying model was retrained. With a local model, you pin a specific version. Your tool behaves the same way tomorrow as it does today. Determinism in developer tools is undervalued.

Conclusion

Running Code-Llama locally in VS Code is not a drop-in replacement for Copilot. It is a different tool with different trade-offs. The setup takes 15 minutes. The completions are slower and less accurate. The context window is smaller. In exchange, you get a coding assistant that works offline, respects privacy, costs nothing, and can be customized. For many developers, that bundle is already good enough. As quantization techniques improve and smaller models get smarter, the gap will narrow further. The real question is not whether a local model can match Copilot today, but whether the trajectory of local AI will make the $10 monthly fee feel increasingly unnecessary. Try it for a week. If you find yourself reaching for the chat panel more than you expected, and if the privacy and cost savings matter to you, cancel the subscription. The code you write belongs to you. The assistant that helps you write it can, too.