Putting Ollama and Continue in VS Code gives you a coding workflow whose inference path you can control. The useful question is how its chat, editing and autocomplete perform on your own repository, with the machine and context window you actually have. This guide sets up the YAML configuration and uses that working setup as the basis for a comparison with Copilot.

Decide What You Need: Chat or Autocomplete

The case for separate local models begins with the work you ask them to do. Autocomplete has to arrive quickly and fit the code around the cursor; a small code-specific model can be more usable there than a larger general chat model. Chat and editing give a larger instruction-tuned model more room to work with context and tools, while asking more of the machine's memory and your patience. Continue assigns those jobs through roles in config.yaml. One model can serve several roles, but separating them lets you tune the trade-off for each task.

Install Ollama and Verify the Backend

Install Ollama from its official download page[1] and the Continue extension for VS Code[2], then pull an explicit model tag. Continue’s current autocomplete guide uses Qwen2.5-Coder 1.5B as a local example:

ollama pull qwen2.5-coder:1.5b

Code Llama remains available for an older coding-model setup; the chat example below uses its instruction-tuned 7B tag:

ollama pull codellama:7b-instruct

Run the model once. Leave its interactive chat before running ollama ps in the terminal to inspect the loaded-model report:

ollama run qwen2.5-coder:1.5b
ollama ps

The PROCESSOR column shows whether a model is on CPU, GPU, or split. Ollama’s documented Apple GPU backend uses the Metal API.

Model size, context length, backend, unified or discrete memory, and other applications determine what fits. Latency and memory figures therefore apply only to a retained test system and workload.

Configure Continue with YAML

Continue’s config.json format is deprecated; config.yaml is the preferred format. Create or edit config.yaml in Continue’s global directory: ~/.continue on macOS or %USERPROFILE%\.continue on Windows:

name: Local coding models
version: 0.0.1
schema: v1

models:
  - name: Qwen2.5-Coder 1.5B Local
    provider: ollama
    model: qwen2.5-coder:1.5b
    roles:
      - autocomplete

  - name: Code Llama 7B Local Chat
    provider: ollama
    model: codellama:7b-instruct
    roles:
      - chat
      - edit

Continue’s Ollama provider uses http://localhost:11434 by default, so this same-machine configuration needs no apiBase. If you choose a separate inference server, set apiBase to that service’s address and arrange secure transport and authentication. The prompt then crosses the network to that host, which changes the privacy boundary even if you own both machines.

After saving, check “Enable Tab Autocomplete” through the Continue button in VS Code’s status bar or the extension settings. Use the model selector to choose the configured model for the role you want to use. If a model is missing, ollama list shows the installed tags; check that the configured tag matches one you pulled.

Continue’s model roles and schema evolve. If this example stops loading, check its current configuration and Ollama-provider documentation rather than falling back to an old JSON fragment.

Choose a context window the machine can support

Meta’s Code Llama paper says the models were trained on sequences of 16,000 tokens and reports improvements on inputs up to 100,000 tokens. Those figures describe the model's training and reported evaluation; choosing a practical context window also requires looking at what the local runner and machine can sustain.

An advertised or evaluated context length still does not mean a local runner should use the maximum. A larger context allocation increases the memory required to run the model. Continue may also request a context longer than an Ollama setup can fit. Start with the smallest context that covers the task and watch Ollama’s memory behavior.

Compare the Local Setup with Copilot

Copilot changes over time across models, plans, features, and data-handling rules, so compare the service available today with the local model running on your hardware. Measure both on the same repository tasks and include response time, acceptance rate, test results, and resource use.

GitHub’s documentation says inline suggestions combine surrounding code and contextual information such as snippets from open editor tabs, then send a prompt to a hosted model. Current data-use rules also differ by plan. GitHub says Business and Enterprise customer data is not used to train AI models. Since April 24, 2026, interactions on Free, Pro, Pro+ and Max plans may be used for training and improvement; individual users can disable “Allow GitHub to use my data for AI model training” in their Copilot settings. Review the documentation and data-use setting for the plan you use.

Keeping the Ollama inference prompt on the machine is a concrete advantage when that prompt contains sensitive code. It is only one part of the workflow, however. Configure the intended local provider for every sensitive role and check Continue’s tools, context providers and development-data destinations; its configuration supports HTTP destinations as well as local files. Inspect VS Code and extension telemetry separately, and disable remote services you do not mean to use. Ollama also offers local-only mode with OLLAMA_NO_CLOUD=1, followed by a restart. Once the models are downloaded and the intended configuration is in place, try the workflow disconnected from the network. If the threat model calls for closer scrutiny, inspect its connections as well: source control, package managers and other extensions can still use online services even when the model runs locally.

Run a Task-Specific Evaluation

The comparison becomes useful when it resembles the work you normally do. Give both assistants a function body to complete with tests, a gap with an existing prefix and suffix, and a bounded rename or refactor. Add an explanation of a project-specific API from supplied context, then ask for tests against seeded bugs to see whether the generated tests catch them. Include a security-sensitive proposal that should be rejected or corrected in review; accepting that change would count against the assistant, even if the code compiled.

Record accepted suggestions as a share of all suggestions shown, the time from a pause to a useful suggestion, and the compile and test pass rate. Keep the security mistakes and invented APIs beside those results, along with how much editing each accepted suggestion needed and whether the task fit within the configured context window. That record gives the comparison a denominator and keeps speed from hiding expensive corrections.

Use the same files and task order where practical. Hosted products may change during the test, so record the date, plan, model selection, extension versions, Ollama version, and local model digest.

Daily-Use Trade-offs

A small code-specific model is worth trying first for short completions and familiar patterns, then judging against the tasks above. For chat or editing, compare it with a larger instruction-tuned model only if your machine can accommodate that model and the needed context. Record the actual resource use and response time alongside the quality of its changes rather than treating parameter count as a performance result.

Copilot provides hosted capacity and integrations that a local setup may not reproduce. Its current features and prices should be checked on GitHub rather than copied into a static comparison table. The local setup’s advantages are control over the model artifact, offline inference, and the option to keep model prompts on the machine; its costs are hardware, electricity, setup time, and model limitations.

Before attempting fine-tuning, I would try supplying the relevant repository context and measuring where that still fails. If you proceed to training, budget for permission to use the examples, representative tasks and regression evaluation. Those are part of the comparison with a simpler context-supplying approach, and the decision should turn on the failures you have actually observed.

Review Everything the Model Writes

Whichever model you choose, review its suggestions as patches that still have to earn acceptance. Both local and hosted models can invent APIs, omit error handling, introduce insecure defaults or pass shallow tests while missing the requirement. Read the complete change, run the formatters, type checks and tests, and add tests for the actual behavior requested. Review any dependencies and generated commands too, with particular scrutiny for authentication, cryptography, permissions and data deletion. A useful assistant reduces the work around that judgment; it does not remove the need to make it.

Sources