VS Code Settings for Better Local LLM Completion and Chat

For better local LLM code completion and chat in VS Code, configure the extension that connects to your model: choose the correct local provider and model, enable the features you need, and tune context, output length, and completion behavior to fit your hardware. There is no universal VS Code setting that improves every local model; extension settings, model capability, and available memory all matter.

Which settings matter most

VS Code itself does not provide a single built-in switch for local LLMs. An extension supplies the model connection and its own settings, so exact labels and available options vary. Common choices include extensions that connect to a local model server or runtime, and extensions that manage models themselves. Check the extension’s documentation for supported providers and setting names rather than copying configuration keys from a different tool.

Start with these settings or their equivalents in your chosen extension:

• Provider or server endpoint: Select the local provider and enter the address and port used by its running service. A mismatch prevents requests from reaching the model.
• Model identifier: Choose the model actually installed or served locally. Some providers require a tag or exact model name, not a general family name.
• Completion provider and trigger: Enable inline or tab completions if supported. Decide whether suggestions appear automatically, after a delay, or only when requested.
• Chat context: Allow the extension to include selected code, open files, or workspace content when appropriate. More context can help, but it also takes processing time and may exceed a model’s context limit.
• Maximum output tokens: Set a practical response limit. Short completions usually need less output than explanations or code transformations.
• Temperature: If exposed, use a low value for predictable code completion; a somewhat higher value can make exploratory chat less rigid. The useful range depends on the model and extension.

These are starting points, not guaranteed improvements. Test changes one at a time against a representative task so you can tell whether a setting helped.

How local completion and chat work

A VS Code extension gathers a prompt from the editor, sends it to a model through a local service or runtime, and displays the returned text. For inline completion, the prompt commonly includes nearby code and the cursor position. For chat, the extension may also send your question, selected code, conversation history, and files you explicitly attach or ask it to inspect.

“Local” describes where inference runs, not necessarily how every part of the extension works. Model generation may happen on your computer while extension features, downloads, or optional integrations behave differently. Read the extension’s privacy and provider documentation to understand what is sent where. Avoid assuming that a local model automatically means every prompt or related feature stays on-device.

Generation speed depends on the model, its size and quantization, the prompt length, and your machine’s available CPU, GPU, and memory. Settings cannot make a model exceed its capabilities. A smaller or more heavily quantized model may respond faster but produce less reliable code; a larger model may be more capable while using more memory and taking longer to respond.

Set up the provider and model

1. Install and start the local runtime or model server required by the extension. Confirm that it is running before troubleshooting VS Code.
2. In VS Code, open the extension’s settings, usually through the Settings view or its dedicated configuration panel. Search by the extension name if you are unsure which settings belong to it.
3. Select the local provider, then enter the endpoint and model identifier exactly as the provider expects. Use the extension’s own connection or model test if it offers one.
4. Try a short prompt in the extension’s chat panel. Once chat works, test inline completion in a small code file.

If the extension offers separate models for chat and completion, select each deliberately. A model that works well for long conversational answers may be too slow for frequent inline suggestions. Conversely, a compact completion model may not be the best choice for explaining a multi-file change.

Tune completion and context

For code completion, prioritize responsiveness and relevant context. If suggestions arrive late, first try a smaller or faster model, reduce unnecessary context, or increase the trigger delay so the extension does not request a completion after every brief pause. If suggestions are intrusive, use a manual trigger or disable automatic suggestions while retaining chat.

Context settings control how much code the extension sends, but larger is not always better. A focused selection and the current file may be more useful than a broad workspace prompt, especially when a model has a limited context window. Include related definitions or files when the task depends on them; do not expect the model to know code it was never given.

For chat, use workspace indexing or codebase search only if the extension supports it and you need repository-wide answers. Indexing can take time and consume disk space or memory. Check exclusions and respect project boundaries: generated files, dependencies, secrets, and unrelated directories rarely improve answers. Never paste credentials, private keys, or other sensitive material into a prompt.

Common problems and practical fixes

• Connection refused or no response: Confirm the local server is running, the endpoint matches its listening address, and the selected provider is correct. A firewall or proxy configuration can also affect connections.
• Model not found: Check the exact installed model name and any required tag. Refresh the extension’s model list or restart its connection after adding a model.
• Slow completions: Shorten the prompt, reduce context, use a smaller model, or check whether the runtime is using the expected hardware. Long chat history can also slow requests.
• Truncated or incomplete answers: Raise the output limit if the model and extension allow it, while keeping within the model’s context capacity. Ask for a smaller, clearly bounded change when appropriate.
• Irrelevant or incorrect code: Provide a focused prompt, include necessary surrounding code, and verify suggestions before accepting them. Model output can contain errors even when it looks plausible.

When changing a setting appears to do nothing, check whether it belongs to the active extension and whether the extension needs a reload or new chat session. Also distinguish editor settings from provider settings: VS Code’s general suggestion preferences may affect how completions appear, but cannot fix a model server that is unavailable.

Choosing settings for your machine

There is no one best model size or context length for every computer. Begin with a model that fits the memory available to the runtime, then evaluate speed and answer quality in your own projects. If the system starts swapping memory or the editor becomes sluggish, reduce the model size or context and close unnecessary workloads.

Keep completion and chat expectations separate. Inline completion benefits from short, quick responses; chat benefits from enough context and output room to explain or modify code. If the extension exposes a shared model configuration, consider a balanced model first, then move to separate models if the workflow supports it. Change one variable at a time and compare the same prompt before and after.

Frequently Asked Questions

Which VS Code extension settings should I change first?
Set the correct local provider, endpoint, and installed model. Then enable the completion or chat feature you want and adjust context and output limits only after confirming a basic request works.

What is a good temperature for local code completion?
If your extension exposes temperature, start low for consistent code suggestions. There is no universal best value; model behavior differs, and some extensions or models may ignore or hide this control.

Why does local LLM completion feel slow in VS Code?
Common causes are a large model, long context, limited memory, or CPU-only inference. Try a smaller model and shorter context before changing unrelated VS Code editor settings.

Can VS Code use one model for chat and another for completion?
Some extensions support separate model choices, while others use one model for both. Check the extension’s configuration; VS Code itself does not impose a universal model-selection feature.

Does a local model send my code to the internet?
Inference may run locally, but data handling depends on the extension and any connected services. Review its documentation and settings, and avoid including secrets in prompts.

Will a bigger context window always improve answers?
No. More context can help with related code, but irrelevant material increases processing and may distract the model. Provide the smallest useful set of files and instructions.

Related

• VR Training at Work: How It Works & Real Uses
• Docker on Raspberry Pi: What I Run and How
• Funny Siri Commands: 25+ That Actually Work