My Local AI Coding Workflow (Hardware, Setup, Implementation)

Summary

Philipp Lackner walks through a local AI coding setup, from choosing hardware and models to connecting a model to an agent and testing it on a small app. The main trade-off is privacy, offline access, and avoiding per-token costs versus the higher reliability and capability of cloud frontier models.

Tools and setup

• LM Studio is used to download and run open-source models, and to expose a selected model through a local server.
• OpenCode, an open-source agent-coding tool, connects to that server. Lackner prefers it for local models, though he notes that other coding agents can also be configured to use local models.
• Once a model is downloaded, prompts can be processed locally without sending data to an external provider. LM Studio’s developer settings control the server, selected model, and optional API-key authentication.
• To connect OpenCode, configure its JSON settings with the LM Studio endpoint, the model’s exact identifier, and matching context-window information. The model’s actual context limit must be set in LM Studio; the value in OpenCode tells the agent what limit to expect.

Local-model concepts

• Parameters are the model’s learned weights. Larger models often handle nuance, vague instructions, and complex tasks better, but they use more memory and can take longer to begin responding. Quality gains diminish as models grow.
• Quantization stores weights at lower precision to reduce memory use. Four-bit models can be much smaller than higher-precision versions, with some loss of quality—but, in the presenter’s view, often a useful compromise.
• GGUF is presented as a common model format; MLX is suited to Apple silicon.
• Some models use a mixture-of-experts (MoE) design, where only some experts are active for a given task, reducing the number of active parameters.

Hardware and model recommendations

Lackner identifies GPU memory (VRAM) and memory bandwidth as key factors: model weights need to be accessible while generating tokens, and spilling into slower system RAM can sharply reduce speed.

His approximate recommendations are:

• Windows: an Nvidia RTX 4090 or 5090 as a comfortable starting point.
• Apple silicon: shared unified memory makes large models convenient to run, though bandwidth may be lower than that of high-end Nvidia GPUs.
• Minimum for comfortable coding: around 24 GB of VRAM, or 32 GB of Apple unified memory.
• More comfortable: over 48 GB of VRAM, or 64 GB of unified memory.

These are the presenter’s guidelines, not universal requirements. He advises against buying expensive hardware solely to run local AI, since cloud subscriptions may be better value.

For models, he suggests:

• Qwen 2.5 Coder, 32B parameters, at roughly 18 GB in a four-bit version, as an option for mid-range hardware.
• Qwen 3 Coder Next, described as around 80B parameters and about 45 GB at four-bit quantization, for higher-end hardware. He says it has given him the best results among the local models he tested.

Benefits and limitations

Local models can be useful when privacy matters—for example, when handling sensitive business or legal material—or when working offline. After download, their use does not incur token charges, though the computer consumes electricity.

The cost is capability and consistency. Lackner says local models are less dependable than leading cloud models on complex agent-coding tasks, with more errors and hallucinations. He recommends local AI for privacy-conscious use, experimentation, and less complex work—particularly when suitable hardware is already available.

Demonstration and practical result

Lackner loads a Qwen model in LM Studio, starts the local server, configures OpenCode, and asks it to build a simple browser-based note-taking app. The model takes about three minutes and initially produces a file that is not readily runnable; he has to prompt it again to use Vite. The resulting app runs, but some features have errors and editing does not work as expected.

The demo shows that a local model can generate and run code without online access, but also illustrates why it may require more iteration and supervision than a frontier cloud model.

Guide and tutorial points

1. Install LM Studio and OpenCode.
2. Download a model appropriate to the available memory.
3. Set the model’s context window in LM Studio and load it.
4. Start the local server, optionally create an API token, and note the model identifier.
5. Configure OpenCode’s JSON settings to match the endpoint, model ID, and expected context length.
6. Test with a small coding task, then check the output and iterate.

Main speaker/source: Philipp Lackner, presenting his own local AI coding workflow, model comparisons, hardware recommendations, and live setup demonstration.