Local AI Agent for Coding: Run an Offline Stack with Ollama (2026)

Short answer: A local AI agent is a coding agent whose model runs on your machine (or a box you control), so prompts and repo context do not leave your network by default. In 2026 the practical stack is Ollama (or llama.cpp / LM Studio) as the model server, plus a harness: Continue for in-editor chat and autocomplete, Cline for Plan/Act agent loops in VS Code, Aider for git-native terminal edits, or OpenCode for a terminal coding agent. The hard parts are not the install commands. They are hardware fit, raising Ollama’s tiny default context window, and picking tasks a mid-size local model can finish without looping.

Last updated: November 6, 2026. Setup steps and context numbers come from the official Ollama, Continue, Cline, Aider and OpenCode docs, all read today. DeepSeek Flash prices come from api-docs.deepseek.com. Forum notes are paraphrased from recent threads on r/LocalLLaMA, r/LocalLLM, r/ollama, r/ChatGPTCoding and the Oct 31, 2025 Ask HN on local coding assistants. I have not run a full local agent on a shipped product; where I show commands from this Linux test machine, they are labeled as preflight only.

What “local AI agent” means (and what it doesn’t)

People type local ai agent when they want one of three things:

1. Privacy — client code, API keys and customer data stay on disk you control.
2. Offline / low-latency work — planes, flaky hotel Wi‑Fi, or a day when every cloud provider is rate-limiting you.
3. Cost control — no per-token bill for autocomplete, commit messages and small refactors (you already paid for the GPU or RAM).

It is not “Claude Code quality with zero cloud.” Frontier hosted models still win on hard multi-file agent loops. Even heavy local setups on HN (128GB Mac Studios, RTX 3090 desktops, Framework boxes) usually keep Sonnet / GPT / Codex for the nasty bugs and use local models for completion, docs, and well-scoped edits. Treat local as a lane in your stack, not a religion.

This post is different from my free Claude Code alternatives write-up (OpenCode / Cline / Aider with BYO keys) and from the monthly AI indie stack cost post. Here the goal is a mostly or fully local coding agent: model + harness + context config on hardware you own.

Hardware: pick a lane before you pull a 30B

Cline’s local-models docs give a useful RAM map:

RAM	Realistic local setup
16–32 GB	Small / quantized models (autocomplete, short chat)
32–64 GB	Mid-size coding models with a usable context window
64 GB+	Larger models and bigger agent contexts

Continue’s Ollama guide is similar: plan ~8 GB RAM for 7B-class models, ~16 GB for 13B, ~32 GB for 32B. Disk: leave at least 10 GB free before you start pulling.

From recent forums (paraphrased):

• On r/LocalLLM, people running huge Mac Studios still report Cline looping or chewing prompt-processing time on local coding models; Continue felt fine for assistance but weak as a full agent.
• On r/LocalLLaMA, users trying llama.cpp backends found Aider and Cline picky about file-edit formatting; Continue was the least annoying starting point for some.
• On r/ollama, VS Code + Copilot Chat bloated prompts against Ollama; Cline also bloats; Continue answered fastest but with fewer agent features.
• The big Ask HN (Oct 31, 2025) thread keeps repeating: Ollama is easiest; llama.cpp is faster once you care; Qwen coder ~30B and gpt-oss variants are common; many people use local for FIM / small tasks and escalate hard agent work to the cloud.

My take: if you only have a 16 GB laptop, aim for a small autocomplete model + chat for one-file questions. Do not expect Cline-style multi-file YOLO on that box. If you can spare a desktop with 32–64 GB (or a Mac with lots of unified memory), a 30B-class coder MoE becomes interesting.

The four harnesses that matter for local coding

Harness	Best for	Local backend	Agent strength
Continue	Autocomplete + sidebar chat in VS Code / JetBrains	Ollama provider in config.yaml	Light; great FIM with a tiny model
Cline	Plan / Act loops with approval gates in VS Code	Ollama / LM Studio / Atomic Chat	Strong agent loop; needs compact prompt + big context
Aider	Terminal pair-programmer, every edit as a git commit	ollama_chat/<model>	Strong for repo edits; uses search-replace formats
OpenCode	Terminal coding agent (read / edit / run commands)	ollama launch opencode or opencode.json	Needs 64K+ context per Ollama’s docs

If you already compared these tools with cloud keys, start from free Claude Code alternatives. Below is the local wiring only.

Step 0: install Ollama and prove the server is up

# macOS
brew install ollama

# Linux (official install script)
curl -fsSL https://ollama.com/install.sh | sh

# Windows: download the app from ollama.com

Start the server and sanity-check:

ollama serve
# other terminal
curl http://localhost:11434
# expect: Ollama is running

ollama pull qwen2.5-coder:7b          # chat / edit candidate
ollama pull qwen2.5-coder:1.5b        # autocomplete candidate
ollama list

Preflight on my Linux test machine today (Ollama is not installed there yet — labeled as preflight only):

=== preflight 2026-11-06T09:10Z ===
uname: Linux … x86_64
Mem: 15Gi total / ~6.4Gi available
python: Python 3.13.5
node: v20.19.2
ollama: not-installed
curl localhost:11434: (not running)

A 15 GB machine is a poor host for a serious local agent. Check free -h / Activity Monitor before you pull a 20 GB model.

The #1 local-agent pitfall: Ollama’s tiny default context

Ollama’s default context is often 2K–4K tokens. Agent harnesses send a system prompt, tool schemas, file contents and history. When you overflow, Ollama does not always explode with a clear error — it silently truncates. The agent then “forgets” the plan, rewrites the same function, or invents APIs that do not exist. LogRocket’s Cline + Ollama walkthrough and Aider’s Ollama page both call this out.

Fix A — raise context when starting the server (Aider’s recommendation)

OLLAMA_CONTEXT_LENGTH=8192 ollama serve
# for heavier agents, try 32768 or 65536 if VRAM/RAM allow

Fix B — bake num_ctx into a Modelfile (Cline / OpenCode style)

cat > Modelfile <<'EOF'
FROM qwen2.5-coder:7b
PARAMETER num_ctx 32768
EOF

ollama create qwen2.5-coder-7b-agent -f Modelfile

Use the custom name (qwen2.5-coder-7b-agent) in Cline / OpenCode / Aider, not the base tag.

Fix C — Aider’s per-request approach

Aider sets num_ctx large enough for each request plus ~8K for the reply by default. To pin a fixed window, put this in .aider.model.settings.yml:

- name: ollama_chat/qwen2.5-coder:7b
  extra_params:
    num_ctx: 65536

Rule of thumb from official docs:

• Cline (Ollama integration): set Context Window to at least 32K.
• OpenCode (Ollama integration): needs 64K or higher.
• Continue: set defaultCompletionOptions.contextLength to what your RAM can hold; drop it if you see “model requires more system memory.”

Continue: local autocomplete + chat (fastest win)

Continue’s Ollama guide: install the extension, keep Ollama running, pull exact model tags, then configure ~/.continue/config.yaml (or the configs folder). Example:

name: Local coding
version: 0.0.1
schema: v1
models:
  - name: Qwen2.5-Coder 7B (chat / edit)
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles: [chat, edit, apply]
    capabilities: [tool_use]
  - name: Qwen2.5-Coder 1.5B (autocomplete)
    provider: ollama
    model: qwen2.5-coder:1.5b
    roles: [autocomplete]

Notes from the docs:

• Pull the exact tag the config expects (:7b vs :latest mismatch = 404).
• Agent mode needs real tool support; if Continue says agent is not supported, try capabilities: [tool_use] or switch models.
• Autocomplete wants a small model so every keystroke pause stays under ~1 second.

This is the setup HN folks keep recommending for FIM on a laptop: tiny coder for completion, bigger model only when you open chat.

Cline: local Plan / Act agent

From Cline’s local-models overview and Ollama’s Cline page:

1. Install Cline in VS Code.
2. Start Ollama (or LM Studio on http://localhost:1234).
3. Cline Settings → API Provider = Ollama, Base URL http://localhost:11434, pick your model (prefer the Modelfile variant with 32K+).
4. Set Context Window to at least 32768 (match the Modelfile).
5. Enable Use Compact Prompt under Features — Cline’s own tip for local models, because the default system prompt is huge.

Pitfalls users report (r/LocalLLM, r/ollama): loops, long prompt-processing stalls, and prompt bloat. Keep tasks small. Start a new task when context gets fat. Do not expect cloud-Sonnet behavior from a 7B.

Aider: terminal agent that commits every change

python -m pip install aider-install
aider-install

export OLLAMA_API_BASE=http://127.0.0.1:11434   # Windows: setx, then new shell
OLLAMA_CONTEXT_LENGTH=8192 ollama serve          # separate terminal

cd /path/to/your/repo
aider --model ollama_chat/qwen2.5-coder:7b

Use the ollama_chat/ prefix (Aider docs prefer it over ollama/). Aider edits via search-and-replace style diffs and commits, so undo is git reset — nice for review. On r/LocalLLaMA some people said local models sometimes return file formats Aider cannot parse; if that happens, try a stronger coder model or fall back to Continue for chat-only.

OpenCode: ollama launch opencode

Ollama’s OpenCode integration is the shortest path:

# install OpenCode (one of these)
curl -fsSL https://opencode.ai/install | bash
# or: npm install -g opencode-ai

ollama pull qwen2.5-coder:7b
# bake 64K if you can afford it
cat > Modelfile <<'EOF'
FROM qwen2.5-coder:7b
PARAMETER num_ctx 65536
EOF
ollama create qwen2.5-coder-7b-64k -f Modelfile

ollama launch opencode --model qwen2.5-coder-7b-64k

Manual opencode.json provider (from Ollama docs):

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen2.5-coder-7b-64k": {
          "name": "Qwen2.5-Coder 7B 64k"
        }
      }
    }
  }
}

OpenCode needs a large context. If you skip the Modelfile / OLLAMA_CONTEXT_LENGTH step, agent turns will truncate and look “dumb.”

Local vs cheap cloud: where DeepSeek Flash fits

Local is not always cheapest in your time. A weak local agent that loops for 40 minutes costs more than a dollar of API tokens.

Verified DeepSeek API prices (api-docs.deepseek.com, model deepseek-flash / DeepSeek-V4.1-Flash-1017, USD per 1M tokens):

	Off-peak	Peak
Input (cache hit)	$0.003	$0.006
Input (cache miss)	$0.15	$0.30
Output	$0.60	$1.20

Off-peak is half of peak. Peak windows are 01:00–04:00 and 06:00–10:00 UTC, Monday–Friday (excluding Chinese public holidays).

DeepSeek Flash is the cheapest hosted overflow lane I use for bulk work — not the smartest. Pair it like this:

• Local: autocomplete, commit messages, “explain this file,” offline days, anything with secrets.
• DeepSeek Flash API: long refactors when the local model loops, or when you are away from the GPU box.
• ChatGPT Plus / work Codex / Google AI Pro / Grok: the jobs where quality still matters more than privacy (I pay for Plus personally; Codex Pro 20x is a work account).

I used Claude Code heavily in the past; I do not use it now. If you still do and you are burning usage credits, see Claude extra usage cost and how to reduce token usage. For Cursor limit walls, see “You’ve hit your usage limit”.

Privacy: “local” has footnotes

A local model server on localhost keeps inference on-device. You still leak data if:

• The harness phones home for telemetry, model lists or account checks (read each tool’s docs; turn optional cloud features off).
• You enable web fetch / browser tools that send URLs or page text out.
• You expose Ollama with OLLAMA_HOST=0.0.0.0 on a public network.
• You paste secrets into prompts that later sync to a cloud backup of the chat log.

For product security after the agent writes code, keep using the vibe-coded app security checklist. Local inference does not fix missing RLS or a leaked Stripe key.

A practical workflow for indie founders

1. Morning / plane / secrets work: Continue + small local coder for autocomplete; Aider or OpenCode on a 7B–30B for scoped edits.
2. Hard bug / multi-package refactor: escalate to your best cloud agent (Codex at work, ChatGPT Plus, or a cheap DeepSeek Flash session). Hand off with a short brief:

Handoff from local agent:
- Goal:
- Files already touched:
- Commands already run + results:
- What failed / looped:
- Constraints (no new deps, keep tests green):
Do not redo steps 1–N. Continue from the failure.

3. When cloud usage is exhausted: flip back to local instead of doom-scrolling billing pages (Cursor alternatives and usage-limit fixes help here).
4. Ship: local or cloud, the deploy path is the same — turn code into an app, host pick in Vercel vs Netlify.

Copy-paste: local agent system prompt

Put this in Continue rules, Cline custom instructions, Aider’s convention file, or OpenCode instructions:

You are a local coding agent. Rules:
1. Prefer small, reviewable diffs. One concern per change.
2. Before editing, list the files you will touch (max 5). Wait if unsure.
3. Never invent APIs. If a symbol is missing, search the repo first.
4. Run the project's test or typecheck command after edits when one exists.
5. Do not print or commit .env, tokens, or customer data.
6. If you lack context, ask one clarifying question instead of guessing.
7. If you loop on the same error twice, stop and summarize what you tried.

FAQ

What is a local AI agent?

A coding (or general) agent whose model inference runs on hardware you control — typically via Ollama, LM Studio or llama.cpp — while a harness like Continue, Cline, Aider or OpenCode handles tools, edits and approvals.

Can I run a local AI agent fully offline?

Yes for inference, if the model is already downloaded and the harness does not require a login. First-time installs, extension marketplaces and optional cloud features still need the network. Pull models while online, then switch airplane mode to verify.

How much RAM do I need for a local LLM coding agent?

Plan from Cline/Continue guidance: 16–32 GB for small quantized models, 32–64 GB for mid-size coders, 64 GB+ for larger models and big agent contexts. Autocomplete can work with much less if you use a 1.5B–3B FIM model.

Is Ollama enough, or do I need llama.cpp?

Ollama is the easiest path and what most agent docs integrate first. People who care about raw tokens/sec often move to llama.cpp or vLLM. Start with Ollama; switch only when speed or context control becomes the bottleneck.

Should I replace Cursor / Claude Code with a local stack?

Usually no. Use local for privacy, offline work and cheap volume. Keep a strong hosted agent for hard problems. Hybrid beats purity for shipping.

Building the rest of the stack? Start from the AI workflow for indie products hub, then plug in what is vibe coding and the security checklist before you launch.