# How to Run an AI Coding Agent Locally with Ollama — No Cloud API Key Needed
I’ve been testing AI coding agents for months. Most of the time, I ran them through cloud APIs — type a prompt, get a answer, repeat. It works. But it always felt like renting your brain out to someone else’s server.
Then I discovered the quiet shift happening right now: you can run a full AI coding agent on your own machine, connected to a local model, with zero cloud API keys. No Anthropic account. No OpenAI key. No billing dashboard. Just your laptop, a model, and a single command.
This is the same wave the TechCrunch piece “Here are the top AI agents that can live in your text messages” is pointing at — agents moving from chat windows into your workflows. I’ll show you how to get one running on your machine, step by step. For a primer on local AI models, see my guide How to Run Open-Source AI Models Locally with Ollama.
—

## Why go local?
There are three good reasons.
**Privacy first.** Your code never leaves your machine. No shipping source files to a cloud API. No worrying about what the server logs. For anything sensitive — payment code, auth flows, company secrets — local is the only sane option.
**Cost predictable.** Cloud agents charge per token, per session, per seat. Local runs on your existing hardware. After the model download, there’s no per-use cost.
**Speed.** You get instant feedback. No waiting for a distant server to think. The model runs at the speed of your machine.
The trade-off is hardware. Local models need RAM and, ideally, a GPU. The NVIDIA CLI coding agent playbook (build.nvidia.com/playbooks/cli-coding-agent/overview) documents the full setup for wiring OpenCode, Claude Code, or Codex CLI to a local Ollama model. But even an entry-level laptop can run small models for everyday coding tasks.
—
## What you need
You need three things:
1. **Ollama** — the local model runner
2. **A coding model** — I use Qwen3.6 as the default, but there are many
3. **A CLI coding agent** — OpenCode, Claude Code, or Codex CLI, each launched through Ollama
The beauty of the new model is that Ollama has a built-in launch method that wires the agent directly to the local model. No environment variables, no provider config files, no external cloud API keys.
—
## Step 1: Install Ollama
Head to [ollama.com/download](https://ollama.com/download) and grab the version for your OS.
On Linux, it’s a single command:
“`bash
curl -fsSL https://ollama.com/install.sh | sh
“`
Then start the server:
“`bash
ollama serve
“`
Verify it’s running:
“`bash
ollama –version
“`
You should see the Ollama version number printed out. That’s it. Ollama is now your local model server.
—
## Step 2: Pull a coding model
Not all models are good at coding. You want one trained or fine-tuned on code. The NVIDIA playbook recommends **Qwen3.6** as a reliable default, but there are many options.
Pull a model (this downloads it — it’s a few gigabytes):
“`bash
ollama pull qwen3.6
“`
Check what you have:
“`bash
ollama list
“`
If the install is working, you’ll see the model listed. That’s your local AI brain.
—
## Step 3: Launch an AI agent through Ollama
This is the part that makes the whole thing click. You have three choices, all launched the same way:
“`bash
ollama launch claude
ollama launch codex
ollama launch opencode
“`
Pick one. I use OpenCode because it’s open-source, self-hosted, and has no vendor lock-in. But you can try any of the three.
“`bash
ollama launch opencode
“`
You’re now in the agent’s prompt. Type a question. The agent routes your request to the local model you pulled. No API key. No cloud. Just your machine.
—
## Step 4: Try a real coding task
Here’s what I had the agent do on my machine. I asked it to review a small Python script I was writing for a data-processing pipeline:
> “Review this script for bugs, then rewrite the data-loading function to use a generator instead of loading everything into memory.”
The agent read the file, found the inefficiency, and produced the generator version with a short explanation. All local. No round-trip to a cloud API.
Try this yourself with a small project — a script you’re writing, a config file, a Dockerfile. Let the agent read it, suggest improvements, and show you the diff.
—
## What you can actually do with it
Once it’s running, the practical workflows are:
– **Code review.** Paste in a function or file, ask for bugs, security issues, and style improvements.
– **Refactoring.** “Rewrite this to use Python’s dataclasses” or “Convert this to async.”
– **Testing.** “Write unit tests for this function, covering edge cases.”
– **Documentation.** “Generate a README for this project.”
– **Learning.** Ask the agent to explain a concept from your own code.
The local model is not as capable as the biggest cloud models. But it’s more than enough for review, refactoring, and exploration. And you can tune your expectations — local models are for working with code you already understand, not for writing greenfield systems from a paragraph of description.
—
## Security: the part nobody mentions
Local models are safer than cloud APIs, but not automatically safe. Here’s what I check before trusting an agent with real code:
1. **The model.** Is this model well-known and reviewed? Stick with the big names for now.
2. **The agent’s permissions.** Some agents can run shell commands. Set a sandbox first.
3. **The code you paste in.** Never paste secrets, keys, or credentials into any AI tool, local or cloud.
4. **Diffs before merges.** Always review what the agent changes before committing.
I wrote in detail about vetting AI coding agents in my post “How to Vet an AI Coding Agent Before Trusting It With Your Code” — the six-step checklist I actually use before letting an agent touch my codebase.
—
## The bottom line
Running a local AI coding agent takes five minutes to set up and it changes how you work with code. You get instant, private, free-as-in-beer assistance — right on your machine.
The same trend that has AI agents living in your text messages also puts them on your laptop. The difference is choice: you decide where your code goes.
Start small. Pull a model. Launch an agent. Ask it to review one file. Then expand from there.
—
**References:** The NVIDIA CLI coding agent playbook (build.nvidia.com/playbooks/cli-coding-agent/overview) and the Ollama quickstart (docs.ollama.com/quickstart) were the authoritative sources for the `ollama launch` workflow and model recommendations. I verified each command against a clean Linux machine.