Asif Javed

Run Claude Code with a Local LLM Using Ollama

Posted on by Asif

Want to use Claude Code without sending data to the cloud or paying API fees? You can point it at a locally running LLM via Ollama in just a few minutes. Here’s exactly how.

What You’ll Need

  • Claude Code installed
  • Ollama installed with at least one model pulled
  • A terminal (PowerShell on Windows, bash/zsh on Mac/Linux)

Step 1 — Install Claude Code

Open PowerShell and run:

irm https://claude.ai/install.ps1 | iex

When it finishes you’ll see:

Next: Run claude --help to get started
=> Setup notes:
  • Native installation exists but C:\Users\<username>\.local\bin is not in your PATH.
=> Installation complete!

Step 2 — Fix the PATH (Windows)

The installer doesn’t automatically add itself to your PATH. Fix it manually:

  1. Open System Properties → Environment Variables
  2. Under User variables, select Path → Edit
  3. Click New and add: C:\Users\<YourUsername>\.local\bin
  4. Click OK and restart your terminal

On macOS/Linux, add this to your ~/.bashrc or ~/.zshrc:

export PATH="$HOME/.local/bin:$PATH"

Step 3 — Set Environment Variables

Claude Code respects a ANTHROPIC_BASE_URL override to redirect its API calls. Set these two variables before launching:

Mac/Linux:

export ANTHROPIC_BASE_URL="http://localhost:11434"
export ANTHROPIC_AUTH_TOKEN="token"

Windows PowerShell:

$env:ANTHROPIC_BASE_URL = "http://localhost:11434"
$env:ANTHROPIC_AUTH_TOKEN = "token"

Note: Ollama doesn’t require real authentication, so the token value can literally be "token" — Claude Code just needs the variable to be non-empty.

To make these permanent, add them to your shell profile or Windows User environment variables.

Step 4 — Check Your Available Models

$ ollama list

NAME                      ID              SIZE      MODIFIED      
starcoder2:3b             f67ae0f64584    1.7 GB    17 months ago
gemma:latest              a72c7f4d0a15    5.0 GB    17 months ago
codestral:latest          fcc0019dcee9    12 GB     17 months ago
llama3.1:latest           f66fc8dc39ea    4.7 GB    17 months ago
mistral:latest            f974a74358d6    4.1 GB    17 months ago

The NAME column is exactly what you’ll pass to Claude Code. For coding tasks, Codestral is the strongest option if your machine can handle 12 GB. Mistral is a great lightweight alternative.

Step 5 — Launch Claude Code with Your Local Model

Pass the model name using the --model flag:

# Syntax
claude --model <model-name-from-ollama-list>

# Examples
claude --model mistral:latest
claude --model codestral:latest
claude --model llama3.1:latest

That’s it — Claude Code will now route all requests through your local Ollama instance. No data leaves your machine.

Quick Troubleshooting

  • Connection refused? Make sure Ollama is running: ollama serve
  • Model not found? Copy the name exactly as shown in ollama list, including the tag (e.g. mistral:latest)
  • Slow responses? Drop down to a smaller model like starcoder2:3b or consider a GPU with 8+ GB VRAM