A practical guide to running AI models on your own machine with Ollama: installing it on Windows, downloading and trying a model, setting the context that agents need and connecting it to OpenCode or Claude Code so you can code without sending your code anywhere. All the commands are also in the README.md on the left.
What you’ll learn
- What Ollama is and why you would run AI models on your own machine.
- How to install Ollama on Windows and check that it works.
- How to download a model, try it in the terminal and manage the models you have downloaded.
- How to set the context length that coding agents need.
- How to connect an agent (OpenCode or Claude Code) to your local model.
Prerequisites
- Windows 10 22H2 or later (Home or Pro); no administrator rights needed. macOS 14 or later and Linux also work.
- A graphics card helps a lot: NVIDIA with driver 551.61 or later, or a compatible AMD card. Without one it runs on the processor, more slowly.
- Disk space: around 4 GB for Ollama, plus whatever each model takes (from a few GB to hundreds).
- For the last step, an installed agent: OpenCode or Claude Code (see the basic setup guide for an AI coding agent).
Step by step
1. Understand what Ollama does
Ollama downloads open AI models and runs them on your machine, with a local API at http://localhost:11434. Ollama itself is not an agent: it is the “engine” that a coding agent such as OpenCode or Claude Code can use instead of a model in the cloud.
The main benefit is privacy: with local models, your prompts and your code never leave your machine. In return, the models that fit on a laptop are usually less capable than the large cloud models.
2. Install Ollama
On Windows, download the installer from ollama.com/download (it installs into your user folder and keeps itself up to date) or install it from PowerShell. Then check the version and that the service responds “Ollama is running”:
# Option 1: the Windows installer (OllamaSetup.exe, at https://ollama.com/download)
# Option 2: from PowerShell
irm https://ollama.com/install.ps1 | iex
# Check the version and that the service responds
ollama -v
curl.exe http://localhost:11434
On Linux and macOS:
# Linux (and macOS)
curl -fsSL https://ollama.com/install.sh | sh
ollama -v
In PowerShell type curl.exe, not curl: curl is an alias for a different command.
3. Download a model and try it
Models are downloaded by name from the library at ollama.com/library. ollama run opens a chat in the terminal:
# Download a model and chat with it in the terminal (leave with /bye)
ollama pull qwen3.5
ollama run qwen3.5
# Manage your models
ollama ls # the models you have downloaded
ollama ps # the ones loaded right now (and whether they use GPU or CPU)
ollama stop qwen3.5 # unload the model from memory
ollama rm qwen3.5 # delete the model from disk
Some coding-oriented models in the library, with their download size:
| Model | Tag | Approximate download |
|---|---|---|
| Qwen3.5 | qwen3.5 (9b) | 7 GB |
| Qwen2.5-Coder | qwen2.5-coder:7b · :14b · :32b | 4.7 GB · 9 GB · 20 GB |
| Devstral | devstral (24b) | 14 GB |
| gpt-oss | gpt-oss:20b | 14 GB |
| Qwen3-Coder | qwen3-coder (30b) | 19 GB |
Start with a small one and use ollama ps to check whether it runs on the GPU (PROCESSOR column): if it shows CPU, the model does not fit in your graphics card’s memory and will be slower.
4. Call it from the API
Any application on your machine can use the model through the local API:
# Ollama's native API (PowerShell adaptation of the curl example in the documentation)
Invoke-RestMethod -Uri http://localhost:11434/api/generate -Method Post -ContentType 'application/json' `
-Body '{"model":"qwen3.5","prompt":"Write a Python function that adds two numbers","stream":false}'
# OpenAI-compatible API, for tools that already use it:
# http://localhost:11434/v1/ (they ask for a key, but Ollama ignores it)
5. Set the context for agents
Coding agents send a lot of text with every request (files, instructions, command output). OpenCode and Claude Code need a context of 64,000 tokens or more, and by default Ollama uses less if your card has little memory. On Windows, you change it with an environment variable:
1. Quit Ollama from its icon in the taskbar.
2. Search for "environment variables" in Settings and open "Edit environment variables for your account".
3. Create the variable OLLAMA_CONTEXT_LENGTH with the value 64000.
4. Click OK and open Ollama again from the Start menu.
A larger context takes more memory: if the model starts running on the CPU, try a smaller one.
6. Connect a coding agent
The simplest way is ollama launch, which opens the agent already set up to use Ollama:
# Open an agent already connected to Ollama (if you do not give a model, it lets you choose)
ollama launch opencode --model qwen3.5
ollama launch claude --model qwen3.5
# Only configure OpenCode to use Ollama, without opening it
ollama launch opencode --config
OpenCode also detects Ollama on its own when it is running: in OpenCode, run /connect, choose Ollama and then the model with /models. You can also set up Claude Code by hand, pointing it at the Ollama API:
# Claude Code against Ollama, by hand (only for this PowerShell window)
$env:ANTHROPIC_AUTH_TOKEN = "ollama"
$env:ANTHROPIC_API_KEY = ""
$env:ANTHROPIC_BASE_URL = "http://localhost:11434"
claude --model qwen3.5
# Inside Claude Code, /status shows what it is connecting to
7. Keep Ollama on your own machine
By default Ollama only listens on your own machine (127.0.0.1:11434). If you change OLLAMA_HOST to 0.0.0.0, anyone who can reach your machine over the network will be able to use your models: the local API does not check keys. Only do it on a trusted network and behind a firewall.
Common errors
curl.exe http://localhost:11434 does not respond
Ollama is not running. Open it from the Start menu (its icon should appear in the taskbar) and try again. Check the logs in %LOCALAPPDATA%\Ollama (server.log and app.log).
The model is very slow
Look at ollama ps: if the PROCESSOR column shows CPU, or a split between CPU and GPU, the model does not fit in your graphics card’s memory. Use a smaller model or reduce the context. Also check that your NVIDIA driver is 551.61 or later.
The agent does not use its tools properly or “forgets” earlier work
The context is too small. Raise OLLAMA_CONTEXT_LENGTH to 64000 or more (step 5) and restart Ollama. A model designed for coding also helps.
I changed an environment variable and nothing happened
Ollama reads its variables when it starts. Quit it from the taskbar icon (closing the window is not enough) and open it again from the Start menu.
I am running out of disk space
Check what you have downloaded with ollama ls and delete what you do not use with ollama rm. Models are stored in C:\Users\<user>\.ollama\models; to store them on another disk, set OLLAMA_MODELS as in step 5.
It does not download models behind the company proxy
Set the HTTPS_PROXY variable to the proxy address (do not use HTTP_PROXY) and restart Ollama.
Next steps
- Set up the agent, its context and its permissions (see the basic setup guide for an AI coding agent).
- Compare several models from the library on the same task from your project before choosing one.
- Integrate the local API into your applications through the OpenAI compatibility (
http://localhost:11434/v1/). - Combine local models for tasks involving sensitive data with cloud models for the most complex ones.