Ollama makes local AI deployment as simple as a single command — full data privacy, zero-cost inference, and offline use, with no cloud accounts or per-token fees. This guide covers install, the popular models, hardware needs, the API, and how local stacks up against cloud.

Installation

# macOS (or download from ollama.com/download, needs macOS 11+)
brew install ollama

# Windows (or the installer at ollama.com/download, needs Windows 10+)
winget install Ollama.Ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Docker
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Then run a model — Ollama auto-downloads it (~4.7 GB for Llama 3.1 8B) and drops you into an interactive chat:

ollama run llama3.1

Popular Models

Essential Commands

ollama pull llama3.1                              # download a model
ollama run llama3.1                               # interactive chat
ollama run llama3.1 "Explain Docker in a line"    # one-shot prompt
ollama list                                       # list downloaded models
ollama ps                                         # running models + memory
ollama rm llama3.1                                # delete a model

Hardware Requirements

Model size determines the RAM/VRAM you need:

• No GPU? Ollama runs on CPU, just 5-10x slower.
• Apple Silicon uses unified memory — a 16 GB M1/M2 runs 7B-8B models well.
• NVIDIA GPUs with 6+ GB VRAM give significant speedups; AMD/Intel GPUs work via the Vulkan backend.

Using the API (Python)

Ollama serves a fully OpenAI-compatible API at http://localhost:11434/v1 — any OpenAI SDK code works by just changing the base URL.

pip install openai

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama",  # required by SDK but not used
)

response = client.chat.completions.create(
    model="llama3.1",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is machine learning?"}
    ]
)

print(response.choices[0].message.content)

Streaming and the native ollama library are both supported:

# Streaming (OpenAI SDK)
stream = client.chat.completions.create(
    model="llama3.1",
    messages=[{"role": "user", "content": "Write a short poem about coding"}],
    stream=True
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

# Native Ollama library (pip install ollama)
import ollama
response = ollama.chat(
    model="llama3.1",
    messages=[{"role": "user", "content": "Explain Docker in one paragraph"}]
)
print(response["message"]["content"])

Connect Ollama to OpenClaw (Free Local AI Agent)

Pair Ollama with OpenClaw for a fully local, zero-cost AI agent — shell commands, file operations, browser control, scheduled tasks, and chat on WhatsApp or Telegram, all on your own hardware. OpenClaw auto-discovers your local models:

# 1. Make sure Ollama is running with a model
ollama pull llama3.1

# 2. Install and configure OpenClaw
npm install -g openclaw@latest
openclaw onboard
# Select "Ollama" from the provider list

Performance Tips

Quantization compresses models to fit less memory with minimal quality loss:

• Reduce KV cache memory: OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve
• Keep models loaded: OLLAMA_KEEP_ALIVE=-1 prevents unloading between requests
• Concurrent requests: OLLAMA_NUM_PARALLEL=4 for multiple sessions
• A 14B Q4 model on GPU beats a 70B model on CPU — smaller can be faster.

Local Ollama vs Cloud APIs

Pro tip: Most developers in 2026 run a hybrid — Ollama for high-volume and sensitive tasks, cloud APIs for complex reasoning — which typically cuts cloud costs by 60-80%.

Which Model Should You Start With?

• 8 GB RAM, no GPU: Gemma 3 4B or Phi-3 Mini 3B
• 16 GB RAM or 8 GB VRAM: Llama 3.1 8B or Qwen 3 8B
• 32 GB RAM or 16 GB VRAM: DeepSeek-R1 14B or Gemma 3 27B
• 64+ GB RAM or 24+ GB VRAM: Llama 3.1 70B or Qwen 3 32B

Frequently Asked Questions

Is Ollama really free?

Yes — it's open-source and runs on your own hardware, so there's no account, no API key, and no per-token fee. Your only cost is the machine it runs on.

Does Ollama work without a GPU?

Yes. If no GPU is detected it runs on CPU, roughly 5-10x slower. Apple Silicon Macs use unified memory and run 7B-8B models well on 16 GB.

Can I use my existing OpenAI code with Ollama?

Yes. Ollama exposes a fully OpenAI-compatible API at http://localhost:11434/v1 — point the SDK's base URL there and pass any placeholder api_key.

Get Started

ollama.com | Model Library | GitHub