I spent last weekend building a private ChatGPT clone on my home server. Not because I don’t trust the cloud providers — I do, mostly — but because there’s something satisfying about owning the entire stack. The chat history, the model weights, the API layer: all of it sitting on my hardware, talking only to machines I control.
That setup is Open WebUI plus Ollama, and if you’ve ever wanted your own ChatGPT-style interface without sending every prompt to OpenAI, this guide will get you there in under twenty minutes. I’ve tested these exact commands on a fresh Ubuntu 22.04 box with 16 GB of RAM and an NVIDIA RTX 3060. Your hardware will differ, but the steps are the same. All the commands below come from the official Open WebUI quickstart, the Open WebUI GitHub repo, and the Ollama docs.
What Open WebUI and Ollama Actually Do
Ollama is the model runner. It downloads open-weight LLMs, loads them into memory, and serves them over a REST API on port 11434. Think of it as your local equivalent of OpenAI’s inference endpoint. Open WebUI is the frontend — the chat interface, the admin panel, the user management layer. Together they give you a browser-based ChatGPT experience that runs entirely on your hardware.
The reason I pair these two instead of rolling something custom is simple: Ollama handles the messy parts (model quantization, GPU offloading, context window management) and Open WebUI gives you a polished UI out of the box. You don’t need to write a single line of frontend code. If you want to go beyond chat and build actual AI workflows around this stack, I covered the broader daily workflow in How to Run Local AI for Daily Tasks.
Prerequisites
Before you start, make sure you have Docker installed. That’s it. If you’re on WSL2, Ubuntu, macOS, or even a Raspberry Pi, Docker is the only hard requirement. Open WebUI also works with Podman, but Docker is the most tested path.
Hardware-wise, the minimum depends on the model. A 7B parameter model at Q4 quantization needs roughly 8 GB of RAM or VRAM. If you want to run larger models — 13B, 34B, or the newer 70B variants — plan on 16 GB minimum and a decent GPU if you care about speed. CPU inference works, but expect single-digit tokens per second on anything above 7B.
Step 1: Install Ollama
Ollama is a single binary. On Linux and macOS, the install script handles everything:
curl -fsSL https://ollama.com/install.sh | shOn Windows, grab the MSI from ollama.com/download or use winget:
winget install Ollama.OllamaOnce it’s installed, verify the daemon is running:
ollama --version
ollama serveThe ollama serve command starts the background service. On most systems it runs automatically after install, but it’s worth confirming. If you see a version number and no errors, you’re good.
Step 2: Pull Your First Model
Ollama maintains a library of open-weight models. For general-purpose chat on modest hardware, I recommend starting with Qwen3-Coder:8B if you want a coding-focused assistant, or Qwen3:8B for balanced everyday use. For very lightweight setups, Phi-4:14B punches above its weight class.
ollama pull qwen3:8bThis downloads the model to disk — about 4.7 GB for the 8B Q4_K_M quant. It only downloads once. After that, Ollama loads it on demand.
Test it interactively:
ollama run qwen3:8b "Explain dependency injection in three sentences"If you get a coherent answer back, the model runner is working. Press Ctrl+D to exit. And if you need the model to return machine-readable JSON instead of prose — for automations, API calls, or tool use — I covered the format:json pattern in How to Get Reliable JSON From Local LLMs With Ollama.
Step 3: Run Open WebUI with Docker
Now for the chat interface. Open WebUI’s official Docker command is straightforward, but there’s one flag that trips people up: the host.docker.internal host mapping. Without it, the container can’t reach Ollama running on your host machine.
docker run -d \
-p 3000:8080 \
--add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data \
-e WEBUI_SECRET_KEY=$(openssl rand -hex 32) \
--name open-webui \
--restart always \
ghcr.io/open-webui/open-webui:mainLet me break down the important flags. The -p 3000:8080 mapping exposes the UI on port 3000 of your host. The volume open-webui:/app/backend/data persists your chats, users, and settings across container recreations — never run without it. The WEBUI_SECRET_KEY encrypts session data; generate it once with openssl rand -hex 32 and keep the value stable, otherwise every container recreate logs everyone out.
If you’re on a machine with an NVIDIA GPU and want faster inference, use the :cuda tag instead and add --gpus all:
docker run -d \
-p 3000:8080 \
--gpus all \
--add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data \
-e WEBUI_SECRET_KEY=$(openssl rand -hex 32) \
--name open-webui \
--restart always \
ghcr.io/open-webui/open-webui:cudaConfirm the container is healthy:
curl -fsS http://localhost:3000/healthAn HTTP 200 means Open WebUI is up. Open http://localhost:3000 in your browser.
Step 4: Connect Ollama to Open WebUI
The first time you load Open WebUI, it walks you through admin account creation. After that, open Settings → Admin → Connections. You should see Ollama listed as a local provider. If it doesn’t auto-discover, add it manually:
- Name: Ollama
- Type: Ollama
- Base URL:
http://host.docker.internal:11434
Save the connection. Return to the chat interface, open the model dropdown, and you should see qwen3:8b (or whatever you pulled) available for selection. Send a test message. If you get a streaming response back, the full stack is live.
Step 5: Secure the Instance
By default, Open WebUI allows anyone who can reach port 3000 to register an account and chat. That’s fine on a personal laptop, but on a home server or VPS you’ll want to lock things down.
The quickest hardening steps:
- Enable authentication: In Settings → Admin → Auth, turn on “Allow user registration” only if you want self-signup. For a single-user instance, disable it after creating your admin account.
- Set up a reverse proxy: Put Nginx or Caddy in front of Open WebUI with HTTPS. Let’s Encrypt makes this free, and Caddy’s automatic HTTPS removes most of the configuration pain.
- Restrict network exposure: Bind the Docker container to
127.0.0.1:3000:8080if you only access it locally, or use Tailscale/Cloudflare Tunnel for remote access without opening ports. - Rotate the secret key: If you ever need to invalidate all sessions, change
WEBUI_SECRET_KEYand recreate the container.
I run mine behind a Cloudflare Tunnel with Zero Trust access rules. Only my devices can reach the UI, and all traffic stays encrypted end to end. The whole tunnel setup took about ten minutes.
Step 6: Add a Knowledge Base
Open WebUI supports Retrieval-Augmented Generation out of the box. This means you can upload PDFs, text files, or entire directories and chat with them. The model answers questions based on your documents, not just its training data.
To set it up, go to Settings → Admin → Connections and make sure an embedding provider is configured. Ollama works as an embedding engine — just pull nomic-embed-text and point the RAG embedding setting at it:
ollama pull nomic-embed-textThen in Open WebUI’s admin panel, set RAG Embedding Engine to Ollama and RAG Embedding Model to nomic-embed-text. After that, any chat has a “Knowledge” panel where you can attach files or folders. Open WebUI chunks them, embeds them, and retrieves relevant passages automatically.
I’ve used this for internal documentation at work. Upload the employee handbook, the IT policy PDFs, the network runbooks, and suddenly you have a searchable chat layer over your own knowledge base. It’s not perfect — long documents can overwhelm the context window — but for targeted Q&A it’s genuinely useful.
What This Setup Actually Costs
Let’s talk numbers. The software is free. The hardware depends on what you already own. If you have a gaming PC with a 3060 or better, you’ve got a capable local inference box. If you’re on a laptop with integrated graphics, CPU inference is slower but still usable for 7B models.
Electricity is the hidden cost. Running a GPU at full tilt draws 200–300W. A CPU-only server draws less but inference is slower. My 3060 rig idles around 60W and spikes to 180W during generation. At my local rate of roughly PHP 13 per kWh, even a few hours of daily inference adds up — but it’s still cheaper than cloud API bills for moderate use. If you’re trying to decide between self-hosting and paying for cloud access, my analysis in Done Renting AI breaks down the actual cost tradeoffs.
Where This Setup Falls Short
Open WebUI is impressive, but it’s not magic. The biggest limitation is model quality. Even the best open-weight models still trail GPT-5.6-class systems on complex reasoning, long-horizon planning, and nuanced instruction following. If you need frontier-level capability, you’ll still reach for a cloud API.
The second limitation is multi-user resource contention. Ollama loads models into GPU VRAM or system RAM. If two people hit the same 7B model simultaneously, they share that memory. With smaller models it’s fine. With 34B or 70B models on consumer hardware, concurrent requests can cause out-of-memory crashes.
I solve this by routing heavy workloads to a cloud model via Open WebUI’s multi-model chat. Lightweight tasks — code review, document Q&A, quick drafts — run locally. Complex analysis gets routed to a hosted provider. The interface handles both transparently. For squeezing larger models onto smaller GPUs, the techniques in AirLLM: Run a 70B LLM on a 4GB GPU apply here too.
The Bottom Line
Open WebUI and Ollama give you a private, ChatGPT-style interface without ongoing API costs or data leaving your network. The setup takes under twenty minutes, the commands are tested, and the stack is mature enough for daily use. It won’t replace every cloud LLM in your workflow, but for the tasks where privacy and cost matter most, it’s the most practical self-hosted option available right now.
If you’ve been on the fence about local AI, this is the week to try it. The hardware you already own is probably enough.