Ollama and Open WebUI: Build Your Own Private ChatGPT at Home

Every time you paste a client contract, a financial table, or an internal email into a cloud chatbot, that text leaves your machine. For many people that is fine. For others, it is a policy problem, a compliance question, or just an uncomfortable feeling. And the monthly subscriptions add up.

There is a practical alternative: run the AI model on your own computer. Two free tools make this much easier than it sounds. Ollama runs the models. Open WebUI gives you a ChatGPT-style interface in your browser. This guide explains how they fit together, what hardware you need, how to set everything up, and where the limits are.

What Ollama and Open WebUI Each Do

Ollama is an open-source tool that downloads, runs, and manages large language models (LLMs) on your own machine. It is released under the MIT license, so it is free to use, including commercially. It works on macOS, Windows, and Linux, and it exposes a local API that other apps can talk to.

Open WebUI is a self-hosted web interface that sits in front of Ollama. It is designed to work offline when paired with local models. It adds the features people expect from a modern chatbot: saved conversations, document upload with retrieval (often called RAG, short for retrieval-augmented generation, where the AI looks up passages from your files before answering), optional web search, reusable model presets, and role-based user access.

Think of it as two layers: Ollama is the engine, Open WebUI is the dashboard.

What You Need Before You Start

The software is light. The models are not. Your RAM (and graphics card memory, if you have a GPU) decides which models run comfortably.

Rule of thumb: a commonly cited estimate is about 1.2 GB of memory per billion parameters for the 4-bit compressed models Ollama usually downloads. A 7B model therefore needs roughly 8 to 9 GB free, before your operating system takes its share. If a model does not fully fit in GPU memory, Ollama splits it between GPU and system RAM. That still works, but speed drops noticeably.

Apple Silicon Macs (M1 and later) are a good fit because the CPU and GPU share one pool of memory.

Step-by-Step: Install Ollama and Open WebUI

Step 1: Install Ollama

• Windows and macOS: download the installer from the official Ollama website and run it. Ollama starts in the background and appears in your system tray or menu bar.
• Linux: run the official install script:

curl -fsSL https://ollama.com/install.sh | sh

Step 2: Download and test a model

Open a terminal or command prompt and pull a small model first. This one is a sensible starting point for modest hardware:

ollama pull gemma3:4b
ollama run gemma3:4b

Type a question. If it answers, the engine works. Press Ctrl+D (or type /bye) to exit. Useful everyday commands are ollama list (see installed models), ollama ps (see what is loaded in memory), and ollama rm <model> (delete one).

Step 3: Install Open WebUI with Docker

With Ollama running on your computer, start Open WebUI in a container:

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

What the key parts do:

• -p 3000:8080 makes the interface available on port 3000 of your computer.
• -v open-webui:/app/backend/data stores your accounts, settings, and chat history in a Docker volume. Do not skip it, or you will lose data when the container is replaced.
• --restart always starts it again automatically after a reboot.

Now open http://localhost:3000 in your browser and create an account. The first account becomes the administrator.

If you would rather have a single container: Open WebUI also publishes an image that bundles Ollama inside it (the :ollama tag). It is convenient, but running Ollama natively on your machine usually makes GPU use and updates simpler.

Step 4: Connect Open WebUI to Ollama (if no models appear)

If the model list is empty, the container probably cannot find Ollama. Go to Admin Settings > Connections > Ollama and set the URL to http://host.docker.internal:11434 when Ollama runs on the same machine as Docker. On Linux, adding --network=host and pointing to http://127.0.0.1:11434 is another common fix. You can also pull new models from inside Open WebUI by typing a model name in the model selector.

Features Worth Setting Up

Chat with your own documents

Upload PDFs, notes, or policies and ask questions about them. For better retrieval, pull a dedicated embedding model (ollama pull nomic-embed-text) and select it under Admin Panel > Settings > Documents. If answers seem off-target, a weak or missing embedding model is the usual cause.

Web search (optional)

Open WebUI can pull in live search results through providers such as SearXNG, Brave Search, or Google PSE. Note the trade-off: when search is on, your queries go to that provider. For a fully offline setup, leave it off.

Model presets

Under Workspace > Models you can save a model with a fixed system prompt, for example "Customer support drafter" or "Contract summarizer", so your team gets consistent behavior without retyping instructions.

Multiple users and permissions

Open WebUI has admin, user, and pending roles, plus groups and model-level access control. That makes it workable as a shared assistant for a small team on one office server.

Security: The Mistake That Undoes the Privacy

Running locally keeps your data on your hardware, but only while the setup stays closed. Ollama listens on 127.0.0.1 by default and its API has no built-in authentication. If you expose port 11434 to the internet, anyone can use your models and your hardware.

• Do not publish the Ollama port. Let only Open WebUI talk to it.
• If you share Open WebUI beyond your own computer, put it behind HTTPS and a reverse proxy, and review the sign-up settings so strangers cannot create accounts.
• Keep both tools updated. To update Open WebUI, pull the new image, remove the old container, and run the same command again. Your data volume is kept.

The Honest Drawbacks

• Model quality: models that fit on a normal PC are smaller and generally less capable than the top cloud models. They are fine for drafting, summarizing, and simple coding help, and weaker at complex reasoning.
• Speed: without a decent GPU, responses can be slow.
• Upkeep: you handle updates, backups, and troubleshooting. There is no support team.
• Hardware cost: "free" assumes you already own a capable machine. A dedicated GPU server is a real expense.
• No built-in live knowledge: local models only know what they were trained on unless you enable web search or upload documents.

Who Should Use This (and Who Should Not)

Alternatives to Consider

• LM Studio: a desktop app with a built-in chat window. It is free to use but not open source.
• GPT4All: one of the simplest options for beginners who want a point-and-click app.
• Cloud assistants (ChatGPT, Claude, Gemini): stronger models and zero setup, but your prompts are processed on third-party servers and plans are paid.
• A hybrid setup: Open WebUI can also connect to OpenAI-compatible cloud APIs, so you can keep sensitive work on local models and use a cloud model only when needed.

A Note for Readers in the Gulf Region

For organizations in the UAE, Saudi Arabia, and neighboring countries, local hosting can simplify conversations about data residency, because prompts and documents stay on hardware you control, in a location you choose. Whether this satisfies the UAE or Saudi personal data protection laws (PDPL) depends on your data and your sector, so confirm with your legal or compliance team.

For Arabic use, test before you commit. Open WebUI supports multiple interface languages, but Arabic answer quality varies a lot between models, and right-to-left display can differ by version and browser. Try a few models with your real Arabic prompts and documents.

Frequently Asked Questions

Is Ollama really free?

Yes. Ollama is open source under the MIT license, and most models it runs are released under open licenses that allow free use. Check each model's license if you plan commercial use. Your costs are hardware and electricity.

Does this work without an internet connection?

Mostly yes. You need internet to download Ollama, Docker images, and models. After that, chat and document Q&A with local models can run offline. Web search obviously needs a connection.

Can I run it without a GPU?

Yes, but it is slower. Small models (around 3B to 4B parameters) are usable on a modern CPU with enough RAM. A GPU, or an Apple Silicon Mac, makes the experience much smoother.

Is a local setup actually private?

The model runs on your machine, so prompts are not sent to an AI provider. Privacy can still be lost through your own setup: exposing the Ollama port, enabling web search, or connecting cloud APIs. Review those settings.

Can my team use it at the same time?

Yes. Open WebUI supports multiple accounts and permissions, and Ollama can serve several users from one machine. Expect slower replies as more people use a single GPU. You can also connect more than one Ollama server.

Next Step

Start small. Install Ollama, pull one small model, and spend an afternoon testing it on your real tasks. If the quality is good enough, add Open WebUI and try document upload. You will quickly learn whether local AI covers your needs or whether a hybrid approach fits better.