How to Run AI Locally on Your Computer: Free, Private, Step by Step

Every week, the same question shows up in AI communities, worded a hundred different ways: “Can I run something like ChatGPT on my own computer — without paying, without an internet connection, and without sending my data to anyone?”

The answer is yes. It takes about 15 minutes, it’s completely free, and you don’t need a monster gaming PC or any coding background. By the end of this guide you’ll have a private AI chatbot running on your machine — and as a bonus, you’ll be able to chat with your own PDF documents, fully offline.

I’ll walk through every single step. Nothing is assumed, nothing is skipped.

Why Would You Run AI Locally?

Before the how, a quick why — because this explains a few decisions we’ll make later.

Privacy. When you use ChatGPT or any cloud AI, your messages go to a company’s servers. A local model runs entirely on your machine. Your medical questions, your business documents, your diary — nothing leaves your computer.
It’s free, forever. No subscription, no per-message cost, no rate limits. Once the model is downloaded, it costs you nothing but electricity.
It works offline. On a plane, in a village with bad internet, during an outage — your AI still works.
Nobody can take it away. Cloud models get changed, restricted, or retired. A model file on your disk is yours.

The honest trade-off: a model that fits on your laptop is not as smart as the biggest cloud models. Think “very capable assistant” rather than “genius.” For everyday tasks — writing, summarizing, explaining, brainstorming, chatting with documents — it’s more than enough, and the gap shrinks every year. The open-source AI models of 2026 are better than the paid cloud models of 2024.

Step 1: Check What Your Computer Can Handle

This is the step most guides skip, and it’s why beginners end up frustrated with a model that crawls or crashes. A large language model is basically a huge file of numbers (parameters) that has to fit into your computer’s memory. The rule is simple: your RAM decides which models you can run.

Find your RAM first:

Windows: Right-click the Start button → System → look for “Installed RAM.”
Mac: Apple menu () → About This Mac → look for “Memory.”
Linux: Open a terminal and run free -h.

Now match it to this table:

Your RAM	What you can run comfortably	What that feels like
8 GB	Small models (3–4B parameters)	Fast, good for chat, summaries, simple writing
16 GB	Medium models (7–8B parameters)	The sweet spot — handles most everyday tasks well
32 GB+	Large models (12–30B parameters)	Noticeably smarter, better reasoning and coding
Any + NVIDIA GPU (8GB+ VRAM)	Same sizes, but much faster	Answers stream in near-instantly

Two notes so nothing surprises you:

Macs with Apple Silicon (M1/M2/M3/M4) are excellent at this. Their unified memory doubles as graphics memory, so even a base MacBook Air runs local AI smoothly.
No graphics card? Totally fine. Models run on your regular processor — just a bit slower (words appear at reading speed rather than instantly).

You might wonder how an AI that needed a data center to train fits on a laptop at all. The trick is called quantization — compressing the model’s numbers to use less memory with only a tiny quality loss. The tool we’re about to install does this for you automatically, so you never have to think about it.

Step 2: Install Ollama (the Engine)

Ollama is a free, open-source program that downloads and runs AI models for you. It’s the most popular tool for this job by far — in Stack Overflow’s latest developer survey it was the #1 tool developers use for running models — and it’s the easiest.

Go to ollama.com in your browser.
Click Download and pick your system (Windows, Mac, or Linux).
Windows/Mac: Run the downloaded installer and click through it like any normal app. Linux: paste the one-line install command shown on the site into your terminal.
When it finishes, Ollama runs quietly in the background (you’ll see a little llama icon in your system tray or menu bar).

To confirm it worked, open a terminal — don’t worry, this is the only scary-looking part, and it’s two commands total:

Windows: Press the Start button, type cmd, press Enter.
Mac: Press Cmd+Space, type terminal, press Enter.

Type this and press Enter:

ollama --version

If you see a version number, you’re in business. If you see “command not found,” restart your computer (the installer needs a fresh session) and try again.

Step 3: Download and Run Your First Model

Still in that terminal, run one of these commands based on the RAM table from Step 1:

ollama run llama3.2        # 8 GB RAM  — small and quick (2 GB download)
ollama run qwen3:8b        # 16 GB RAM — the sweet spot (5 GB download)
ollama run qwen3:14b       # 32 GB RAM — noticeably smarter (9 GB download)

The first time, it downloads the model — a few minutes on decent internet, longer on slow connections. This only happens once.

When the download finishes, you’ll see a >>> prompt. That’s it. You are now talking to an AI running entirely on your computer. Try it:

>>> Explain what RAM is like I'm ten years old.

Watch the words appear. That’s happening on your hardware — you could unplug your router right now and it would keep working. Each word it generates is a token being predicted by the model in your memory, live.

To exit the chat, type /bye and press Enter. To chat again later, just run the same ollama run ... command — no re-download.

If it feels painfully slow: you picked a model too big for your RAM. Exit, then try the next smaller one. Slow-but-working almost always means “too big,” not “broken.”

Step 4: Get a Proper Chat Interface (Open WebUI)

The terminal works, but you probably want something that looks like ChatGPT — chat history, multiple conversations, easy model switching. That’s Open WebUI, a free open-source interface that connects to Ollama.

The simplest install is with Python’s package manager, which most computers already have:

Check you have Python: in the terminal, run python3 --version (Mac/Linux) or python --version (Windows). You need 3.11 or newer. If you don’t have it, grab it from python.org — standard installer, click through it (on Windows, tick the “Add Python to PATH” checkbox during install).
Install Open WebUI:

pip install open-webui

Start it:

open-webui serve

Open your browser and go to: http://localhost:8080

You’ll be asked to create an account the first time — this is a local account stored only on your machine, not an online signup. Once you’re in, pick your model from the dropdown at the top, and you have your own private ChatGPT. Chat history, folders, everything — all stored on your own disk.

(If pip gives you trouble, Open WebUI’s site also offers a Docker install — but for most beginners the pip route above just works.)

Step 5 (The Superpower): Chat With Your Own PDFs — Fully Offline

This is where local AI stops being a toy and starts solving real problems. Open WebUI has document chat built in — the same technique known as retrieval-augmented generation, where the AI reads your files and answers from them instead of guessing.

In Open WebUI, start a new chat.
Click the + button next to the message box and choose Upload Files.
Pick any PDF — a contract, a research paper, a manual, your notes.
Ask a question about it: “Summarize the key points” or “What does section 3 say about payments?”

The model reads the relevant parts of your document (using embeddings under the hood to find the right passages) and answers with the document as its source. Your file never leaves your computer. This is the thing people pay $20/month for — running free on your laptop.

For a permanent library, go to Workspace → Knowledge and upload documents there once; then any chat can reference them.

Troubleshooting: The Five Problems Everyone Hits

“Model is slow / computer freezes.” Model too big for your RAM. Use the next smaller size. Also close Chrome tabs — browsers eat RAM.
“ollama: command not found” after install. Restart the terminal, then restart the computer. On Linux, log out and back in.
“Open WebUI can’t find my models.” Make sure Ollama is running (llama icon in tray/menu bar). If not, just run ollama run llama3.2 once — it wakes the background service.
“Out of memory” errors. Same cause as slowness: smaller model, or a more heavily compressed version (add :q4 variants from the model’s page on ollama.com).
Wrong or made-up answers. Small models hallucinate more than big cloud ones — double-check facts, and use the PDF-chat approach from Step 5 when accuracy matters, since grounding the model in a real document cuts hallucinations dramatically.

Where to Go From Here

You now have something genuinely rare: an AI that answers to you and no one else. A few natural next steps, in order of fun:

Try different models. Browse ollama.com/library — there are models tuned for coding, for writing, for languages other than English. Swapping is one ollama run command.
Sharpen your prompts. Everything in our prompt engineering guide works on local models too — clear prompts matter even more when the model is smaller.
Understand what’s under the hood. If you’re curious how these models actually work, start with how large language models actually work.

The cloud AI companies would love you to believe this stuff requires their servers. It doesn’t. It requires a laptop, fifteen minutes, and the two commands you just ran.