How to run AI locally on Windows 11 with Ollama or Foundry Local

Install Ollama from ollama.com, open a terminal and run ollama run gemma4:e2b, or install Microsoft's Foundry Local with winget install Microsoft.FoundryLocal and run foundry chat qwen2.5-0.5b. The model downloads once, then runs on your PC with no internet connection and no per-use cost. Apps connect to it through a local address such as http://localhost:11434.

Privacy and cost

Ollama and Foundry Local are the two easiest ways on Windows 11 to run a language model on your own processor, graphics card or NPU instead of a company's servers. Ollama's FAQ says that when you run locally, it doesn't see your prompts or data. Microsoft says the same of Foundry Local: prompts and outputs are processed on the device, and the network is used only to download models and components.

There are no per-token charges and no subscription, and Foundry Local doesn't need an Azure subscription. The trade-off is that your PC does the work, so speed and model size depend on your hardware.

Hardware

Any recent Windows 11 PC can run a small model, and a graphics card with its own memory, or an NPU, makes it faster. GPU memory matters most. For the small gemma4:e2b model, Ollama recommends 8 GB of available VRAM; with less, it uses system RAM instead, and responses may be slower.

Ollama supports Windows 10 22H2 or newer. The app needs at least 4 GB of disk space, and models can take "tens to hundreds of GB", according to its Windows docs. On NVIDIA cards it needs driver 551.61 or newer. AMD Radeon cards use ROCm where the driver supports it, with Vulkan as the fallback.

Copilot+ PCs have an NPU of at least 40 TOPS and 16 GB of RAM. Foundry Local can run models on Qualcomm Snapdragon X NPUs, on Intel NPUs from Arrow Lake onward, and on supported AMD NPUs, each with a minimum driver version.

Ollama

Setup takes a few minutes, plus the model download.

1. Download OllamaSetup.exe from ollama.com/download. It doesn't need administrator rights and installs into your home folder.
2. Run the installer. To put Ollama elsewhere, run OllamaSetup.exe /DIR="D:\Ollama".
3. Open Windows Terminal and download a model (about 7.2 GB):

ollama pull gemma4:e2b

1. Start a chat with it:

ollama run gemma4:e2b

Type a question and press Enter. Type /bye to leave.

1. Run ollama ps to see where it's running. The Processor column says whether the model is on the GPU, the CPU or split between them.

Models are stored in your user folder. If your system drive is small, set the OLLAMA_MODELS environment variable for your account to another folder, then quit and restart Ollama. By default a model stays in memory for 5 minutes after you last use it, then it's unloaded.

Foundry Local

Microsoft's local AI runtime downloads models from a curated catalog, picks the best version for your hardware, and runs it on the CPU, GPU or NPU. It's free, and its command-line tool is in public preview.

1. Open a terminal and run winget install Microsoft.FoundryLocal. You need admin rights to install it.
2. Check it worked: foundry --version.
3. See which models suit your PC: foundry model list. The first run downloads the execution providers for your hardware.
4. Start a chat: foundry chat qwen2.5-0.5b. The model downloads on first use.

When you use a model's alias like qwen2.5-0.5b, Foundry Local chooses the variant for your hardware: a CUDA version for NVIDIA GPUs, an NPU version for Qualcomm NPUs, or the CPU version as a fallback. If you have an Intel NPU, install Intel's NPU driver first. If a command reports "Request to local service failed", run foundry server restart.

Which model size suits my PC?

Pick a model that fits in your GPU memory, or in RAM if you have no GPU. Ollama's library lists the download size of each model in a family, which gives a first guide. For Google's gemma4 family:

On Foundry Local, foundry model list shows the models available to you, and Microsoft's own examples use the small qwen2.5-0.5b.

Context length adds to memory use too. Ollama's FAQ says required RAM scales with the number of parallel requests times the context length, and its default context window is 4,096 tokens. Start with the smallest model and move up only if its answers aren't good enough for your task.

Limits compared with cloud models

The models that fit on a PC are smaller than the largest cloud models, and your hardware sets the speed. Foundry Local's catalog is deliberately curated to models small enough to distribute to ordinary PCs. Ollama's library is broader, but large models need large amounts of memory, and a model that doesn't fit in GPU memory runs partly on the CPU and slows down. Each model is gigabytes on disk, and they add up. Foundry Local is also designed for one person on one device, not for serving many users.

Before you rely on a local model, try it on the tasks you actually do, such as rewriting or summarizing short text, and keep a cloud model as an option for the rest.

Connecting apps

Apps reach a local model through a web address on your own PC, called localhost, so nothing goes over the internet.

Ollama serves its API at http://localhost:11434 and only listens on your own machine unless you change OLLAMA_HOST. Its OpenAI-compatible API is at http://localhost:11434/v1/. Clients that require an API key can send any value, because Ollama ignores it.

Foundry Local has an OpenAI-compatible local server too. Its port is assigned each time the server starts, so run foundry server status to see the current address.

PowerToys Advanced Paste lists Foundry Local and Ollama as local providers for its Paste with AI feature. OceanBreeze, a ring of tools that opens around your pointer when you hold Caps Lock, finds Ollama on localhost:11434 by itself and can download a Foundry Local model for you, including NPU models on Copilot+ PCs. You choose a model for each tool, so Rewrite can stay on your PC while Ask AI uses a cloud model. With a local model, nothing you select leaves your PC.

How to use AI on selected text in any Windows app compares OceanBreeze with Click to Do, Copilot and PowerToys for that kind of work.

Common questions

Sources

Mythic Brain makes OceanBreeze. Facts about other products come from the sources above and were checked on 14 November 2026.