mirror of
https://github.com/ollama/ollama.git
synced 2026-06-05 21:05:00 +08:00
9db4bdbad6
* broad lint fixes to sidestep CI scope glitch * runner: Remove CGO engines, use llama-server exclusively for GGML models Remove the vendored GGML and llama.cpp backend, CGO runner, Go model implementations, and sample. llama-server (built from upstream llama.cpp via FetchContent) is now the sole inference engine for GGUF-based models. (Safetensor based models continue to run on the new MLX engine.) This allows us to more rapidly pick up new capabilities and fixes from llama.cpp as they come out. On windows this now requires recent AMD driver versions to support ROCm v7 as llama.cpp currently does not support building against v6. * llama/compat: load Ollama-format GGUFs in llama-server Squashed from upstream/jmorganca/llama-compat on 2026-04-29. Source tip:0c33775d37. Original source commits: -25223160dllama/compat: add in-memory shim so llama-server can load Ollama-format GGUFs -7449b539allm,server: route Ollama-format gemma3 blobs through llama/compat -436f2e2b1llama/compat: make patch-apply idempotent -8c2c9d4c8llama/compat: extend gemma3 handler to cover 1B and 270M blobs -021389f7bllama/compat: shrink clip.cpp injection from 18 lines to 1 -61b367ec2llama/compat: shrink patch to pure call-site hooks (34 -> 20 lines) -36049361cllama/compat: simplify shim (gemma3-tested) -8fa664865llama/compat: add qwen35moe text handler -db0c74530llama/compat: add qwen35moe vision (clip) support -2a388da77llama/compat: split shared infra into a util TU -9a69a17dcllama/compat: document non-public API dependencies -d0f38a915llama/compat: add gpt-oss and lfm2 handlers -086071822llama/compat: add mistral3 text handler (vision TODO) -63bde9ff7llama/compat: add mistral3 vision (clip) support -3a57b89d5llama/compat: apply LLaMA RoPE permute to mistral3 vision Q/K -99cb87439llama/compat: add qwen35, gemma4, deepseek-ocr handlers -2c7850dballama/compat: add nemotron_h_moe handler (latent FFN + MTP skip) -9e3b54225llama/compat: add llama4 text + clip handlers -034fee349llama/compat: add gemma4 clip handler (gemma4v projector) -9945c5a93server: remove dhiltgen/* compat redirect table -5d4539101llama/compat: rewrite gemma4 tokenizer model to BPE -7e0765327llama/compat: add glm-ocr text handler + text-loader load-op hook -f1bd1a25allama/compat: add glm-ocr clip handler (glm4v projector) -4b5cf3420llama/compat: collapse text-loader hook back to one new patch line -eb4ecf4fcllama/compat: extend gemma4 clip handler to gemma4a (audio) -a23a5e76fllama/compat: fix gemma4a per-block norm tensor mapping -cd2dcaff4llama/compat: add embeddinggemma handler -1ce8a6b26llama/compat: add qwen3-vl + qwen2.5-vl handlers -fd98ffa1ellama/compat: add gemma3n + glm4moelite handlers -cc7bdf0bcllama/compat: handle null buft in maybe_load_tensor -0c33775d3llama/compat: disable mmap when load_op transforms text-side tensors * refine implementation * ci: fix windows MLX build * ci: fix windows llama-server build * ci: fix windows rocm build * ci: windows mlx tuning Shorten long-tail on build, and get OllamaSetup.exe back under 2g limit * ci: fix windows dependencies * win: fix dependency gathering * disable openmp * win: arm64 cross-compile build also DRY out CI steps * scheduler improvements * ci: improvements from #15982 * win: favor ninja for faster developer builds * win: fix build * win: fix arm64 cross-compile * win: avoid spaces in compiler path * misc discovery fixes, and bos handling * lint fixes * win: fix arm cross-compile build/CI bugs * llama.cpp update * win: handle multiple CRT dirs * vulkan: add windows iGPU detection * fix creation bugs for patched models, other refactoring work * tune batch size for better performance * ci and lint fixes * fix repeat_last_n bug * build: revamp build for better developer UX * amd, sampler, qwen3next fixes * version bump * fix mlx build * revamp GPU discovery Scanning the output of llama-server is turning out to be too error prone across llama.cpp updates, so this switches to a thin dynamic library load against the bundled GGML libraries so more details can be gathered from the API. * version bump * missing file * ci: fix cache miss on rocm build * refine vulkan dep handling * fix ps reporting bug on full GPU load * improve cmake wiring for customized local builds * version bump * docker build arg cleanup * improve windows exit error logs * fix community gemma4 support and ci flakes * fix mlx unit test * tighten up ps logic to avoid double counting fit log lines * version bump * fix ps view for full gpu layer offload * add MTP wiring for llama-server and create with GGUFs * pick best template by capabilities * version bump * ci: harden apt repos * remove unused cpu core discovery * adjust batch default logic to reduce OOMs * support larger tool calls * fix audio support, template show * qwen35 mtp patch support * flesh out dtypes * rocm deps * version bump * lint fix * block broken gfx1150 on windows * fix qwen3.5 moe mtp tensors in patch * mmproj oom fallback and vulkan on by default * qwen MTP compat fix * version bump * ci: fix WoA cross-compile * ci: workaround ui tool in cross-compile * version bump * win: enable OpenMP for CPU builds * build: improve developer UX * ci: windows path workaround for CPU build * win: fix WoA dependencies * win: fix large offset reads for mmproj patched loads * version bump * fix vulkan dup detection * add OLLAMA_IGPU_ENABLE and largely disable iGPUs by default * opt-in MTP, win large offset, integraton fixes * fix unit test scheduler interaction hang * fix multi-gpu filtering * version bump * review comments * fix thinking level * fix linux rocm ordering and granite 3.3 template * version bump * ci fix - non-shallow MLX checkout * bypass linux sysfs unit test on windows --------- Co-authored-by: jmorganca <jmorganca@gmail.com>
104 lines
4.6 KiB
Plaintext
104 lines
4.6 KiB
Plaintext
---
|
|
title: Windows
|
|
---
|
|
|
|
Ollama runs as a native Windows application, including NVIDIA and AMD Radeon GPU support.
|
|
After installing Ollama for Windows, Ollama will run in the background and
|
|
the `ollama` command line is available in `cmd`, `powershell` or your favorite
|
|
terminal application. As usual the Ollama [API](/api) will be served on
|
|
`http://localhost:11434`.
|
|
|
|
## System Requirements
|
|
|
|
- Windows 10 22H2 or newer, Home or Pro
|
|
- NVIDIA 452.39 or newer Drivers if you have an NVIDIA card
|
|
- AMD ROCm v7 / HIP7-capable driver stack for ROCm acceleration, or a Vulkan-capable AMD Radeon driver for Vulkan acceleration
|
|
|
|
Ollama uses unicode characters for progress indication, which may render as unknown squares in some older terminal fonts in Windows 10. If you see this, try changing your terminal font settings.
|
|
|
|
<Note>
|
|
Some RDNA2 / Radeon RX 6000 systems, including RX 6800-class cards, may not
|
|
expose ROCm v7 on current Windows AMD drivers. Vulkan is enabled by default
|
|
and is the recommended fallback for those systems. If a mixed iGPU/dGPU
|
|
system selects an unstable Vulkan iGPU, set `GGML_VK_VISIBLE_DEVICES` to the
|
|
discrete GPU index.
|
|
</Note>
|
|
|
|
## Filesystem Requirements
|
|
|
|
The Ollama install does not require Administrator, and installs in your home directory by default. You'll need at least 4GB of space for the binary install. Once you've installed Ollama, you'll need additional space for storing the Large Language models, which can be tens to hundreds of GB in size. If your home directory doesn't have enough space, you can change where the binaries are installed, and where the models are stored.
|
|
|
|
### Changing Install Location
|
|
|
|
To install the Ollama application in a location different than your home directory, start the installer with the following flag
|
|
|
|
```powershell
|
|
OllamaSetup.exe /DIR="d:\some\location"
|
|
```
|
|
|
|
### Changing Model Location
|
|
|
|
To change where Ollama stores the downloaded models instead of using your home directory, set the environment variable `OLLAMA_MODELS` in your user account.
|
|
|
|
1. Start the Settings (Windows 11) or Control Panel (Windows 10) application and search for _environment variables_.
|
|
|
|
2. Click on _Edit environment variables for your account_.
|
|
|
|
3. Edit or create a new variable for your user account for `OLLAMA_MODELS` where you want the models stored
|
|
|
|
4. Click OK/Apply to save.
|
|
|
|
If Ollama is already running, Quit the tray application and relaunch it from the Start menu, or a new terminal started after you saved the environment variables.
|
|
|
|
## API Access
|
|
|
|
Here's a quick example showing API access from `powershell`
|
|
|
|
```powershell
|
|
(Invoke-WebRequest -method POST -Body '{"model":"llama3.2", "prompt":"Why is the sky blue?", "stream": false}' -uri http://localhost:11434/api/generate ).Content | ConvertFrom-json
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
Ollama on Windows stores files in a few different locations. You can view them in
|
|
the explorer window by hitting `<Ctrl>+R` and type in:
|
|
|
|
- `explorer %LOCALAPPDATA%\Ollama` contains logs, and downloaded updates
|
|
- _app.log_ contains most resent logs from the GUI application
|
|
- _server.log_ contains the most recent server logs
|
|
- _upgrade.log_ contains log output for upgrades
|
|
- `explorer %LOCALAPPDATA%\Programs\Ollama` contains the binaries (The installer adds this to your user PATH)
|
|
- `explorer %HOMEPATH%\.ollama` contains models and configuration
|
|
- `explorer %TEMP%` contains temporary executable files in one or more `ollama*` directories
|
|
|
|
## Uninstall
|
|
|
|
The Ollama Windows installer registers an Uninstaller application. Under `Add or remove programs` in Windows Settings, you can uninstall Ollama.
|
|
|
|
<Note>
|
|
If you have [changed the OLLAMA_MODELS location](#changing-model-location), the installer will not remove your downloaded models
|
|
</Note>
|
|
|
|
## Standalone CLI
|
|
|
|
The easiest way to install Ollama on Windows is to use the `OllamaSetup.exe`
|
|
installer. It installs in your account without requiring Administrator rights.
|
|
We update Ollama regularly to support the latest models, and this installer will
|
|
help you keep up to date.
|
|
|
|
If you'd like to install or integrate Ollama as a service, a standalone
|
|
`ollama-windows-amd64.zip` zip file is available containing only the Ollama CLI
|
|
and GPU library dependencies for Nvidia. Depending on your hardware, you may also
|
|
need to download and extract additional packages into the same directory:
|
|
|
|
- **AMD GPU**: `ollama-windows-amd64-rocm.zip`
|
|
- **MLX (CUDA)**: `ollama-windows-amd64-mlx.zip`
|
|
|
|
This allows for embedding Ollama in existing applications, or
|
|
running it as a system service via `ollama serve` with tools such as
|
|
[NSSM](https://nssm.cc/).
|
|
|
|
<Note>
|
|
If you are upgrading from a prior version, you should remove the old directories first.
|
|
</Note>
|