Digitalisation4 min read
Local AI with Ollama on Windows: Installation, Hardware and Limits
Ollama runs language models directly on your own PC, and your prompts stay there. What the computer needs, how installation works and where local models reach their limits.
- #ki
- #produktivitaet
- #digitalisierung
Ollama is a free, open-source program for running language models directly on your own computer. The appeal for a small business is obvious: text you give a local model does not leave the PC. Whether that is enough for everyday work depends mainly on the hardware and on what you expect from the model.
What Ollama is, and what it is not
Ollama downloads models, starts them and makes them available through the command line and an API. The project is released under the MIT licence (Ollama on GitHub). The models themselves come from other providers, such as Llama from Meta or Gemma from Google, and each has its own licence terms.
According to the Ollama FAQ, prompts stay on the machine when you run models locally (Ollama FAQ). That does not apply to everything Ollama now offers, though. There are also cloud models, recognisable by the :cloud tag in the name, for example gemma4:cloud. They run on Ollama's servers and require you to sign in (Ollama docs on cloud models). Anyone using Ollama for privacy reasons should stick to local models.
Hardware: what the computer needs
For Windows, Ollama requires Windows 10 version 22H2 or newer. An NVIDIA graphics card needs driver 551.61 or later, and AMD cards have their own driver requirements. The program itself takes at least 4 GB of disk space, plus the models (Ollama docs for Windows).
The library on ollama.com shows how large each model is. A few examples:
- llama3.1:8b: 4.9 GB download (Ollama library: Llama 3.1).
- gemma4:e2b: 4.6 to 7.5 GB depending on the variant, described as designed for laptops and mobile devices.
- gemma4:26b: 16 to 19 GB (Ollama library: Gemma 4).
As a rough guide, the model has to fit completely into system memory or graphics memory, and Windows and other programs need their share as well. On a graphics card, a model usually responds faster than on the processor. Ollama's documentation does not give fixed minimum values. The most reliable test is to try a small model.
Installing on Windows
- Download the installer from ollama.com/download/windows and run it. Administrator rights are not needed; Ollama installs into your user folder.
- After installation, Ollama runs in the background with an icon in the system tray.
- Open Command Prompt or PowerShell and type
ollama --version. If a version number appears, everything worked.
By default, models are stored in the .ollama folder in your user directory. If space is tight there, set a user environment variable called OLLAMA_MODELS to point to another folder. Then quit Ollama from the tray icon and start it again.
Loading and testing a model
With ollama run gemma4:e2b, Ollama downloads the model the first time and then opens a prompt directly in the terminal. You can ask a question there or paste in a text and ask for a summary or a plainer version. /bye ends the session.
A common misunderstanding is that the model can access files or emails. It cannot. A request like "Summarise the latest project report" only works if you paste the text of the report in yourself.
To see whether the model is running on the graphics card, type ollama ps in a second window. The PROCESSOR column shows whether it runs fully on the GPU, fully on the CPU or split between them. By default, Ollama frees the memory after five minutes without a request.
Settings worth knowing
The "temperature" controls how freely the model phrases things. According to the Modelfile documentation, the default is 0.8, and higher values make answers more creative (Ollama Modelfile docs). For factual tasks such as summaries, a lower value can make sense. It is set in a file called Modelfile with the line PARAMETER temperature 0.5, and ollama create turns that into a custom model. For getting started, there is no need to change it.
Network access matters more. By default, Ollama is only reachable on your own machine at localhost:11434. The OLLAMA_HOST variable can open it to the whole network. Only do that when it is clear who will then have access, because other devices on the network can then talk to the API.
Connecting other programs
Other programs can use the model through the API at http://localhost:11434. Some note-taking and chat front ends have an Ollama option in their settings; whether a particular program supports it is in its documentation. Office software such as Microsoft To Do cannot simply be connected; that would need custom programming.
When local AI is not worth it
Models of the size that runs on an ordinary office PC are generally weaker than the large cloud services. They are often good enough for summaries, rewording and first drafts, but reach their limits sooner with complex questions, current knowledge or long documents. Facts need checking with any language model, and with small ones in particular.
The effort is part of the equation too: someone has to keep Ollama and the models up to date. For the occasional rewrite, a cloud service with a proper data processing agreement is often simpler. Running models locally pays off mainly when confidential texts are processed regularly and should not be seen by any external service.
Read next
Digitalisation
European AI Tools 2026: Privacy-First Solutions for SME Daily Productivity
In 2026, 58% of Germans use AI but crave more European options. Discover privacy-compliant AI tools tailored for SMEs to boost organization, time management, and digitalization – practical and GDPR-safe.
Digitalisation
AI in Everyday Life 2026: 58% of Germans Use It – Practical Tips for SMEs
Bitkom survey reveals: 58% of Germans aged 16+ use AI regularly. This article provides SMEs with practical tips on integrating AI into daily operations for boosted productivity.
Digitalisation
Digitalization 2026: Rising Budgets, Missing Benefits
Companies are boosting digitalization and AI budgets in 2026, but true benefits often fail to materialize. Discover common pitfalls and practical tips for SMEs to make digital projects pay off.