Setting Up a Local AI Coding Agent with Ollama and Aider

Over the past few months, I've experimented with several workflows for using LLMs in software development. In this article, I describe the local setup I've validated on my own PC: Ollama running on Windows, Aider inside Ubuntu on WSL2, and a code-focused model running entirely on the local machine.

The machine I used for my first experiment runs Windows 11 with an Intel Core i7-1355U CPU, 48 GiB of RAM, and no dedicated NVIDIA GPU. It's generally a capable machine, but not particularly well suited for AI inference. Because it lacks a dedicated GPU, I had to run the model in CPU-only mode.

The goal of this setup was to create an environment where both the source code and the data remain entirely on the local machine—far from the cloud-based frontier model approach.

From an architectural standpoint, I decided to keep Ollama on Windows because I also use the installed models with other Windows applications, and I wanted to avoid maintaining duplicate model installations. Aider, on the other hand, runs inside WSL, where installation and configuration are simpler and the Linux command-line environment is more convenient.

Ollama remains exposed on the local port 11434. Aider, running inside Ubuntu WSL2, connects to Ollama through the OLLAMA_API_BASE environment variable.

This approach avoids duplicating models inside WSL while keeping development repositories in the Linux filesystem under ~/repos, where Git and other command-line tools work much more naturally.

Local Setup

Since this machine is not an AI workstation, the solution had to be realistic. That meant avoiding massive GPU-hosted models, cloud APIs, and overly complex automation.

The final architecture looks like this:

Windows
 ├─ Ollama
 │   └─ qwen2.5-coder:14b
 │
 └─ WSL Ubuntu
     ├─ Aider
     ├─ Git
     └─ Development repositories

Keeping Ollama on Windows while running Aider inside WSL was a natural choice. I already use Ollama with several Windows applications, so duplicating the models inside WSL made little sense. At the same time, I much prefer working with the Linux command line for software development.

Installing Ollama on Windows

I installed Ollama on Windows and downloaded the primary coding model:

ollama pull qwen2.5-coder:14b

I also installed the smaller model for quicker testing:

ollama pull qwen2.5-coder:7b

To verify the installation:

ollama list

The WSL-to-Windows Connectivity Issue

Initially, Aider running inside WSL couldn't communicate with Ollama running on Windows.

To make it work, I had to configure WSL to use mirrored networking.

From PowerShell:

notepad $env:USERPROFILE\.wslconfig

Then I added the following configuration:

[wsl2]
networkingMode=mirrored
dnsTunneling=true
firewall=true
autoProxy=true

Finally, I restarted WSL:

wsl --shutdown

After reopening Ubuntu, I tested the connection:

curl http://127.0.0.1:11434/api/tags

The response confirmed that WSL could successfully reach the Ollama server running on Windows at 127.0.0.1:11434.

Installing Aider in WSL

Inside Ubuntu/WSL:

sudo apt update
sudo apt install -y git python3 python3-pip python3-venv pipx curl
python3 -m pipx ensurepath
exec $SHELL -l
pipx install aider-chat

To verify the installation:

aider --version

I also configured Git so I could version-control my experiments:

git config --global user.name "xxxx"
git config --global user.email "[email protected]"

Creating the First Repository

I created a simple test project:

mkdir -p ~/repos
cd ~/repos
mkdir my-project
cd my-project
git init

Then I launched Aider:

cd ~/repos/my-project
OLLAMA_API_BASE=http://127.0.0.1:11434 aider --model ollama_chat/qwen2.5-coder:14b --no-show-model-warnings

When Aider displayed:

Aider v0.86.2
Model: ollama_chat/qwen2.5-coder:14b
Git repo: .git

I finally knew the basic setup was working correctly.

Opening the Repository from Windows Explorer

For convenience, I also wanted to access the repository directly from Windows Explorer.

From inside WSL:

explorer.exe .

This command opens the current WSL directory directly in Windows File Explorer.

What's Next?

Once I had Aider working with a local Ollama instance, there was still one thing I wanted to understand better: what was happening under the hood. How many tokens were being processed? How long did inference take? What kind of throughput could I expect?

We'll cover all of that in the next installment. 😉