When my computer spoke to me for the first time

Computers fascinated me from an early age. I could run programs, write code and tell mine exactly what to do. But talk to it? Simply ask a question and hear a spoken reply? For a long time, that was something I knew from films and wished I could have myself.

When I first saw Jarvis in a film, that thought came straight back. That was what I found so cool: a computer you could have a conversation with. One you could speak to instead of opening the right program or menu for every little task.

Of course, I knew how much movie magic was involved. But the wish stayed with me. Eventually, I wanted to find out how far I could get with my own hardware.

First, just an answer

My first step was a small console application written in C#. It ran in the terminal, initially without a graphical interface. A terminal, a text input and a connection to a local language model. I did not need anything more to get started.

The question was simple: would I get an answer back at all?

First, I wanted to get the connection working. Enter some text, send it to the model, receive the response and display it in the terminal. Before I could think about a voice or a conversation, that basic path had to work.

For my first experiments, I ran a small Qwen language model locally on my computer. My RTX 2080 handled the computations. There was something particularly appealing about that: the answer was being generated on my own computer. I could explore for myself what was possible and where the limits were.

Even a text response was an important step. That old wish was becoming something I could actually work on. And once this part worked, the next question was obvious: how could I get the computer to say its answer out loud?

Giving the text a voice

That brought text-to-speech, or TTS, into the picture. I used Piper for this. At first, the aim was to make the answer to my typed input audible. The microphone would come later.

For an actual conversation, I would also need the reverse path: turning spoken language into text. Whisper would handle that. The local language model would process the recognised text and generate a response, which Piper would then read aloud.

On paper, the goal was a simple sequence:

In practice, I first had to connect those individual pieces. An answer in the terminal did not mean anything would come out of the speakers. That also required working speech synthesis and playback of the resulting audio.

That final step was the one I was particularly excited about. Text on a screen was familiar. A voice coming from my own application would feel different.

It spoke. I just could not understand it.

Then the moment arrived: I heard a word from the speakers for the first time.

Except it sounded like neither German nor English. At least, I could not understand any of it. I finally had a voice, just not one I could make sense of yet.

Looking back, that moment sums up quite a few of my projects. You connect several components, work towards a particular result and initially get something that is both a success and a failure. There is sound. You are heading in the right direction. But a crucial detail is still missing somewhere.

It did not dampen my enthusiasm. I had heard that the basic idea worked. Now I had something specific to investigate and improve.

The beginning of my own assistant

Those first words had certainly not created the Jarvis from the films. But I had a starting point: my own application, a local language model and a pipeline that could turn responses into audible speech.

That was enough to make me want to keep going.

This series tells the story of how it became my personal AI assistant. I also want to record the steps that are barely visible in the finished system: small experiments, wrong assumptions and mistakes I could laugh about later.