Now I could call Joris

After image analysis, the question of how to reach my assistant was far from settled. It was there in the browser. Through Telegram, I could write to it while away. But at home, there were other devices I already used to communicate.

Why shouldn't I be able to call Joris?

A small phone system on my own network

I installed miniSIPServer as a Docker container on my NAS. That gave me a small phone system on my own network. SIP is the protocol participants use to register and establish calls. For this experiment, I didn't initially need an external telephone connection.

I registered Joris as its own SIP participant—effectively a phone with its own extension. My Gira G1 panels and the DoorBird doorbell joined the system too. Devices with different jobs could now be reached through the same phone system.

The doorbell could already call the Gira panels together, and its camera image was visible there too. Now my self-built assistant belonged to that same group of participants.

My assistant answered the phone

Both directions eventually worked with the upstairs Gira G1: Joris could call it, and I could reach Joris from it. When a call came in, it answered and greeted me in its selected voice. Then the conversation could begin.

Getting there took some troubleshooting. A saved favourite on the Gira initially pointed to the wrong SIP address. The call never reached Joris at all. Once the destination pointed to the reachable phone system, that route worked too. Sometimes the next obstacle was simply an address.

A new way to reach the familiar assistant

As with Telegram, I wanted to use the conversation processing I already had. Telephony handled the call and carried the audio. What I said then passed through my assistant's existing components.

Call from the Gira G1
          ↓
Internal SIP system connects to Joris
          ↓
Whisper turns my speech into text
          ↓
Joris processes the input through its shared conversation core
          ↓
Piper generates the spoken response
          ↓
The answer can be heard on the Gira G1

I didn't have to build a second assistant. Speech recognition, conversation logic, and speech synthesis already existed. What changed was the route my voice took in and the answer took back to me.

That route still needed work. A phone call needs a clear transition between listening and answering. The greeting couldn't be processed as my input, and hanging up had to stop any processing still under way. A phone widget in the web interface showed the call and conversation history; I could hang up there too.

The familiar waiting times returned as well. If the language model only began loading at the first sentence, I noticed immediately. It therefore stayed loaded while telephony was enabled. Interrupting a response by speaking over it remained an open task.

And then there was the doorbell

I had tried the connection between Joris and the Gira in practice. With the door station on the same system, I could now consider which conversations I wanted to pass to my assistant. A ringing device, a conversation, and an action in the house were still different things. Making Joris reachable shouldn't automatically trigger every possible action.

Internal calls came first

This setup initially ran within my own network. That was exactly where I wanted to start: with my devices and a conversation route I could try for myself.

Even an internal call felt like a big step. At the beginning, I had waited for my computer to say anything at all. Now I could call my assistant from another device in the house and hear its answer there.

An application on my computer had become a conversation partner I could reach. Not everywhere and not for every task. But in another part of my everyday life that I hadn't even considered during the first experiment.