DIY

Rhasspy

Posted ulysses

Installation for Debian or Raspberry Pi OS

The installation process is detailed in the official homepage of Rhasspy; see here. Below I provide the process specific to Raspberry Pi. First, use the command dpkg-architecture | grep DEB_BUILD_ARCH= to check the architecture. For Raspberry 4 with a 32bit system, it is armhf. Then download and install the corresponding .deb file.

cd ~
wget https://github.com/rhasspy/rhasspy/releases/download/v2.5.10/rhasspy_2.5.10_armhf.deb

sudo apt install -y ~/rhasspy_2.5.10_armhf.deb

The installation will install and start a standalone Mosquitto component. This might cause error if Mosquitto has been installed as an add-on of Home Assistant. In this case, the Mosquitto add-on can be shut down temporarily, and restarted after Rhasspy is installed. Also, Rhasspy will install the Mosquitto instance to /usr/sbin and experiences the errror can’t find command ‘mosquitto’ when starting Rhasspy service. This is solved by creating a symbolic link.

sudo ln -s /usr/sbin/mosquitto /usr/bin/mosquitto

If a Mosquitto add-on is used in Home Assistant, then we shall shut down and disable the standalone mosquitto service:

sudo systemctl stop mosquitto
sudo systemctl disable mosquitto

We shall run Rhasspy as a service. For this, we create a new file called rhasspy.service with the following content, and save it to /etc/systemd/system/rhasspy.service, where en should be replaced with the language of your choice.

[Unit] 
Description=Rhasspy Service 
After=syslog.target network.target 

[Service] 
Type=simple 
ExecStart=/bin/bash -c '/usr/bin/rhasspy --profile en 2>&1 | cat' 
RestartSec=1 
Restart=on-failure 
StandardOutput=syslog 
StandardError=syslog 
SyslogIdentifier=rhasspy 

[Install] 
WantedBy=multi-user.target

Then, use the following commands to set up the service.

cd /etc/systemd/system 
sudo chmod ugo+x rhasspy.service 
sudo systemctl start rhasspy.service
sudo systemctl enable rhasspy.service

Installation as Home Assistant Add-on

For this installation, we need to activate the Advanced Mode of Home Assistant. Then navigate to the Add-on Store and click the three-dot icon to add repositories by providing the link https://github.com/synesthesiam/hassio-addons. Then Rhasspy Assistant will appear as an available add-on and can be installed as usual. Start the add-on and now it can be accessed at http://localhost:12101.

Configuration

Mosquitto. We set to External with the following parameters:

  • Host: the IP address of Mosquitto service; in my case, as the Mosquitto service is an add-on of Home Assistant, it is set to the IP address of the Home Assistant (or localhost if Rhasspy and Home Assistant are hosted in the same device).
  • Port: 1883
  • Username: a user name of Home Assistant
  • Password: the corresponding password

Audio Recording. There are various options. In my case, the audio is recorded from a respeaker (on a Raspberry Pi Zero W), I set it to local command with the following parameters:

  • Record Program: gst-launch-1.0
  • Record Arguments: udpsrc port=12333 ! rawaudioparse use-sink-caps=false format=pcm pcm-format=s16le sample-rate=16000 num-channels=1 ! queue ! audioconvert ! audioresample ! filesink location=/dev/stdout
  • Sample Rate: 16000
  • Sample Width: 2
  • Channels: 1

Wake Word. Use the recommended Porcupine.

Speech to Text. I use Kaldi. This also requires downloading some files. See the Section Speech to Text below for more details.

Intent Recognition. I use Rasa NLU with the following parameters:

  • Server URL: http://192.168.1.99:5005, where 192.168.1.99 is the IP address is my Rasa server.

Text to Speech. Espeak

Audio Playing. aplay

  • The sound wav files can be customized by uploading your wav files to ${RHASSPY_BASE_DIR}/etc/wav/ and configure the Sounds section, where ${RHASSPY_BASE_DIR} points to /usr/lib/rhasspy.

Dialogue Management. Rhasspy

Intent Handling. Home Assistant with the following parameters:

  • Hass URL: http://192.168.1.99:8123, where 192.168.1.99 is the IP address of my Home Assistant instance.
  • Access Token: this is a long string obtained by logining into Home Assistant instance, clicking your username, scrolling down to the bottom, and then creating a token there.
  • Check “Send events to Home Assistant (/api/events)”

Note: If multiple instances of Rhasspy are installed in the server with satellites, at least for version 2.5.10, I recommend to disable intent handling in satellites to avoid errors like

[ERROR:2021-02-06 10:45:39,560] rhasspyserver_hermes: NotFound(404)
Traceback (most recent call last):
File “/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py”, line 1942, in full_dispatch_websocket
result = await self.dispatch_websocket(websocket_context)
File “/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py”, line 1984, in dispatch_websocket
raise websocket_.routing_exception
quart.exceptions.NotFound: NotFound(404)

Server and Satellites

When multiple Rhasspy instances are running in the same network, one will be configured to be base server and others should be configured to be satellites. The base server is typically responsible for speech to text, intent recognition, text to speech and intent handling, while satellites are responsible for audio recording, wake word detection and audio playing. In this setup, it is important to set distinct siteId for each Rhasspy instance.

For the base server, we disable Microphone, Wake, Sounds and Dialogue while configure the others properly. It is also important to list all satellite siteIds (comma-separated) under the following settings: Speech to Text, Intent Recognition, Text to Speech and Intent Handling. Otherwise, service requests from satellites will be ignored by the base server.

For satellites, we

  • configure Audio Recording, Wake Word and Audio Playing properly
  • set Speech to Text, Intent Recognition, Text to Speech to Remote HTTP, where we shall use the IP address or hostname of Rhasspy server
  • set Dialogue Management to Rhasspy
  • disable Intent Handling
  • set MQTT to internal (otherwise, the base server will receive excessive messages).

If you use Raspberry Pi Zero W + Respeaker Pi HAT 2-mic, you can use a simplified version of Rhasspy here.

Speech to Text

The shipped Kaldi can operate in two modes: open transcription mode and custom command mode. The former is more flexible but often less accurate, while the latter can only transcribe the voice commands defined in the sentences.ini file of Rhasspy. Here, for more accurate recognition, I use the custom command mode. For turn on/off devices, an example of sentences.ini is given below.

[TurnOnOff]
dev_name =  (tv | television | air conditioner | aircon | fan | speaker | light | lamp) {name}
dev_state = (on | off) {state}
room = (master | study | living) {room}
floor = (basement | ground | first | second | third | fourth) {floor}
turn <dev_state> [the] <dev_name> [in] [the] [<room>] [room] [of] [the] [<floor>] [floor]
turn [the] <dev_name> [in] [the] [<room>] [room] [of] [the] [<floor>] [floor] <dev_state>

Any change made to the sentences.ini file requires restarting Rhasspy and retraining the profile. If an error is encounter during training about the use of Rasa, the intent recognition component can be changed Fsticuffs and then changed back to Rasa after training.