Projects

Part 2: Google Assistant Controlled Devices

Mike Lewis & Rob Bell

Issue 23, June 2019

This article includes additional downloadable resources.
Please log in to access.

Log in

Use the Raspberry Pi to control satellite nodes within your network! Better yet, you can make satellites from Raspberry Pi, Arduino, or anything else which supports MQTT!

BUILD TIME: 90 MINUTES
DIFFICULTY RATING: Intermediate

In Issue 021 we set up a way to contact our Raspberry Pi from the outside world, and issue voice commands from a Google Assistant device such as the Google Home to control the Pi’s GPIO, We will now extend this and use the Pi as a hub, which will control other devices on our local network without them having to be opened to the internet.

There are multiple methods we could use to achieve setting up our Pi to communicate with our other devices. HTTP, MQTT or other network protocol such as UDP or TCP, for example.

We’ve chosen to go with MQTT since it’s a good fit for our devices and simple to set up.

In this article, we will demonstrate communication with two additional devices, an additional Raspberry Pi running a python script and an ESP8266 based WiFi Mini board using the Arduino IDE.

We will refer to our existing Pi that receives Google assistant voice commands as the server, and additional devices as clients.

Parts Required: JaycarAltronicsCore Electronics
1 × Raspberry Pi Zero W (or other WiFi Pi)-Z6303ACE04754
1 × Red 5mm LED*ZD0150Z0800CE05103
1 × 330Ω 1/4W Resistor*RR0560R7546PRT-14490
1 × WiFi MiniXC3802--
1 x Arduino Compatible RGB LED ModuleXC4428--

Parts Required:

* Quantity shown, may be sold in packs.

Server Build:

The Pi Web Server & MQTT Broker

The idea here is that when our server receives a particular HTTP request, it then publishes an MQTT message to a topic that our clients are subscribed to. Once the client receives the message it can then perform a certain task.

Continuing from part one, we should already have our Raspberry Pi web server set up. If not, you can catch up by reading Part 1 in Issue 021 or online at diyode.io/021pxgw

Instead of using a third party MQTT broker, we will add this additional role to the Pi Zero, using Mosquitto.

To install Mosquitto, run the following command in the terminal.

sudo apt-get install mosquitto mosquitto-clients

Adding the mosquitto-clients package allows us to send MQTT commands from the Pi’s terminal as well, which is useful for testing our devices later on.

Next, enable the broker and configure it to start on system startup.

sudo systemctl enable mosquitto

Check Mosquitto is running as expected.

sudo systemctl status mosquitto

If so, we know we have our broker running alongside our web server and testing the limits of our trusty Pi Zero.

It’s also good to know we didn’t run into any performance issues.

Let’s set up a client device to communicate with.

Client 1:

Additional Raspberry Pi

We went with another Rasberry Pi, but you could configure any controller capable of networking and MQTT. Like our Pi web server, it’s installed with a fresh copy of Raspbian Stretch.

We’ve also wired up a simple LED as a visual test. Connect this Pi to your network with internet access then update the system.

sudo apt-get update
sudo apt-get dist-upgrade

Next, install the python mqtt library.

sudo pip3 install paho-mqtt

Create and edit a new script in the home directory called client.py.

cd ~
nano client.py

Add the following python 3 code, replacing MQTT_SERVER with your server IP.

#! /usr/bin/python3
import paho.mqtt.client as mqtt
from gpiozero import LED
from time import sleep
MQTT_SERVER = "192.168.1.93"
MQTT_PATH = "client/led"
led = LED(4)
def on_connect(client, userdata, flags, rc):
  print("Connected with result code "+str(rc))
  client.subscribe(MQTT_PATH)
def on_message(client, userdata, msg):
  print(msg.topic+" "+str(msg.payload.decode("utf-8")))
  led.on()
  sleep(3)
  led.off()
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect(MQTT_SERVER, 1883, 60)
client.loop_forever()

You could also look into starting this script automatically when the Pi boots, but it’s beyond the scope of this tutorial.

Now start the script by running:

python3 client.py

This simple script will subscribe to the client/led topic and when it receives any message on this topic. It will then blink an LED on pin 4. We can now test if MQTT is working correctly by running the following command from your PI server.

mosquitto_pub -h localhost -t client/led -m "test"

If all went well, the LED should blink and you should see the following terminal output on the client Pi:

PI SERVER - ADDITIONAL ROUTE AND VOICE COMMAND

We will now modify our Pi web server and add a route that, once called, will publish the MQTT message for our client Pi to receive.

Node js MQTT library

Building from the existing code we created in part 1, navigate to the projects root directory and install the mqtt package into our node js app.

cd ~/google-home-project 
npm install mqtt --save

Now we can require the library and connect to our MQTT broker, which is hosting locally.

Edit app.js and add the following code below the existing require statements.

JavaScript:

const mqtt = require("mqtt");
const client  = mqtt.connect("mqtt://localhost");
// optional:
client.on("connect", function() {
  console.log("connected to broker");  
});

Adding an additional route

Next, add the following route above the app.listen line.

app.get("/client-test", (req, res) => { 
  client.publish("client/led", "test");
  res.send("Message sent"); 
});

When we receive a request for the /client-test route a message will then be sent using the client.publish method.

You can connect this with a Google Assistant command by heading over to https://ifttt.com and creating a new applet.

Select Google Assistant command using a simple phrase as the trigger, and a webhook for the service that makes a HTTP GET web request to https://<>/client-test

Once you have this in place, start your node web server and test out your voice command.

node app.js

Client 2:

ESP8266 Based WiFi Mini

Next, we’ll explore using an Arduino based board as a client and we’ll set up the variable Google Assistant command: “Hey Google, change the light to ($colour)”

We’re using a WiFi Mini ESP8266 Main Board. This is modelled on a Wemos D1 Mini, so if you are using Wemos hardware, the process should be basically the same.

We’ve also wired up an RGB LED Module.

SETTING UP THE WIFI MINI

Attach the RGB module.

For the WiFi Mini, we first need to download and install CH340 USB Driver from http://www.wch.cn/download/CH341SER_ZIP.html

The page is not in English so we used Google Chrome to translate it.

ADD BOARD SUPPORT FOR THE ESP8266 IN YOUR ARDUINO IDE

Go to File › Preferences › Additional Board Manager URLs and add: http://arduino.esp8266.com/stable/package_esp8266com_index.json

Now go to Tools › Boards › Boards Manager and install esp8266 by ESP8266 Community.

Select Wemos D1 R2 and mini from the boards list. This is compatible with the WiFi Mini ESP8266 and most incarnations of this type of WiFi board.

Finally, select the USB port under Tools › Port to allow Arduino IDE to establish connections with your hardware.

Download the code package from the online resources. The code is located in the arduino-client/client.ino sketch within our download package.

The sketch performs the following tasks:

Open client.ino in your Arduino IDE and edit with your WiFi SSID, password and your MQTT broker’s IP address.

Also, ensure you have the PubSubClient and ESP8266WiFi libraries installed then upload the sketch.

Open the serial monitor to see if everything was connected ok.

To test the device is receiving MQTT messages as expected, go back to your Pi server and run the following command:

mosquitto_pub -h localhost -t arduino/colour -m 
"blue"

The RGB module should change to blue and you should also receive the following message in the Arduino serial monitor:

Let’s now set up a voice assistant command, and one final web server route to control these from a Google Assistant device.

IFTTT

Create a new Applet and select Google Assistant for the trigger. Next select “Say a phrase with a text ingredient”.

Create the command using a $ for the variable text, in our case the colour.

Next for the service, select Webhooks › Make a web request.

URL:https://<>/colour
Method:POST
Content Type:application/json

For the body enter the following JSON object with our variable as the colour:

{ "colour": "{{TextField}}" }

PI SERVER - ADDITIONAL ROUTE

Working back on your PI web server, we’ll need an easy way to read the received JSON data. For this, we can install the body-parser library. From the projects root directory, run:

npm install body-parser --save

Then edit app.js and require the library:

const bodyParser = require("body-parser");

And before the routes add:

const jsonParser = bodyParser.json();

Now create the route using the jsonParser middleware to give us access to the JSON.

app.post("/colour", jsonParser, function (req, 
res) {
  console.log(req.body);
  if (!req.body.hasOwnProperty("colour")) {
    res.status(404).send("Not found");
    return;
  }
  switch(req.body.colour) {
    case "red":
    case "blue":
    case "green":
      client.publish("arduino/colour", 
req.body.colour);
      res.status(200).send("Colour message
 sent");
      break;
    default:
      res.status(404).send("Not found"); 
  }  
});

Since this is just a prototype, we’ve only added support for 3 colours. Anything else will send a 404 response.

With all the pieces in place, start your server and test the voice commands for each programmed colour (red, blue & green).

node app.js

AUTOMATIC START

Automatically start your Node.js server when Pi boots. We found PM2 is a great tool to achieve this.

To quote their website: "PM2 is a production process manager for Node.js applications with a built-in load balancer.

It allows you to keep applications alive forever, to reload them without downtime and to facilitate common system admin tasks."

To install:

sudo npm install -g pm2

Start our server using PM2:

pm2 start app.js

If pm2 is not found, you may have to create a symlink. Since we had a manual installation of nodejs, for our version it was:

sudo ln -s /usr/local/node-v8.11.3-linux-armv6l/bin/pm2 /usr/bin/pm2

Then we set it to start when the Pi Boots.

pm2 save
pm2 startup

Finally, copy and paste the output from the previous command, which will look something like this:

sudo env PATH=$PATH:/usr/local/node-v8.11.3-linux-armv6l/bin /usr/local/node-v8.11.3-linux-armv6l/lib/node_modules/pm2/bin/pm2 startup systemd -u pi --hp /home/pi

WHERE TO FROM HERE?

What we have just demonstrated, is how easy it is to utilise one wormhole enabled Pi to drive many more simpler devices.

It also means that you could implement LoRa-connected nodes, while still having the wormhole taken care of with the hub Pi.

Having an Arduino Web Server, for instance, would be much more difficult to get working without directly exposing ports on your router and other similar security risks.

This option of a hub and spoke configuration, also adds versatility to what devices you can run. If you only want to turn an LED on or off, that's probably something of a waste of hardware.

It's worth noting that we did intend to setup two-way communication in this project, whereby we could use Google Home to check the temperature of a remote node, for instance. This would have invaluable potential for remote monitoring.

Imagine; "Hey Google, read through the current temps", and it proceeds to read out all the current temperatures from all of your satellite nodes.

However we struggled to get this to work as we had hoped. On further investigation, it appears to be a security restriction (which does make sense), that Google Assistant can only push commands, not pull them (even in response to a request).

We'll keep exploring this however. Just because it's not meant to work that way, doesn't mean we can't get it to work that way!

Part One