In this issue, we discuss the pump controller and remote switch of our LoRaWAN pump control system.
BUILD TIME: A few days
DIFFICULTY RATING: Advanced
A Note from the Editor
This project was developed by one of our readers, Ashley Woods from regional Queensland. Like most of DIYODE readers, Ashley is a hobbyist who started out dabbling with an Arduino and graduated to other microcontrollers and on to the Raspberry Pi. Along the way, he discovered the fantastic MQTT protocol and has now used it in several projects. We are impressed with Ashley’s project and are really excited to share it with you.
Keep in mind, even though Ashley’s project has been designed for operating multiple water pumps on a farm, you could adapt the project to suit your own needs where long distance monitoring is required. It could be, say, to operate a gate at the end of your long driveway or operating your outdoor hydroponics garden, for example. You can choose to just build the sensor, and/or the pump control system.
Originally, we were going to present this as a full project, however, it was too large to squeeze into one issue. Last month, we presented the water level sensor circuitry. This month, we present the pump controller and remote switch circuits.
PART 1 RECAP
Let's briefly recap what we presented in Part 1.
Rather than a single project, this is actually a system where several modules all communicate via MQTT to monitor the water level in a tank and to control a pump to keep the tank full. The pump controller also integrates with a solar inverter to ensure the pump runs when electricity is available from the solar array.
Last month, we discussed the LoRaWAN technology and showed you how we built our tank water level sensor, which included the electronics and the pressure sensor that sits at the bottom of the water tank.
We also covered how we send the data to The Things Network, how we logged the data with InfluxDB, and how to present the information visually using Grafana.
The Build - Part 2:
Pump Controller and Remote Switch
In Part 2 of this project, we will discuss the operation of the pump controller and the remote switch. The pump controller is really the most sophisticated piece of hardware and software in the entire system as it needs to manage many interfaces and process numerous rules. We will also reflect on what we have learnt from this project and discuss the planned improvements.
As a recap, we are working through the design, assembly and operation of our tank level sensing and pump control system. In Build 1, we discussed the tank top sensor, the connection to The Things Network and the InfluxDB and Grafana dashboard. In Part 2, we delve into the pump controller itself and the remote switch.
Remember, as we noted in our introduction, these modules can be built and operated independently; there is no need to build the full gamut of modules. If your project just needs water level monitoring, or just needs a way to control a remote electrical appliance, then select the modules you need and operate them standalone.
The Pump Controller
This module is really the brains behind the full system. It uses status information from the water level sensor as well as from the solar inverter to decide when to turn the pump on and off. Our goal is to ensure the tank has a good store of water and to replenish it when it falls below a specific threshold, but preferably when the solar array is generating excess power. Rather than feeding power back to the grid at about 8c per kWh and consuming power from the grid at 28c per kWh, we prefer to self-consume by opportunistically running the pump when the sun shines.
The photograph above shows the Pump Controller installed in situ. A steel enclosure is used to house the electronics. This includes the Raspberry Pi, custom PCB, network switch, and battery. It also includes a power injector for the future wireless bridge link.
To maximise our self-consumption, we take advantage of the digital I/O provided on the Fronius solar inverter. The inverter offers four digital outputs which supply a 12VDC signal based on configuration within the inverter itself. While this is not a “how to” on inverter setup, the inverter offers these four configurable outputs which can be individually configured to activate when a specified excess capacity is reached. Put simply, the more sunshine, the more digital outputs turn on. These inverter outputs source 12VDC and are designed to operate a relay to turn on a pool pump, electric hot water system, or other discretionary loads. In our case, we feed the Fronius digital signals via an optocoupler into our Raspberry Pi GPIO. So now the Raspberry Pi running the Pump Controller knows when we have excess power capacity, and thanks to the MQTT messages from the water level sensor, it also knows how much water is in the tank.
Logic rules (in the form of if .. elif..else) then pull this detail together to decide if the pump needs to run. The Pump Controller has three modes of operation as follows.
- Manual mode, where the controller ignores these additional inputs and we control the pump purely via a MQTT message.
- Semi-automatic mode, where the pump is started manually but gets stopped when the tank is full, and
- Automatic mode, where the pump gets started and stopped based on the inverter signal and the tank water level.
The pump controller can be switched between these modes via a MQTT command. Automatic mode is where the fun happens so let’s explore the logic behind this operating mode.
Our tank has two thresholds that determine if the pump needs to run or not. These are configurable and we have set them to “low” at 30% and “high” at 95%. The logic operates as follows.
- If the tank is below the low threshold (critical), run the pump regardless of the availability of solar power (so the pump can run at night).
- If the tank level is below high but above critical, then run the pump when we have excess solar generation.
- When the tank is above high then turn the pump off.
- We also monitor the “freshness” of the last water level reading received via MQTT. If the reading is too old, then it is considered “stale” and untrusted and the pump is simply turned off. This prevents a stale water level causing the pump to continue running.
These checks are only performed every fifteen minutes so the pump may not stop right at the 95% mark, which is why “high” is less than 100%.
The fun part is the interface to the pump itself. As this interface deals with 415 Volt three phase power, this setup has to be done by a qualified electrician. We used our regular electrician to install a collection of relays to use as sense relays and control relays. The relays are mounted in a separate physical enclosure to the Raspberry Pi and associated low voltage electronics.
The annotated photograph here shows the relay enclosure with the relays mounted on a DIN rail.
This pump uses three phase power so we decided to install a “phase present” relay on each phase. Each of these 240VAC relays is energised by the presence of the respective AC phase (with respect to neutral). The Raspberry Pi simply monitors the Normally Open contacts in each relay to ensure the phase is always present. This monitoring is done on the supply side of the circuit immediately after the circuit breakers. When a phase is lost the Pi will report a status change. This is done because three phase power can be unreliable in rural areas and sometimes one phase may be lost. We want to ensure the pump does not try to run or start on anything less than all three phases.
When the pump is running, we monitor one of the phases going to the pump and assume the presence of this phase into the pump means the pump is running. The existing solenoid operated manual switch also has a “pump fault” circuit which we monitor in the same way.
To start and stop the pump we use a pair of 12VDC relays (K1 and K2 in the photo) wired across the buttons in the existing solenoid switch. These relays are pulsed by the Pi for about 700 mS to simulate the pressing of the manual “pump start” or “pump stop” buttons.
We used a Raspberry Pi 4 with 2GB of RAM, but this could easily be run on a much less powerful version of the Pi. All the code is written in Python and uses the GPIOZERO library as this really simplifies GPIO interrupts. Any time a status change occurs, a MQTT status message is published. A status change could be a pump stop or start, a Fronius IO change, a tamper switch triggered (although the switch is not yet installed), or a temperature alarm.
The Pi also subscribes to command messages which can change the operating mode (manual, semi-automatic, or automatic) of the controller or start or stop the pump. Obviously, security and authentication are important and the authority of the message publisher is checked before accepting a message. This publish/subscribe model decouples the commands from the subsequent status changes. Any command received will first be acknowledged and if appropriate, acted upon. The consequent status change (such as the pump starting) will generate a status change which triggers a message publish.
The Raspberry Pi is battery backed up using a small Sealed Lead Acid (SLA) battery inside the enclosure. This is done to ensure the controller can operate independent of the mains supply. It is tricky to detect a power failure when the failure powers down the Pi! The battery is charged from the mains and the battery supply is used to power the Pi, the stop/start relays and the included five port Netgear network switch. This provides a rudimentary UPS for the system. The network switch is powered from the battery as it provides the network connectivity between the Pi and the solar inverter and the Internet service. The internet is provided by a DLink cellular router. This is currently mains powered but will move to the “UPS” in a future design.
As the 12V supply is responsible for powering everything (including the Netgear network switch) it is important that this supply be maintained at a consistent voltage. To achieve this, the important “peripherals” are powered through a buck/boost converter to maintain the consistent 12V supply. This is particularly important when running on battery and the battery begins to discharge.
The supply and battery voltages are monitored by a couple of INA219 power monitor ICs. Rather than use plug in breakout modules for these, we decided to integrate the components into our PCB and have a go at surface mounting the ICs and associated components. This was done using a conventional soldering iron with a small tip and proved to be easier than expected. This monitoring could have been done using a separate shunt resistor and analogue to digital converter as we did in the tank top sensor but the INA219 elegantly bundles this functionality together and presents an I2C interface.
The Raspberry Pi is set up to boot off a 500 GB Samsung SSD rather than off the traditional SD Card. This provides more storage as well as the improved reliability for the Pi. The Pi is also used as the FTP server for the daily performance reports from the Fronius inverter. Nothing is done with these CSV files just yet, but in the future, we plan to add this data to InfluxDB and report via Grafana.
A very detailed log is also written by the Python code. This uses the Python logging library with a new log being written each day and logs retained for 30 days. All this logging activity is probably better supported on an SSD rather than on the usual SD card. This log has been used during testing to confirm the pump start and stop actions are following the intended rules.
The Raspberry Pi CPU temperature, enclosure temperature and humidity, battery voltage, and charging are all monitored and reported in triggered and scheduled periodic status updates. To simplify the management of many status variables, a single large Python dictionary is used. This is updated as status changes occur. When a status report is requested or published based on the hourly schedule or a status change, the entire dictionary is simply sent as JSON in the MQTT message payload.
Local alarm conditions are raised if environmental readings exceed locally defined preset levels. The threshold on the Pi4 CPU temperature had to be raised during summer as the temperature exceeded the initial modest settings. An alarm condition is also set if periodic messages are not received from the tank level sensor.
| PARTS REQUIRED FOR PUMP CONTROLLER/SOLAR MONITOR: | JAYCAR | ALTRONICS | PAKRONICS |
|---|---|---|---|
| 1 x Raspberry Pi 4 2G or equivalent | XC9100 | Z6302G | PAKR-A0288 |
| 1 x Network Switch (Ideally 12VDC powered) | We used a Netgear GS105 | ||
| 1 x 12V 7Ah SLA Battery | SB2486 | S4538 / S4539 | - |
| 1 x Cellular Router | We used a DLink Cellular Router | ||
| 1 x S18V20F12 Buck/Boost Converter | pololu.com: 2577 | ||
| 1 x SSD HDD (500 GB Samsung or similar) | See text | ||
| 3 x INA219 Power Monitor ICs and associated shunt resistors & bypass caps | See text | ||
| 1 x 19V Power Supply (laptop supply, or similar) | See text | ||
| 4 x PC817 Optocoupler ICs | Element 14: 1244544 | ||
| 1 x TSR 1-2433 TRACO 3.3V Power Regulator | Element 14: 1696319 | ||
| 1 x D36V28F5 Polulu 5V Regulator | pololu.com: 3782 |
Note: Various generic resistors, diodes, LED and PCB headers required. Refer to schematic, photos and PCB overlay.
Mounting hardware, plugs, sockets, wiring and cables also required.
Remember that these messages are generated about every hour by the LoPy4 module on top of the tank and need to flow from the tank top microcontroller through our local LoRaWAN gateway via The Things Network, through our cloud server, and be published via MQTT.
If any device in this chain fails, then the pump controller will cease receiving level updates. Once the tank level reading becomes “stale” in the pump controller, the pump will be turned off. This will happen if a message does not arrive within 9000 seconds (2.5 hours) of the last message.
Alarm conditions (including the “level sensor offline” alarm) are published as part of the JSON payload in the status message.
Power to the entire module is supplied by a repurposed laptop power supply. This provides 19VDC with plenty of power capability.
The Pump Controller Build
The pump controller follows similar thinking to the tank level sensor and the remote switch in that it uses a custom PCB (also designed using KiCad) with plug in breakout modules for voltage regulators, charge modules and temperature and humidity sensors. As noted earlier the power monitoring uses the INA219 power monitor ICs soldered directly to the PCB.
The 40 pin connector on the PCB matches the 40 pin GPIO connector on the Pi – but not all connections are used. This PCB also offers test points and several LEDs to show voltage and status.
The interface to the Fronius solar inverter uses optocouplers to decouple the 12VDC supply from the inverter from the pump controller. We set up these digital outputs from the inverter so that they would activate when power was being returned to the grid. We set the threshold on #1 at 1300 watts as this about the power required by the pump.
The Fronius inverter has a web interface to configure these outputs. It offers various rules around hysteresis, and minimum and maximum run times. We disabled these and just presented the simple IO to the Raspberry Pi so we could make the pump turn on and turn off decisions ourselves.
The entire system is installed in a weatherproof steel enclosure to provide security and protection. Although the enclosure is installed on a switchboard panel in a shed, dust contamination is still an issue, so protection is necessary.
The electronics is mounted on a shelf in the enclosure. This “toolless install” shelf can be easily removed to mount or service the electronics. Unfortunately, this steel enclosure shields the cellular signal so the modem must be located outside the enclosure.
We are looking at alternative options for Internet connectivity to resolve this issue. One option is a point-to-point Wi-Fi bridge to the house where the Internet service is located. Another option is a simple external antenna on the router.
An external LED indicator PCB has also been designed. This uses a MCP23017 (I2C 16 input/output port expander) to drive a number of dedicated LEDs. This module is yet to be assembled but when completed, will be mounted in a separate enclosure with a transparent lid so the LEDs can be seen externally. As these LEDs are all software driven, the pump controller code will be modified to drive these LEDs so they reflect the IO status.
LEDs are assigned for the Fronius digital IO, AC phases, and pump and fault status. As the status information which controls these LEDs is also published, this module could be connected to a controller (such as a Raspberry Pi) and located anywhere with network access.
We are currently working with Home Assistant to develop a software version of this same interface. The screenshot top of page shows the Home Assistant panel. Note the fact that the pump is running when we have no sunshine because the water level is below 30%.
The Remote Switch Module
The remote switch module is an optional module in the overall pump control system. It is intended to be installed at a location that is separate from the manual pump control switch. The manual pump control switch must be installed close by the pump to keep the power as close to the pump as possible. This is an inconvenient location some distance away from any residence.
The remote switch module solves this distance issue by allowing the pump to be turned on or off from any location which has network access. The remote switch is intended to mimic the Start and Stop functions on the manual switch. The following photographs show the manual switch and the remote switch for comparison.
| PARTS REQUIRED FOR REMOTE SWITCH MODULE: | Jaycar | Altronics | Pakronics |
|---|---|---|---|
| 1 x Raspberry Pi Zero W with microSD card | - | - | - |
| 6 x PN2222A Transistors | ZT2298 | - | - |
| 1 x DPDT R/A Toggle Switch | ST0365 | S1355 | - |
| 1 x SHT31 Temp/Humidty Module | - | - | - |
| 1 x Red 30mm Arcade Button | Adafruit 3489 | ||
| 1 x Green 30mm Arcade Button | Adafruit 3487 | ||
| 3 x 16mm illuminated pushbutton white momentary | Adafruit 1479 | ||
| 1 x 16mm illuminated pushbutton red momentary | Adafruit 1439 | ||
| 1 x Traco Power 1-2450 5V regulator | RS Components TSR_1-2450 |
Note: Various generic resistors, diodes, LED and PCB headers required. Refer to schematic, photos and PCB overlay.
The remote switch is actually little more than an “Internet Button” on steroids, but rather than just one button, we have six buttons. Each button on the Remote Switch Module contains a coloured LED which is controlled independent to the button switch. The remote switch publishes messages on the “command” topic and subscribes to messages on the “status” topic.
In general, a button press will trigger the publish of an MQTT message on the “command” topic. The pump controller subscribes to these messages and will act on the received command. If the action is permitted and succeeds, a status change will occur at the pump controller which will trigger a MQTT status update message. The Remote Switch Module subscribes to and receives this status update message and sets the status LEDs accordingly.
The remote switch holds a copy of all status information in the pump controller. This is the simple Python dictionary discussed earlier - which is published by the pump controller as the status update message.
For example, pressing the “Start” button will examine the local copy of the status dictionary and if conditions are suitable for a pump start, a start message will be issued. Once issued, the Start LED will flash slowly pending the receipt of the status change.
The start message will again be checked at the pump controller and assuming all conditions (such as power) are satisfied, the pump will be started. This will be detected as a status change at the pump controller which will trigger a status update message. This will be received by the Remote Switch Module which will cause the LED in the Start button to turn solid green.
Before any command is sent to the controller the local copy of the pump controller status is checked to ensure the validity of the command. For example, a start command will not be sent if the pump is already running. If such an action is attempted the button will flash rapidly several times to indicate an inappropriate action.
This Python script (like all the other scripts) runs as a service under the control of system.
The remote switch module is built as follows.
- It is powered by a Raspberry Pi Zero W. This is seated on a Raspberry Pi “Mat” board. We have coined this term to describe a custom PCB which sits underneath the Raspberry Pi – as opposed to the traditional Pi “Hat” which sits above the Raspberry Pi. Maybe this term will catch on?
- While this construction uses a Pi Zero W, any Raspberry Pi with the standard 40 pin GPIO connector may be used. Other form factors of the Raspberry Pi will require a cable to connect the Pi to the Mat Interface board.
- The custom PCB provides power to the Raspberry Pi. Other models of Raspberry Pi may require a higher current voltage regulator. Alternatively, the Raspberry Pi may be powered via the usual USB connector.
- All connections to the console, buttons and LEDs are made to the custom PCB.
- The interface board also uses a local temperature and humidity sensor to monitor the enclosure environment.
- The software runs on the Raspberry Pi as a Python3 script. It uses the GPIOZERO library for button and LED control.
- This implementation uses Wi-Fi for network connectivity. Other network transports such as Ethernet or Wireless WAN may be used providing the Raspberry Pi supports these.
- The remote switch module offers dedicated buttons which can control the run mode of the controller. This allows the controller to be switched between Manual Mode, Semi-Automatic Mode and Automatic Mode.
- It includes a very basic Alarm light which illuminates if an alarm or fault condition exists on the pump controller or the pump. The alarm LED will flash at different rates or turn on solid for different severity alarms.
The following photograph shows the Remote Switch PCB attached to the 3D printed mounting board ready for installation in the enclosure.
All communication between the Pump Controller and the Remote Switch Module is via MQTT. This architecture allows for multiple remote switch modules if desired. In this case, all will receive the same status, and any may initiate an action.
As with the Pump Controller module, the Remote Switch Module offers a serial console port. This connects to the Raspberry Pi UART. Any USB to serial adapter may be used to connect to the console. This should use 3.3 Volts only to suit the Raspberry Pi. The raspi-config must enable UART for the console and I2C (for the environment sensor).
Upon startup of the Python script, the Remote Switch Module will do the following.
- Load the configuration from the YAML configuration file.
- Sequence all the LEDs to test them.
- Connect to the broker with the LWT message and subscribe to the relevant MQTT topics.
- Publish a status update request. Once this is responded to, the LEDs will be set accordingly.
- The scheduled jobs will be started.
- One of the scheduled jobs checks the validity of the status mimic. If it is found to be out of date an alarm condition will be raised.
- The system will then respond to any button presses. If a button press is permitted and logical, the appropriate JSON command message will be published and the button will flash slowly. If the button press is illogical or not possible a rapid flashing of the button will occur.
- Local environment checks will be performed on the one minute schedule. Conditions outside of the threshold will raise an alarm condition.
As with the other Python scripts and the Pump Controller itself, we do the MQTT Connect with the optional LWT message. This means the broker will issue a message on behalf of the client if an unexpected disconnect occurs. We have found this a really useful MQTT feature, particularly as we have a “LWT Monitor” script which monitors for these messages and sends a Telegram alert when one is received.
Alarm conditions will all cause the Alarm LED to be illuminated in some way; ranging from a repeated brief flash to a solid on condition. In general, the higher priority alarms will cause the Alarm LED to have a longer on time than off time.
The Alarm button may be used to attempt to clear any alarm conditions. When pressed the following occurs.
- All LEDs are extinguished.
- A “status update” command is published.
- The Alarm LED begins flashing. Until a response (“status update”) is received the system remains in this condition. This may be as little as sub-second if the controller is online, or much longer if the controller is offline and requires intervention.
- When a response is received, the status update will be used to set the LEDs to reflect the state of the pump controller.
The Alarm button may be pressed at any time to prompt such a refresh. Once any Pump Controller alarm has been cleared, the Remote Switch Module will resume normal operation.
Power to the remote switch module is provided by a DC plug pack and a 2.1mm barrel connector. This Pi Zero W does not use any battery backup and boots from the conventional SD Card.
The Evolution of MQTT Message Formats
Like most projects, the goals and capabilities evolved during the build. There were numerous “wouldn’t it be great if we did this” moments; some of which required changes to the MQTT message formats and a couple which needed hardware changes.
The original MQTT message format evolved as requirements grew and one of the key changes was to consolidate different messages into just command messages and status messages. The former would instruct a device to change some state (possibly to turn the pump on or off) and status messages would be published when a status changes. This decouples the response from the command.
While it removes the expected feedback from the command, it works well in a multi-client system as every client gets notified of a status change, regardless of what triggered the change. For example, if the pump is started manually, the light on the remote switch module changes from red to green.
If the controller operation is changed from manual to automatic by a MQTT message from the mobile, then again, all devices learn about the change.
The various command messages evolved to simple numeric codes where “0” instructs the receiver to issue a status message, “1” may start the pump, and “8” may activate manual mode.
One of the really useful features of MQTT is the Last Will and Testament (LWT) messages. These are issued by the broker on behalf of a client when an unexpected disconnect occurs. We implemented LWT on all important clients and then wrote a “LWT Monitor” which simply subscribes to LWT messages for nominated devices and sends a Telegram alert if or when something fails.
This is particularly useful to help ensure all devices and scripts continue to run. Python scripts on the Pi and on the server are run as services under systemd, so these generally restart should they fail or should the Pi be restarted.
Wrap Up
This whole system has evolved from what was intended to be a simple Internet connected remote pump switch to a rather larger integrated system including the water level sensor, dashboard and monitoring scripts to send a Telegram message when something fails. We had a couple of issues along the way with the Generation 1 water level sensor. These included the battery life, corrosion and ultrasonic sensor not providing reliable data. Another issue was setting out to build the pump controller with a microcontroller rather than a more powerful micro-computer such as the Raspberry Pi.
Using a microcontroller to control the pump made excellent sense at the beginning of the project and we built a working system based on the Particle Electron cellular microcontroller. It used a few thousand lines of C code managing MQTT and the required digital IO to the relays as well as a rather basic command line interpreter for configuring and querying operation of the system.
Once we decided to add the solar inverter into the mix, it became obvious that more computing power was needed so we could also host the FTP server. This meant rewriting the code from scratch - this time in Python. The Python code does much of what we developed in C, but in about 40% of the lines of code.
Like many hobby or professional projects, we experienced “scope creep” and continued to have more of those “what about if we did this” moments. At the end of the project – well the end for now – we have a very functional system which does significantly more than originally intended. Along the way, we wondered if anybody had done this as a commercial product. While we located a couple of suppliers who have remote water level sensing, we found none that have the level of customisation or integration that we have ended up developing.
This project has also allowed us to embrace and deal with issues such as reliability and resiliency. How do you design and build something where it just needs to keep working? This reliability and resilience were always front of mind and really drove our decision to build our own PCB rather than use some form of prototyping board. It makes one appreciate the engineering expertise that must go into a Mars Rover.
We have also had the chance to touch a huge range of technology from schematic and PCB design, digging through datasheets to find a better sensor technology, designing a messaging and communications system, setting up the Mosquitto broker, the Things Network gateway, real time databases and graphing tools, 3DCad and 3D printing and the REST API used by the Telegram messaging system. But we do not stop here.
Sourcing parts to build these projects has taken a significant effort as we wanted to achieve a "production" appearance rather than a "prototype" appearance. We wanted to use proper circuit boards, professional connectors, wiring harnesses and robust mounting and enclosures. To achieve this our parts were sourced from many suppliers.
The PyCom microcontrollers, Raspberry Pi modules and various PCB headers came from Core Electronics. Specialty items such as the SMD components, DIN Rail relays and MOLEX connectors came from RS Components. Jaycar has a great selection of polycarbonate enclosures and batteries as well as the generic resistors, diodes and LEDs and braided tubing to make any wiring harness look like a factory made product.
The pressure sensors came from eBay. The PCB standoffs, mounting screws and hardware came mostly from Banggood with test points and buttons and some power and charge modules from Ali Express. We used JLC PCB for our PCB fabrication, but chose to assemble the boards ourselves. Let's also not forget the old laptop power supply. So while the parts lists shown in the text identify the major components for each module, the miscellaneous hardware for mounting and connectors may be tweaked by makers to meet individual needs.
Next comes the Bluetooth console on the various modules and the improved integration with Home Assistant so we can present a more elaborate dashboard showing all the status details – along with a touch screen to control it all. So, stay tuned for a future update to this project.