Take your Grippy Robot Arm to the next level by making it remote controlled using your Smartphone.

BUILD TIME: 2 HOURs

DIFFICULTY RATING: Intermediate

Last month, we showed you how you can build your very own 3D printable robotic arm, affectionately called Grippy. This month, we are going to show you how you can quickly and easily create a mobile phone-based controller for your Grippy using your mobile phone, an ESP32 development board and the Blynk app.

THE BROAD OVERVIEW

We started this project with one simple goal, we wanted to be able to control Grippy using a mobile phone. Therefore, we needed a control system that worked with both Android and iOS devices. This quickly presents a problem. Whilst there are many applications to quickly and easily build apps to work on Android devices, the same cannot be said for iOS. The Apple OS is a closed system and presented a significant challenge. After some research, we came across a solution to solve this issue when we learned about Blynk.

Blynk is an internet-of-things centred application builder that works with both Android and iOS devices. It came to life in 2015 after a very successful Kickstarter program where it raised 5 times its initial goal. Blynk allows a maker to quickly and easily build applications that interface with many popular microcontroller development platforms from Arduino, Raspberry Pi, and even the ESP32. It’s remarkably simple to use due to its drag-and-drop use of modules, and means you have very minimal coding to do. All the heavy lifting is done inside the very clever app.

HOW IT WORKS

You start by first creating an application using the Blynk app on your mobile phone. Here you can add all of the usual application graphical user interface (GUI) assets like buttons, joysticks, and displays, etc. These assets represent user input devices. The Blynk app then sends the information to your microcontroller via the internet, Bluetooth or WiFi, etc. You can program, for example, a button on the GUI to directly control a GPIO pin or, in our case, a slider. The sider will directly report the value to a register/memory location.

The Blynk application provides an authentication code, which links the Blynk app on your phone, to the code running on the microcontroller. The microcontroller, of course, needs to be connected to the internet.

For our project, we opted to use the ESP32 because it comes from the factory with WiFi capabilities.

All we need to do is program the ESP32 to connect to our WiFi router using the SSID and password, and then receive the incoming signals from the Blynk app. We then simply take the incoming value and write that value to the servo position.

For this project, we are going to show you how to set up and program the ESP32 using the Arduino environment, and how to build an app using Blynk.

We will also provide a link, so you can just copy our application if you just want to get it working.

The Build:

^ The standoffs are not needed if you have built the Grippy Robot Arm already.

This is a very simple build because most of the electronics come ready to go on the ESP32 development board. All we need to do is create a PCB that will allow us to make the necessary connections, and add a couple of everyday components. Since no I/O shield seems to exist for the ESP32 development board we need to use perfboard.

The first step is mounting the perfboard to the Grippy base. The base was designed to mount the Arduino Nano I/O shield from DFRobot, which has the same mounting holes as the Arduino UNO development board. We could modify the existing design to fit the new perfboard or we could simply modify the PCB to fit the existing mounting holes. We went with the latter and simply used another PCB with the same mounting holes to mark out the locations on the perfboard.

After marking the mounting holes, we just drilled four 3mm holes in the perfboard that match the mounting screws.

We cut two strips of female headers to allow for 19 pins each. This is where we will socket the ESP32 to the board, so that it can be removed and re-used for other projects, modified, etc. If you’re making a permanent Grippy you can exclude this step if you like.

We suggest you insert the female pin header onto the ESP32 pins prior to cutting. This will help ensure the break is across the correct pin and save on wastage.

We then soldered the ESP32 and the attached female pin headers to the perfboard. Again, leave the headers attached at this point, as it will aid in aligning the pins into the perforated holes and ensure you don’t solder the headers in the wrong place.

After soldering the ESP32 and headers down, remove the ESP32 from the newly created socket. Put it aside so it can’t get damaged in the soldering process.

Run four wires from the G2, G4, G5 and G18 pins, which are the 5th, 7th, 10th and 11th holes on the right side of the ESP32 (when the USB socket is facing towards you).

These wires are the signal wires for the servos and need to go to a 4-pin male pin header on the left side of the board. We need to solder these headers down so that they align with the servo connector.

The photo above from Adafruit shows the servo connector and wiring. The brown wire is the ground connection, the red wire is the 5V connection and the yellow wire is the signal wire. We made a connector for the four servos by using 3 male pin headers cut 4 wide. These were then soldered to the board as shown above. The four 5V pins were soldered together and connected to the 5V input screw terminal, with a 1N4004 diode in series. This diode will prevent the destruction of your servos if the power supply is accidentally connected in reverse polarity.

We also added a 220μF capacitor close to the servo pins. This helps to reduce voltage sag when the servos are moving, if your power supply can’t keep up with the rapid changes in current. If you find your servos are behaving erratically, especially when the servo is first starting to move, try increasing this value to 470μF. After making all of the connections, the underside of your perfboard should look something like this.

Before connecting 5V to the device, we strongly encourage you to test the connections with a multimeter set in continuity mode. This mode is usually found in the resistance setting and has the speaker/beeper symbol that looks a lot like a sideways WiFi symbol. See the image below of our meter set to continuity mode.

On our meter, we needed to select the resistance setting and press the mode button until the continuity symbol illuminated. In this setting, anytime there is a physical and electrical connection between the two probes, the multimeter will beep. To verify all of the connections were correct, we simply put one probe on one side of the connection and the other where we wanted that connection to go. If the meter beeps we knew the connection was good.

So with the ESP32 in the socket, we would connect one probe to a used I/O pin and then press the other to the signal pin for the respective servo. Making sure it beeped indicating an electrical connection to that pin alone.

It is also a good idea to verify which way the screw terminal block is connected. Press one probe to one screw terminal and the other to ground. If it beeps, that is your ground connection, if it does not beep, attach the probe to the anode of the diode. It should then beep, showing that the terminal is positive. Doing this will ensure that all your connections are correct, and should mean you’re good to attach 5V to your servos.

Note that the screw terminals will only power the servos. You still need to power your ESP32 via USB.

With the hardware out of the way, let’s focus on the software side. Our first task is to setup the Arduino IDE.

ARDUINO IDE SETUP

The very first thing we need to do is install the latest Arduino IDE. If you have already installed the Arduino you can skip ahead to the ESP32 support in the Arduino IDE section. If you don’t have the Arduino IDE installed, or an old version, you can find the latest version at the following link: https://www.arduino.cc

You need to mouse over the software button and select downloads.

This will bring up an option to download the installer, select this option. In our example, we are installing on a PC running Windows.

This will open a window where you can either download without donating or donate and download.

After selecting your choice, you will be prompted to save the Arduino install file. Click ‘Save File’ and wait for it to download.

Once it has downloaded, you want to install the program and allow it to download the required drivers. Select the program you downloaded. After opening the file, you will see the following License Agreement. Read or skip through it and select the ‘I agree’ button.

This will bring up the following screen. Make sure the same checkboxes are checked on your install and select the next button.

The next screen asks where you want to install the Arduino IDE. For simplicity, we will keep it in the standard location.

You will receive three prompts during the installation process. These prompts will ask if you want to install other required software. Select install for all of them. Once the installation completes you will see the following screen. Simply select the close button.

Once successfully installed, you will see the following screen, which means you are ready to start preparing the Arduino IDE to work with the ESP32.

ESP32 SUPPORT IN THE ARDUINO IDE

Natively, the Arduino IDE does not support the ESP32 development boards or their variants. Therefore, we need to add this functionality to the IDE. However, it’s very possible that your computer will need to have drivers for the ESP32 Universal Asynchronous Receiver/Transmitter (commonly called a UART). This is the IC that allows your computer to communicate with the ESP32, therefore, you should first download the drivers from: https://www.silabs.com/products/development-tools/software/usb-to-uart-bridge-vcp-drivers

After installing the drivers that tell the PC how to communicate with the new microcontroller, we need to install some software that tells the Arduino IDE how to use the new ESP32 hardware. Therefore, we need to add the third-party Espressif package to the Arduino boards manager. To do this, you need to select ‘File’ and ‘Preferences’ in the Arduino IDE.

This will bring up the following screen.

Under ‘Additional Boards’ manager URLs: you want to add the following: https://dl.espressif.com/dl/package_esp32_index.json

Note: If your IDE already has additional boards added you need to separate the previous listed URL’s from the new URL by using a comma, at the end of the previously added URLs.

Once you have added the Espressif board manager URL select OK. You then want to go to ‘Tools Board’ and ‘Boards Manager’ as shown below.

After selecting ‘Boards Manager’, you will be greeted with a list of available boards. You want to scroll through the list until you find the ESP32 by Espressif systems. Once you find it select the install button. This will download and install the ESP32 board functionality to the Arduino IDE.

Now, if you go to ‘Tools’, ‘Board’ and scroll through the list of available boards, you will see the ESP32Dev Module. Select this board and it will select the ESP32 development board as your chosen microcontroller.

If done correctly, when you select ‘Tools’ you should see the following options.

After successfully installing the ESP32 development module, we want to add Blynk support to the Arduino IDE.

Originally, we started creating the app using the MIT App builder platform. This is a very simple way to build very unique mobile phone apps. However, this app only works on Android mobile phones which would, of course, potentially exclude our readers in the Apple camp. After a little research, we discovered an IoT system called Blynk.

BLYNK SUPPORT IN THE ARDUINO IDE

To add Blynk support to the Arduino IDE, all we need to do is install the BLYNK library. This is easily done using the Arduino IDE library manager. Simply navigate to sketch, include library, manage libraries as shown below.

Once the IDE has populated the list of current libraries, select the search bar at the top and type “blynk”. This will search through the available Arduino libraries and allow you to install them. Once it has finished searching select install on the blynk library.

BUILDING THE BLYNK APP

With all the preparation out of the way, it’s time to get building. You need to download and install the Blynk app onto your phone via the Play Store for Android or the iOS App Store.

Once it has installed, open the app and either log in or create a new account. After creating an account or logging in you will see the following screen.

Select the ‘New Project’ button, give your project a name, and select the device. We will call ours ‘Grippy’ and we need to select ESP32 Dev Board, since that is the microcontroller we are using. You can also select the theme, which we left as dark. When you’re done select the ‘Create’ button. This will send an Auth token to your email address which you will use later.

You will see a blank grid. This is your app-building area. The modules that you select will snap to this grid.

To see the available modules swipe left on the screen. You will see a list of modules that you can drag-and-drop into the app building area. These modules or “Widgets” all represent inputs and outputs, that can be either directly mapped to the I/O on the microcontroller, or directly affect a variable for use in your Arduino sketch.

You will notice you have an “energy balance”. This is the amount of “credits” you have available to effectively buy widgets. You can purchase more of these credits from Blynk, however, we will have enough to make a controller for Grippy, so there is no need to buy anything for this project.

For us to control Grippy, we are going to need to substitute the four potentiometers we used on the original design with the equivalent component in Blynk. The slider widget was the most ideal substitute.

The slider provides a numerical output value between two defined points dependent on the position of the slider. This is very much like the potentiometers we used in the original control scheme, which produced a voltage difference between 5V and GND dependent on the position of the potentiometer.

To add a slider widget to the build area, simply drag it across. Once it is in your app area resize it so it is across the top as shown here.

After adding the slider widget, we need to define its characteristics to make it suitable for our project. To do this simply tap on the slider and it will bring up a bunch of customizable options as shown below.

We can name the slider by tapping “slider”. We are going to call this slider “Slew Servo” as this slider will control the position of Grippy’s slew or rotation.

Next, we can change the colour of the slider by tapping the green dot. This will allow us to select from a few different colours and select the intensity. For the Slew servo, we will change it from green to red.

Now we need to change the pin that the slider is attached to. We can’t directly control the servo from a pin in this fashion as we need a specific Pulse Width Modulated signal at a specific frequency.

It's best to store this value as a variable in the microcontroller's program to be used later. Therefore, we want to select a virtual pin and not a physical one. To do this tap on the word PIN and select V0 for virtual pin 0.

We also want to define the value we send, since Grippy uses 90° servos we can have this app send a value between 0° and 90°. This will mean we don’t need to translate the data in the Microcontroller sketch. To define the maximum output, you simply tap the 1023 value and change that to 90.

The next option to change is the decimal value. Since we are only dealing with an integer value make sure that decimal value is just #.

The last two options are send on release and show value, both should both be set to on.

When you’re done it should look the same as the image above.

Since we have a total of 4 servos to control we need to do this another 3 times to control the other three servos. Boom 1, Boom 2 and the Claw. You can make the name and colours whatever you like, but you need to have them set to the following pin designation.

Slew = V0

Boom 1 = V1

Boom 2 = V2

Claw = V3

When you’re done your app should look something like this.

We want to add a button to return all servos to a pre-defined position or a home position etc. Look through the widgets and pick the button.

Like the other widgets, this button can be customized by tapping it. Let’s name it home and assign it to virtual pin V4. We can also give it an “icon” by selecting an emoji from the keyboard.

Tap the off-field and add the emoji of your choice. When you’re done it will look like this:

Note: the Emoji will not show until the app is running.

That’s the app basics taken care of, and when it's done it should look something like this. Bear in mind this is just how we built the app to look, you’re free to modify your own app to look however you like.

Now as promised you don’t need to build the app yourself. You can simply download the Blynk app, create an account and duplicate our app without going to any effort making one. All you need to do is scan this QR image using the Blynk app on your phone.

To clone the project simply press the QR scanner icon and point your camera at the QR code shown above. This will instantly replicate the app in your phone using your very own unique authorization number.

Now we have the app created, it's time to work on getting the ESP32 and app working together.

To do this we need to go back to the Arduino IDE and write just a very small amount of code.

To get started with the sketch for the ESP32 we first need to open the example sketch for the ESP32 with WiFi capabilities from the Blynk library. Navigate to file, examples, Blynk, Boards_WiFi, ESP32_WiFi as shown in the below image.

This will open an example template for us to work from.

Doing so will open the following Arduino sketch. This sketch gives the very basic outline of our interface, we still need to incorporate our servos and the control structure.

The first step should be to add the app authorization number you received via email when creating the app. This code links your app with your hardware. You then need to enter your username and password for your WiFi router. This is usually found on a sticker on the modem/router itself. If you’re in a school or other public place you may need permission from the network administrator before using the network.

This will allow your ESP32 to connect to the internet and therefore receive information directly from the app you just built in Blynk.

For the sketch to control servos we are going to need to download a servo library compatible with the ESP32. To do that we do the same as adding the Blynk library, we navigate to Sketch, include libraries, Manage libraries.

From there we search for ESP32servo.

Install the library shown and add the following code as the first line of our sketch.

#include <ESP32Servo.h> 

This line will allow us to easily add servo control using the standard Arduino library syntax such as:

servoName.write(value); 

Directly below the Authorization token entry, we declare the constants for the Servo pins. These are stored as constants as we will not be changing them during the course of the code. They are to remain constant so to speak.

static const int slewPin = 4;
static const int boom1Pin = 2;
static const int boom2Pin = 5;
static const int clawPin = 18; 

After defining the constants for the servo pins we are going to define the names of the servos, this will make it much easier to read and understand the code later. To do that we use the following code.

Servo slewServo;
Servo boom1Servo;
Servo boom2Servo;
Servo clawServo; 

From here we need to declare our variables for each of the objects we created in the app.

To do that add the following code below the constants we just entered. This creates a memory location for us to store values into for later use. To begin we initialise each variable to zero.

int slew = 0;
int boom1 = 0;
int boom2 = 0;
int claw = 0;
int homePos = 0; 

Now we need to create the functions to read the incoming data relating to the virtual pins in the app and to apply that data to the variables we just created. To do that we use the BLYNK_WRITE() function built into the Blynk library.

BLYNK_WRITE(V0){
  int pinValue = param.asInt(); 
  // assigning incoming value from pin V0 to a variable
  slew = pinValue;
}
BLYNK_WRITE(V1){
  int pinValue = param.asInt(); 
  // assigning values from pin V1 to a variable
  boom1 = pinValue;
}
BLYNK_WRITE(V2){
  int pinValue = param.asInt(); 
  // assigning values from pin V2 to a variable
  boom2 = pinValue;
}
BLYNK_WRITE(V3){
  int pinValue = param.asInt(); 
  // assigning values from pin V3 to a variable
  claw = pinValue;
}
BLYNK_WRITE(V4){
  int pinValue = param.asInt(); 
  // assigning values from pin V4 to a variable
  homePos = pinValue;
}

COMPLETE CODE

#include <ESP32Servo.h>
#define BLYNK_PRINT Serial
#include <WiFi.h>
#include <WiFiClient.h>
#include <BlynkSimpleEsp32.h>
// You should get Auth Token in the Blynk App.
// Go to the Project Settings (nut icon).
char auth[] = "Your Auth code goes here";
static const int slewPin = 4;
static const int boom1Pin = 2;
static const int boom2Pin = 5;
static const int clawPin = 18;
Servo slewServo;
Servo boom1Servo;
Servo boom2Servo;
Servo clawServo;
int slew = 0;
int currentSlew = 45;
int boom1 = 0;
int currentBoom1 = 45;
int boom2 = 0;
int currentBoom2 = 45;
int claw = 0;
int currentClaw = 45;
int homePos = 0;
// Your WiFi credentials.
// Set password to "" for open networks.
char ssid[] = "Your SSID goes here";
char pass[] = "Your PASSWORD goes here";
BLYNK_WRITE(V0){
  int pinValue = param.asInt(); 
  // assigning incoming value from pin V1 to a variable
  slew = pinValue;
}
BLYNK_WRITE(V1){
  int pinValue = param.asInt(); 
  // assigning incoming value from pin V1 to a variable
  boom1 = pinValue;
}
BLYNK_WRITE(V2){
  int pinValue = param.asInt(); 
  // assigning incoming value from pin V2 to a variable
  boom2 = pinValue;
}
BLYNK_WRITE(V3){
  int pinValue = param.asInt(); 
  // assigning incoming value from pin V3 to a variable
  claw = pinValue;
}
BLYNK_WRITE(V4){
  int pinValue = param.asInt(); 
  // assigning incoming value from pin V4 to a variable
  homePos = pinValue;
}
void setup(){
  // Debug console
  Serial.begin(9600);
  slewServo.attach(slewPin);
  boom1Servo.attach(boom1Pin);
  boom2Servo.attach(boom2Pin);
  clawServo.attach(clawPin);
  //Set all servos to a known position 
  //to aid in servo movement smoothing
  slewServo.write(45);
  boom1Servo.write(45);
  boom2Servo.write(45);
  clawServo.write(45);
  Blynk.begin(auth, ssid, pass);
}

LOOP

void loop(){
  Blynk.run();
  // Slew MOVEMENT
  while (slew > currentSlew){
    currentSlew ++;
    slewServo.write(currentSlew);
    delay(20);
  }
  while (slew < currentSlew){
    currentSlew --;
    slewServo.write(currentSlew);
    delay(20);
  }  // Boom1 MOVEMENT
  while (boom1 > currentBoom1){
    currentBoom1 ++;
    boom1Servo.write(currentBoom1);
    delay(20);
  }
  while (boom1 < currentBoom1){
    currentBoom1 --;
    boom1Servo.write(currentBoom1);
    delay(20);
  }  // Boom2 MOVEMENT
  while (boom2 > currentBoom2){
    currentBoom2 ++;
    boom2Servo.write(currentBoom2);
    delay(20);
  }
  while (boom2 < currentBoom2){
    currentBoom2 --;
    boom2Servo.write(currentBoom2);
    delay(20);
  }  // Claw MOVEMENT
  while (claw > currentClaw){
    currentClaw ++;
    clawServo.write(currentClaw);
    delay(20);
  }
  while (claw < currentClaw){
    currentClaw --;
    clawServo.write(currentClaw);
    delay(20);
  }
  //if(homePos = 1){
    //slew = 45;
    //boom1 = 45;
    //boom2 = 45;
    //claw=45;
    //homePos= 0;
  // }
}

TESTING

When testing we identified the need to substantially slow down the servo action. Initially, the servos were directly sent the users desired position and instantly attempted to go to that position. This resulted in some very rapid actions which made the arm very hard to reliably control and in some situations caused the arm to tip over. To rectify this, we used while loops to slowly increment or decrement the servo position until the users desired position is achieved.

// SLEW MOVEMENT
while (slew > currentSlew){
  currentSlew ++;
  slewServo.write(currentSlew);
  delay(20);
}
while (slew < currentSlew){
  currentSlew --;
  slewServo.write(currentSlew);
  delay(20);
}
    

For this to work we need the sketch to start from a known position. To satisfy this requirement we set all servos to their 45° mid position in the setup section. We simply start with this known position and increment or decrement the current value, until it matches the value sent from the app.

Now, of course, this isn’t perfect. The use of the delay function means that only one servo can be moved at any one time, as the sketch can’t escape the delay.

Admittedly, a better solution for this would be using a timer and interrupt. However, given the purpose this solution is more then suitable.

WHERE TO FROM HERE?

You could look at adding a Home button. In theory, the button is a very simple concept. It will allow you, at the push of a button, to have the arm move to a pre-defined position. This is a very handy function if you have actions that must be repeated many times. For example, picking up an object on an assembly line, etc. Because the app does not appear to allow incoming communications, there seems to be no way to change the position of the sliders after the arm has moved to its “home” position. This would mean that on the very next action the program would try to return to the slider position.