In short, this project has two features. First: the table is connected from Google Smart Home to Heroku using voice commands, and second: Heroku and the table itself communicate using the MQTT Internet of Things protocol. MQTT is a good solution for the Internet of Things, as well as for overcoming some other obstacles that we will have to face.
First of all, I will say that I did this project just for fun. I hope you find this article entertaining and will motivate you to take the time to do something of your own.
Hardware part
The first and probably the most difficult part of the job is redesigning the table. In a past life, the table had a detachable handle, it was located at the edge of the table top. At first I thought about attaching something to the hole of the handle without having to interfere with the design of the table. I purchased several drives to figure out how to attach the motor to the table, but to no avail. Then the idea came up: a rod running along the length of the entire table, which would connect its legs so that they rise and fall at the same time. If I attach a drive that fits the rod, then I can use a belt to connect the rod to the motor. It would also be possible to equip the table with a motor without interfering much with its design.
The importance of torque
After ordering the right drive and belt, I started searching Amazon for a high torque motor. And - oh, miracle! - I have found many suitable engines! Or so it seemed to me ... Having bought a small motor, I waited for its arrival from China for about a month. I was so excited when the motor finally arrived! Couldn't wait for the weekend to finally put it all together and have my motorized desk.
Things didn't go according to plan. I spent the day cutting a hole for a rod in the metal paneling of the table. At that point, I only had hand tools, so the process took longer than I expected. Towards the end of the day, I finished assembling the table and was ready to try it out.
I turned on the motor, voltage on my desktop power supply and ... nothing happened. A few moments later, the motor began to turn and the teeth of the acquired belt rattled. I learned two lessons from this: the belt is obviously not doing its job, and the word "Motor in high torque" does not mean "I can lift anything." The second lesson is to look at how big the motor is compared to your fingers. Mine turned out to be tiny!
On the left in the photo is a motor and a belt. Above right is a motor attached to the table (you will see more about this later). At the bottom right, the motor is in position on the table.
Suitable motor
To select the right motor, it was necessary to calculate how much torque was required to raise the tabletop. I was surprised at how easy it is to do this.
Torque is the force multiplied by the length of the lever arm.
Well, I had a lever arm (this is a table handle), it was only necessary to calculate the force that would easily turn the lever arm. I loaded the table by tying the milk jug to the handle and gradually added water to the jug until the lever began to rotate. By turning the handle upward with the jug full, I made sure the weight turns the handle easily. I found that the lever arm is 11 cm long and the force required is 4 lbs. Substituting these numbers into the formula, I found out that the motor must produce a torque of at least 19.95 kg / cm. And he started looking for him.
I decided to remake the table irreversibly. I knew that the rod going through the middle of the table was hollow. After looking for a twin shaft motor, I could cut the rod and reassemble it with the motor in the middle. By purchasing two motors with 20 kg / cm torque, I ensured that there is enough torque to lift the table.
On another beautiful Saturday, I took my table apart into four pieces, sawing the motor shafts so that they could be used when assembling the rod. I pushed more holes in the metal to fit the motors. There was no belt this time: the motors were connected directly to the rod, the holes were quite large. As evening fell, I reassembled the desk and loaded it up with office supplies.
The top two photos are motors fully mounted on the table. The two bottom photos are an integrated rod that runs along the length of the table with the help of motors.
I hooked up the motors and connected them to the power supply. Turning on the power, I saw the table move! This time I was more confident, because I correctly sized the motors. I doubled the power of the engines for the sake of confidence, but it was amazing to see them move!
However, let me clarify that the table was slow. I shot a video to show a friend how the table works, but he had to turn on the time acceleration on the video in order not to watch for 5 minutes as the table changes the table top position.
The turnover is important. Final version
Finally I realized that it all comes down to two things: torque and revs. It was necessary to find a motor with a sufficient number of revolutions at an already known torque.
It wasn't that hard. Although I didn’t find a twin shaft motor, I did find a rectangular gearbox that converts a single shaft motor into a twin shaft motor.
In short, the next month was a month of waiting for a gearbox from China, and the next Saturday after waiting, I had a table moving at the right speed.
The last motor itself is on the left, and the installed motor is on the right. Little hardware and lots of software.
I was not happy with the huge power supply on my desk, lying just to control the height of the tabletop. In addition, to change the position of the table from one to the other and back, I swapped the wires. Small problem, but the project was done to ideally just press a button and have multiple height presets.
Bluetooth
The first solution was to add Bluetooth to the table. At the end of the day, it looks like almost every device in the house has Nluetooth, and the phone seems to be a convenient control interface for something like my desk.
So now I got a motor controller board, a Nordic NRF52 bluetooth board, distance sensors and started fiddling with the controller firmware.
At the end of the article, I will leave links to software and firmware that I wrote for the project. Feel free to comment on the code: I am not a professional firmware developer and would like to receive some guidance.
As a quick introduction: ESP32 is written in C ++ using Arduino libraries to interact with the BLE Terminal app on the phone. Installing and configuring BLE is quite complex. First, you need to create all the characteristics for the values that you would like to control through BLE. Think of a characteristic as a variable in your code. BLE wraps a variable in multiple handlers to get and set the value of that variable.
The characteristics are then packaged into a service with its own UUID, which makes the service unique and identifiable from the application. Finally, you must add this service to the ad payload so that your service can be discovered by the device. When a remote device connects to your service and sends data through the specifications, the table recognizes that the user wants to adjust the height to a different preset and starts moving.
For height adjustment, the table top has a TFMini-S LiDAR sensor that detects the current height. This is a fun sensor: it's called a LiDAR when it's actually a laser. It uses optics and LEDs to determine the flight time of the infrared radiation. One way or another, the sensor determines the height of the table. The control board then detects the difference between the current altitude and the requested altitude and starts the motor, which rotates in the desired direction. Some of the main parts of the code are shown below, but you can see the entire file here .
void setup()
{
Serial.begin(115200);
Serial2.begin(TFMINIS_BAUDRATE);
EEPROM.begin(3); // used for saving the height presets between reboots
tfminis.begin(&Serial2);
tfminis.setFrameRate(0);
ledcSetup(UP_PWM_CHANNEL, PWM_FREQUENCY, PWM_RESOLUTION);
ledcAttachPin(UP_PWM_PIN, UP_PWM_CHANNEL);
ledcSetup(DOWN_PWM_CHANNEL, PWM_FREQUENCY, PWM_RESOLUTION);
ledcAttachPin(DOWN_PWM_PIN, DOWN_PWM_CHANNEL);
state_machine = new StateMachine();
state_machine->begin(*t_desk_height, UP_PWM_CHANNEL, DOWN_PWM_CHANNEL);
BLEDevice::init("ESP32_Desk");
...
BLEServer *p_server = BLEDevice::createServer();
BLEService *p_service = p_server->createService(BLEUUID(SERVICE_UUID), 20);
/* ------------------- SET HEIGHT TO PRESET CHARACTERISTIC -------------------------------------- */
BLECharacteristic *p_set_height_to_preset_characteristic = p_service->createCharacteristic(...);
p_set_height_to_preset_characteristic->setCallbacks(new SetHeightToPresetCallbacks());
/* ------------------- MOVE DESK UP CHARACTERISTIC ---------------------------------------------- */
BLECharacteristic *p_move_desk_up_characteristic = p_service->createCharacteristic(...);
p_move_desk_up_characteristic->setCallbacks(new MoveDeskUpCallbacks());
/* ------------------- MOVE DESK UP CHARACTERISTIC ---------------------------------------------- */
BLECharacteristic *p_move_desk_down_characteristic = p_service->createCharacteristic(...);
p_move_desk_down_characteristic->setCallbacks(new MoveDeskDownCallbacks());
/* ------------------- GET/SET HEIGHT 1 CHARACTERISTIC ------------------------------------------ */
BLECharacteristic *p_get_height_1_characteristic = p_service->createCharacteristic(...);
p_get_height_1_characteristic->setValue(state_machine->getHeightPreset1(), 1);
BLECharacteristic *p_save_current_height_as_height_1_characteristic = p_service->createCharacteristic(...);
p_save_current_height_as_height_1_characteristic->setCallbacks(new SaveCurrentHeightAsHeight1Callbacks());
/* ------------------- GET/SET HEIGHT 2 CHARACTERISTIC ------------------------------------------ */
...
/* ------------------- GET/SET HEIGHT 3 CHARACTERISTIC ------------------------------------------ */
...
/* ------------------- END CHARACTERISTIC DEFINITIONS ------------------------------------------ */
p_service->start();
BLEAdvertising *p_advertising = p_server->getAdvertising();
p_advertising->start();
xTaskCreate(
updateDeskHeight, // Function that should be called
"Update Desk Height", // Name of the task (for debugging)
1024, // Stack size
NULL, // Parameter to pass
5, // Task priority
NULL // Task handle
);
}
There's a lot more going on in the file, but this code has enough context to understand what's going on. Note that we create and configure all BLE callbacks for all characteristics, including manual movement, setting and retrieving preset values, and most importantly, aligning the table with the preset.
The image below shows the interaction with the characteristics for adjusting the table height. The final piece of the puzzle is a state machine that knows the current table height, the user's desired height, and works with these two values.
So finally I had a table that did whatever I wanted. I could save the height to presets and extract the heights from memory to set the table to my favorite positions. I used BLE Terminalon my phone and computer so I could send raw messages to my desk and monitor its position. It worked, but I knew the battle with BLE was just beginning.
Naked bluetooth interface ... All that was left at the moment was to learn how to write applications for iOS ...
After all this, my wife said something that changed the whole project: "What if you make control of your voice?"
In addition to being cool and adding a new device to the Google Assistant list, there is no need to write an iOS application to control the table. And you no longer had to reach for your phone to adjust the height. Another small victory!
Adding the Internet of Things
Now let's talk about upgrading your desk to voice control via Google Smart Home and how to make it Wi-Fi friendly.
Adding Wi-Fi was easy enough. I replaced the Nordic NRF52 microcontroller with an ESP32 with built-in WiFi. Most of the software was portable because it was written in C ++, and both devices could be programmed with Platform.IO and the Arduino libraries, including the tfmini-s I wrote to measure the current table height.
The architecture of the system of interaction of the table with Google Smart Home is shown below. Let's talk about the interaction between me and Google.
So Bluetooth has been turned on. It's time to figure out how to interact with Google Smart Home. This technology controlled the home using Smart Home Actions . What is interesting about her actions is that the service acts like an OAuth2 server, not a client. Most of the work done with the server has been to implement an OAuth2 Node.js Express application that gets to Heroku and communicates like a proxy between Google and my desk.
I was lucky: there was a decent server implementation using two libraries. The first library, node-oauth2-server, was found here . The second express-oauth-server library for Express connection was found here .
const { Pool } = require("pg");
const crypto = require("crypto");
const pool = new Pool({
connectionString: process.env.DATABASE_URL
});
module.exports.pool = pool;
module.exports.getAccessToken = (bearerToken) => {...};
module.exports.getClient = (clientId, clientSecret) => {...};
module.exports.getRefreshToken = (bearerToken) => {...};
module.exports.getUser = (email, password) => {...};
module.exports.getUserFromAccessToken = (token) => {...};
module.exports.getDevicesFromUserId = (userId) => {...};
module.exports.getDevicesByUserIdAndIds = (userId, deviceIds) => {...};
module.exports.setDeviceHeight = (userId, deviceId, newCurrentHeight) => {...};
module.exports.createUser = (email, password) => {...};
module.exports.saveToken = (token, client, user) => {...};
module.exports.saveAuthorizationCode = (code, client, user) => {...};
module.exports.getAuthorizationCode = (code) => {...};
module.exports.revokeAuthorizationCode = (code) => {...};
module.exports.revokeToken = (code) => {...};
Next comes the configuration of the Express application itself. Below are the endpoints required for an OAuth server, but you can read the full file here.
const express = require("express");
const OAuth2Server = require("express-oauth-server");
const bodyParser = require("body-parser");
const cookieParser = require("cookie-parser");
const flash = require("express-flash-2");
const session = require("express-session");
const pgSession = require("connect-pg-simple")(session);
const morgan = require("morgan");
const { google_actions_app } = require("./google_actions");
const model = require("./model");
const { getVariablesForAuthorization, getQueryStringForLogin } = require("./util");
const port = process.env.PORT || 3000;
// Create an Express application.
const app = express();
app.set("view engine", "pug");
app.use(morgan("dev"));
// Add OAuth server.
app.oauth = new OAuth2Server({
model,
debug: true,
});
// Add body parser.
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
app.use(express.static("public"));
// initialize cookie-parser to allow us access the cookies stored in the browser.
app.use(cookieParser(process.env.APP_KEY));
// initialize express-session to allow us track the logged-in user across sessions.
app.use(session({...}));
app.use(flash());
// This middleware will check if user's cookie is still saved in browser and user is not set, then automatically log the user out.
// This usually happens when you stop your express server after login, your cookie still remains saved in the browser.
app.use((req, res, next) => {...});
// Post token.
app.post("/oauth/token", app.oauth.token());
// Get authorization.
app.get("/oauth/authorize", (req, res, next) => {...}, app.oauth.authorize({...}));
// Post authorization.
app.post("/oauth/authorize", function (req, res) {...});
app.get("/log-in", (req, res) => {...});
app.post("/log-in", async (req, res) => {...});
app.get("/log-out", (req, res) => {...});
app.get("/sign-up", async (req, res) => {...});
app.post("/sign-up", async (req, res) => {...});
app.post("/gaction/fulfillment", app.oauth.authenticate(), google_actions_app);
app.get('/healthz', ((req, res) => {...}));
app.listen(port, () => {
console.log(`Example app listening at port ${port}`);
});
There is quite a lot of code, but I'll explain the main points. The two OAuth2 routes used for the server are / oauth / token and / oauth / authorize. They are used to receive a new token or refresh expired tokens. Next, you need to make the server respond to Google's action. You will notice that the / gaction / fulfillment endpoint is pointing to an object
google_actions_app
.
Google sends requests to your server in a specific format and provides a library to help process those requests. Below are the functions required to communicate with Google, and the entire file is here. Finally, there is the / healthz endpoint, which I'll cover at the end of the article.
The / gaction / fulfillment endpoint uses a middleware called app.oauth.authenticate (), the hard work of getting the OAuth2 server up and running was to get this middleware to work. It verifies that the bearer token provided to us by Google refers to an existing user and has not expired. The route then sends the request and response to the object
google_actions_app
.
Google sends requests to your server in a specific format and provides a library to help analyze and process those requests. Below are the features you need to contact Google, but you can view the entire file here .
const { smarthome } = require('actions-on-google');
const mqtt = require('mqtt');
const mqtt_client = mqtt.connect(process.env.CLOUDMQTT_URL);
const model = require('./model');
const { getTokenFromHeader } = require('./util');
mqtt_client.on('connect', () => {
console.log('Connected to mqtt');
});
const updateHeight = {
"preset one": (deviceId) => {
mqtt_client.publish(`/esp32_iot_desk/${deviceId}/command`, "1");
},
"preset two": (deviceId) => {
mqtt_client.publish(`/esp32_iot_desk/${deviceId}/command`, "2");
},
"preset three": (deviceId) => {
mqtt_client.publish(`/esp32_iot_desk/${deviceId}/command`, "3");
},
};
const google_actions_app = smarthome({...});
google_actions_app.onSync(async (body, headers) => {...});
google_actions_app.onQuery(async (body, headers) => {...});
google_actions_app.onExecute(async (body, headers) => {...});
module.exports = { google_actions_app };
When you add smart action to your Google account, Google will make a sync request. This query lets you know which devices are available from your account. Next comes a polling query: Google queries your devices to determine their current state.
When you first add a Google Action to your Smart Home account, you will notice that Google first sends a sync request and then a poll request to get a holistic view of your devices. The last one is the request is a request to fulfill that Google tells your devices to do something.
"Features" (trait) of the Google Smart Home device
Google uses device-specific features to provide user interface elements for controlling your devices to Google, and to create voice control communication templates. Some of the features include the following settings: ColorSetting, Modes, OnOff, and StartStop. It took me a while to decide which feature would work best in my application, but later I chose modes.
You can think of modes as a drop-down list where one of N predefined values is selected or, in my case, a preset height. I named my mode "height" and the possible values are "preset one", "preset two" and "preset three". This allows me to control my desk by saying, “Hey Google, set my desk height to preset one,” and Google will send a corresponding execution request to my system. You can read more about the features of Google devices here .
Project in action
Finally, Google Smart Home and my computer started communicating. Before that, I used ngrok to run the Express server locally . Now that my server is finally working well enough, it's time to make it available to Google anytime. So, it was necessary to host the application on Heroku - this is a PaaS provider that makes it easier to deploy and manage applications.
One of the main advantages of Heroku is add-on mode. With add-ons, it's very easy to add CloudMQTT and Postgres server to your application. Another benefit of using Heroku is its ease of assembly and deployment. Heroku automatically detects what code you are using and builds / deploys it for you. You can find more details on this by reading about Heroku Buildpacks . In my case, whenever I push code to git remote Heroku, it installs all my packages, removes all development dependencies and deploys the application, all with a simple "git push heroku main" command.
In just a few clicks, CloudMQTT and Postgres were available to my application, and I only needed to use a few environment variables to integrate these services with my application. Heroku asked for no money. However, CloudMQTT is a third-party add-on for $ 5 per month.
I believe the need for Postgres is self-explanatory, but CloudMQTT deserves more attention.
From the Internet to a private network. The hard way
There are several ways to provide access to an application or, in my case, an IoT device. The first is to open a port on my home network to bring the device to the Internet. In this case, my Heroku Express app will send a request to my device using the public IP address. This would require me to have a public static IP as well as a static IP for the ESP32. ESP32 would also have to act as an HTTP server and listen for instructions from Heroku all the time. This is a big overhead for a device that receives instructions several times a day.
The second method is called a hole punch. With it, you can use a third-party external server to access the device to the Internet without the need for port forwarding. Your device basically connects to a server that sets up an open port. Then another service can connect directly to your backend by getting an open port from the outside server. Finally, it connects directly to the device using this open port. The approach may or may not be entirely correct: I read only part of the article about it.
There is a lot going on inside the "hole punch", and I do not fully understand what is happening. However, if you're curious, there are some interesting articles explaining more. Here are two articles I read to better understand the hole punch: Wikipediaand an article from MIT by Brian Ford and others .
From the Internet to a private network via the IoT
I was not very happy with these solutions. I connected a lot of smart devices to my home and I never had to open a port on my router, so there was no port forwarding. Also, punching holes seems much more difficult than what I'm looking for and is better suited for P2P networks. As a result of further research, I discovered MQTT and learned that it is a protocol for IoT. It has some advantages such as low power consumption, configurable fault tolerance, and does not require port forwarding. MQTT is a publisher / subscriber protocol, which means that a table is a subscriber to a specific topic, and a Heroku application is the publisher of that topic.
So Google contacts Heroku, this request is analyzed to determine the requested device and its new state or mode. The Heroku app then publishes a message to the CloudMQTT server deployed as an add-on to Heroku instructing the table to navigate to the new preset. Finally, the table subscribes to the topic and receives a message posted by the Heroku app, finally, the table adjusts its height as requested! In the googleactionsapp file, you will notice that there is an updateHeight function that publishes one MQTT number for a specific device ID. This is how the Heroku application publishes a table move request to MQTT.
The last step is to receive the message on the ESP32 and move the table. I'll show you some of the highlights of the table code below, and all the source code is here .
void setup()
{
Serial.begin(115200);
...
tfminis.begin(&Serial2);
tfminis.setFrameRate(0);
...
state_machine = new StateMachine();
state_machine->begin(*t_desk_height, UP_PWM_CHANNEL, DOWN_PWM_CHANNEL);
setup_wifi();
client.setServer(MQTT_SERVER_DOMAIN, MQTT_SERVER_PORT);
client.setCallback(callback);
...
}
When the table is loaded, we first initiate communication between the TFMini-S - the distance sensor - to get the current table height. Then we set up the state machine for the table movement. The state machine receives commands via MQTT and is then responsible for matching the user's request to the actual table height as read by the distance sensor. Finally, we connect to the Wi-Fi network, connect to the MQTT server, and set up a callback for any data received on the MQTT topic we are subscribed to. Below I will show the callback function.
void callback(char *topic, byte *message, unsigned int length)
{
...
String messageTemp;
for (int i = 0; i < length; i++)
{
messageTemp += (char)message[i];
}
if (messageTemp == "1") {
state_machine->requestStateChange(ADJUST_TO_PRESET_1_HEIGHT_STATE);
}
if (messageTemp == "2") {
state_machine->requestStateChange(ADJUST_TO_PRESET_2_HEIGHT_STATE);
}
if (messageTemp == "3") {
state_machine->requestStateChange(ADJUST_TO_PRESET_3_HEIGHT_STATE);
}
...
}
The state machine registers the state change received in the MQTT topic. Then it processes the new state in the main loop.
void loop()
{
if (!client.connected())
{
reconnect();
}
client.loop();
state_machine->processCurrentState();
}
The main loop does several things: first, it reconnects to the MQTT server if not already connected. Then it processes all the data received through the MQTT topic. Finally, the code works by moving the tabletop to the desired location requested in the MQTT topic.
That's all! The table is completely voice controlled and communicates with Google to receive commands!
Recent Notes
The last endpoint I didn't mention is the / healthz endpoint. This is due to the fact that Google expects a fairly quick response, and loading the Heroku app on every request does not work in my case. I set up a ping service to ping the / healthz endpoint every minute to keep the service healthy and ready to respond. If you plan to do something like this, remember that all free hours at the booth will be spent on it. Everything is fine now: this is the only application used on Heroku. Plus, for $ 7 a month, you can upgrade to Heroku's Hobby plan that keeps the app running.
Building an IoT device comes with a lot of overhead in the beginning. I designed the hardware, built the control scheme, configured the MQTT server, wrote the Express OAuth2 server, and learned how to interact with Google Smart Home through actions. The initial overhead was huge, but I feel like I've accomplished a lot! Not to mention, MQTT Server, Express OAuth2 Application Server and Google Smart Home Actions can be used for another project. I am interested in smart homes, and I can try to expand my repertoire of IoT devices to include sensors that track what is happening around my house and report it through MQTT. Sensors for monitoring soil, temperature and light sensors will be very interesting to monitor and analyze.
What's next?
Countertop heights are now measured unreliable at best. I am using in general a working infrared distance sensor TFMini-S. It has been noticed that the height of the table changes slightly during the day when the ambient lighting in the room changes. I ordered a rotation angle sensor to calculate the revolutions of a rod through the table. This should give me more accurate movement at any time of the day. I also have access to the server which I host in the basement. On it I can explore my own Mosquitto MQTT server, Node-RED and Express OAuth2 applications if I want to host something myself. Finally, now all the electronics are right on my desk. I plan to organize the devices so that everything is nice and neat!
Thanks for reading the article! For convenience, I give all the links.
- Torque Calculator
- 90 degree right angle gear box
- BLE Terminal
- Platform.IO
- TFMini-S Arduino Driver
- Google Smart Home Actions
- Node OAuth2 Server
- Express OAuth2 Server
- ESP32 IoT Desk Server model.js
- ESP32 IoT Desk Server index.js
- ESP32 IoT Desk Server google_actions.js
- Google Smart Home Device Traits
- NGROK
- ESP32 IoT Desk Firmware
- Node-RED
- Heroku
- Heroku Hobby Plan
- Heroku Buildpacks
- Wikipedia Hole Punching
- MIT Paper on Hole Punching by Bryan Ford et al.