Internet of Things - Let's Build Something!

Friday, August 28, 2020 • 19 minutes to read

Over a year ago, I had done a workshop about the Internet of Things. I had targeted this workshop for programmers working with a different technology stack and with a wide range of skills but with little or no knowledge of electronics and IoT in general.

The idea behind this project was to create a small, working example of an IoT device and source code written for both the embedded device and the server-side. During prototyping, we used low-cost, ready-to-use, and easy-to-assemble modules.

The article covers a brief introduction to the Internet of Things, a server-side software written using Go language with a little bit of JavaScript, a hardware solution and wiring, and software written in C for the device itself.

Please note that I created most of this article in February 2019. Although I reviewed, clean up, and updated the source code of the solution before publishing this article, still so some parts may be outdated.

What is the IoT?

The Internet of things is a network of connected devices contain electronics, software, actuators, sensors, and connectivity modules. These devices, vehicles, home appliances connect, interact, and exchange data.

The IoT extends Internet connectivity beyond standard devices, such as desktops, laptops, smartphones, and tablets, to any traditionally non-internet-enabled physical devices and everyday objects. Such devices can communicate and interact over the network, and they can be remotely controlled and monitored.

What are the IoT devices’ categorization and application?

We could put all devices on the Internet of Things into three categories:

There is a wide range of possible applications of such devices that fall into the above categorizations. Let’s list some most common areas where we could use IoT devices, and it could be more than just a buzzword.

Consumer applications:

Commercial applications:

Industrial applications:

Infrastructure applications:

Which network communication fit for IoT projects?

Wireless network communication is way more than a standard Wi-Fi or Bluetooth connection, which you probably use extensively every day. Of course, perhaps all readers hear at least about LTE or some other mobile telephony technologies.

The IoT devices use several different wireless technologies, which I will list and compare in the next sections.

Commonly used wireless technologies.

Short-range wireless:

Medium range wireless:

Long-range wireless:

The comparison.

NFCRFIDBLEWi-FiZigBeeLoPWANLTEWiMAX
TopologyP2PP2PStarStarMesh, Star, TreeMesh, StarMeshMesh
PowerVery Very LowVery LowVery Low - LowLow - HighVery LowVery LowHighHigh
Speed (Mbit/s)0.0040.004 - 0.641 - 2~54 / ~10000.250.001~30070 / 1000
Range<20cm<1m / <100m~100m20m+~20m10km+~5km~50km
Frequency13.56 MHz120 kHz - 5.8 GHz2.4 GHz2.4 GHz / 5.8 GHz784 MHz - 2.4 GHz169 MHz - 915 MHz400 MHz - 3.7 GHz2.3 GHz - 11 GHz

The IoT weather station.

As we learned above, to make an IoT thing, we should add network communication ability to an everyday device. Of course, it makes sense as long as the device is in one of our three categories described at the beginning.

The easiest way to start our journey with IoT devices would be to:

For learning purposes in our example, we will take a weather monitoring station. They usually come with a display unit and one or more sensors. In most cases, they are not sending any data over the Internet, store past data, make them available over the Internet, or display graphs.

In regards to wireless communication, let’s focus on a Wi-Fi connection. It is not the most efficient solution for IoT but would be the easiest to use in our setup.

What should be our goals?

Before we start to build and program our device, let’s set some realistic goals.

  1. We want to create a weather monitoring station (a consumer application).
  2. To simplify the building, we want to use a multi-sensor module (at least temperature and humidity).
  3. Our device should be able to connect to a server and send data using REST API.
  4. We want our device to have: low power usage, low cost, should be small and easy to build.
  5. We want to develop a real embedded device, not a small computer.
  6. Besides building and programming an IoT device, we need to write a simple server application. It will be responsible for collecting data and displaying it as graphs on a webpage.

The hardware.

Keeping in mind our goals from the previous section, the availability, and popularity of hardware, I have chosen ESP826EX SoC, based on Tensilica processor and Bosh BME280 multi-sensor module.

The ESP8266EX System-on-Chip.

Espressif’s ESP8266EX is a Wi-Fi System-on-Chip solution. It could work as a standalone application or as the slave to a host MCU. We could buy it as a standalone chip or a module together with flash memory and some additional passive elements. From the program’s point of view, we could use the integrated SDK.

Hardware:

Wi-Fi support:

Software support on-chip:

The low-power architecture operates in the following modes:

Cost: around 1.5 € for a chip (in reel), about 3 € for a module; about $1.37 sourced from China.

Datasheet

When buying a standalone chip, an SPI flash memory is required, a standard clock oscillator (26 MHz, 10 ppm), and some passive elements.

The BME280 multi-sensor.

Bosh’s BME280 is a tiny multi-sensor chip (2.5mm x 2.5mm x 0.93mm in a metal lid LGA package). It can measure temperature, relative humidity, and pressure. From a technical point of view it also meets all our requirements:

The sensor has three power modes:

Cost: about 3.5 € (in reel); about $2.26 sourced from China.

Datasheet

The hardware solution.

The production solution includes the ESP8266EX module with a recommended antenna setup, a QSPI connection to the NAND Flash memory module, an SPI connection to the BME280 module, and required passive elements.

Solution schematic
Solution schematic
This electronic schematic includes ESP8266EX, 32 Mbit flash module, BME280 module, 26 MHz clock oscillator, and Wi-Fi antenna.

For development purposes, we will use a NodeMCU development board and an SPI connection to the BME280 module. The NodeMCU board includes the ESP8266EX module, a Flash memory module, antenna, USB port, voltage regulator, and USB to UART converter.

Development schematic
Development schematic
This electronic schematic includes the NodeMCU development board and BME280 using the SPI interface connection.

The server-side solution.

The best solution to store data from sensors would be a web server exposing a REST API for IoT devices. For a production environment, we would need a full-featured web server with some relational database system. Possibly all scaled dynamically based on the number of devices.

We will create a web server for the development phase providing a simple API (one endpoint, only index, and store verbs) written in Go. Thanks to Go language and compiler, it will be fully portable, with no external requirements (statically linked).

How to store and read data from sensors?

To save readings that our sensors will send to the server, and read them later, we will be using the SQLite3 database and GORM, the fantastic ORM library for Golang. According to our goals and hardware features, our data model includes:

Database table structure
Database table structure
Diagram representing a database table structure for collecting IoT weather sensor data.

We need to add our GORM model to the code, initialize database connection, and run the migration.

import (
	"github.com/jinzhu/gorm"
	_ "github.com/jinzhu/gorm/dialects/sqlite"
)

type reading struct {
	gorm.Model
	Sensor      uint32  `json:"sensor" binding:"required"`
	Temperature float64 `json:"temperature" binding:"min=-40,max=85"` // min & max from sensor capabilities
	Humidity    float64 `json:"humidity" binding:"min=0,max=100"`
	Pressure    float64 `json:"pressure" binding:"min=300,max=1100"`
	DewPoint    float64
}

var db *gorm.DB

func main() {
	var err error

	db, err = gorm.Open("sqlite3", "api.sqlite3")
	if err != nil {
		panic("Failed to connect to database!")
	}
	defer db.Close()
    db.AutoMigrate(&reading{})
}

Thanks to a great, full-featured web framework for Golang, Gin Gonic, we would be able to start a web server and expose two API calls with just a couple more lines of code. First, let’s add the required imports.

"math"
"net/http"
"github.com/gin-gonic/gin"

We need two functions responsible for reading all readings and adding an entry with new data. We will be sending and expecting a valid JSON data, so we need to remember correctly handling it and returning a valid error code in case of problems.

func getReading(c *gin.Context) {
	var allReadings []reading

	if err := db.Order("created_at desc").Limit(60).Find(&allReadings).Error; err != nil {
		c.JSON(http.StatusServiceUnavailable, gin.H{"message": err.Error()})

		return
	}
	c.JSON(200, allReadings)
}

func postReading(c *gin.Context) {
	var newReading reading

	if err := c.ShouldBindJSON(&newReading); err != nil {
		c.JSON(http.StatusBadRequest, gin.H{"message": err.Error()})

		return
	}
	newReading.DewPoint = math.Round((math.Pow(newReading.Humidity/100, 1.0/8.0)*
		(112.0+0.9*newReading.Temperature)+0.1*newReading.Temperature-112.0)*
		100.0) / 100.0
	if err := db.Create(&newReading).Error; err != nil {
		c.JSON(http.StatusServiceUnavailable, gin.H{"message": err.Error()})

		return
	}
	c.JSON(http.StatusCreated, newReading)
}

The last part is to start the server and register routing for our newly created API. It goes to the main function.

gin.DisableConsoleColor()
router := gin.Default()
router.GET("/reading", getReading)
router.POST("/reading", postReading)
router.Run()

How to display sensors’ data to the end-user?

We already have a working web server, and the most natural way would be to display some charts with the latest data as a web page. Let’s do it with pure HTML and JavaScript, plus a small amount of CSS. To achieve that writing as less code as possible, we will be using the following great libraries:

The HTML and CSS are very basic. It is just two canvases. I use the canvas-container class to style the size of the charts.

<div class="container">
    <h2>Temperature / humidity</h2>
    <div class="canvas-container">
        <canvas id="chart-1"></canvas>
    </div>
    <h2>Pressure</h2>
    <div class="canvas-container">
        <canvas id="chart-2"></canvas>
    </div>
</div>

.canvas-container {
    height: 35vh;
    margin: auto;
    width: 100%;
}

The JavaScript part is responsible for creating a line chart and a bar chart for our sensors’ data. Periodically, every 30 seconds, we call an asynchronous call to /reading API to pull new data and map it to the datasets of charts. I do not include here charts configuration - it is up to the reader’s decision on how to do it.

let chartLines = Chart.Line(document.getElementById('chart-1'));
let chartBars = Chart.Bar(document.getElementById('chart-2'));

function updateData() {
    axios.get('/reading').then(response => {
        all = _.reverse(response.data);
        chartLines.data.labels = _.map(all, (entry) => {
            return moment(entry.CreatedAt).format('H:mm:ss');
        });
        chartBars.data.labels = chartLines.data.labels;
        chartLines.data.datasets[0].data = _.map(all, 'temperature');
        chartLines.data.datasets[1].data = _.map(all, 'DewPoint');
        chartLines.data.datasets[2].data = _.map(all, 'humidity');
        chartBars.data.datasets[0].data = _.map(all, 'pressure');
        chartLines.update();
        chartBars.update();
    }).catch(error => {
        console.error(error);
    });
}

setInterval(updateData, 30000);
updateData();

The embedded software solution.

We already know what our hardware modules’ capabilities are and what format of data we require on the server-side part of our Internet of Things solution. What we need to do is connect to a Wi-Fi network, retrieve weather conditions from the BME280 sensor and send it encoded in JSON format to API endpoint.

To achieve our goals writing as little code as possible, we will use some brilliant libraries:

How to write an embedded hello world?

Before we jump into deep water, let’s start with something straightforward. At the beginning of learning a new programming language or platform, we usually start with a Hello World program. How to translate this to a chip or module?

In most cases, the easiest way would be to blink a LED. Fortunately for us, the NodeMCU module has a built-in LED hardwired with a GPIO1 pin. We do not need to care about how to attach a LED to chip pin, which resistor to choose, where a cathode and anode are, or the maximum current LED could drown.

In general, we need to make sure to set a pin as an output pin, and then, periodically changes a voltage from high to low. Let’s look at the code using the ESP8266 NONOS SDK.

#include "ets_sys.h"
#include "osapi.h"
#include "gpio.h"
#include "os_type.h"

static const uint8_t pin = 1;
static volatile os_timer_t timer;

void timerfunc(void *arg)
{
    if (GPIO_REG_READ(GPIO_OUT_ADDRESS) & (1 << pin)) {
        gpio_output_set(0, (1 << pin), 0, 0);
    } else {
        gpio_output_set((1 << pin), 0, 0, 0);
    }
}

void ICACHE_FLASH_ATTR user_init()
{
    gpio_init();
    PIN_FUNC_SELECT(PERIPHS_IO_MUX_U0TXD_U, FUNC_GPIO1);
    gpio_output_set(0, 0, (1 << pin), 0);
    os_timer_setfn(&timer, (os_timer_func_t *)timerfunc, NULL);
    os_timer_arm(&timer, 1000, 1);
}

Woah, that is a lot of code for changing pin voltage high and low. In reality, we only use gpio_output_set for changing the voltage; the rest of the code is preparing a timer.

Every program trying to do something periodically or delay an execution will need to use timers. At first, it may look complicated, but the idea behind it is relatively simple. However, could we do it with less code?

How blinking LED will look with the Arduino framework?

Anybody at least remotely interesting with DYI electronics and programming 8-bit processors from Atmel hears about Arduino. It will help us to simplify the code (especially timers) and use some easy to understand constructions. Let’s look at the code.

#include <Arduino.h>

void setup()
{
    pinMode(LED_BUILTIN, OUTPUT);
}

void loop()
{
    digitalWrite(LED_BUILTIN, HIGH);
    delay(1000);
    digitalWrite(LED_BUILTIN, LOW);
    delay(1000);
}

It looks a lot shorter and easier to understand. We need to remember that the required setup function is called once after chip boot, the loop function is called in a loop after setup exits.

To be more precise, the loop function is called from a timer every time it exits. That means if we want to run something in parallel, we would still need a timer set up manually. Besides that, we could use a familiar delay function to wait.

To recap, we set our pin to output during setup, and while in a loop, we just change voltage and wait for a second.

How to read data from BME280 multi-sensor via SPI connection?

What does reading data via SPI connection mean?

SPI (Serial Peripheral Interface) is a synchronous serial communication interface commonly used in electronics. How it works is out of the scope of this article. We only need to know that we need four wires to use that connection and that our BME280 library handles it without problems.

How to connect the BME280 module to the NodeMCU via SPI?

The ESP chip provides two hardware SPI buses. The chip use one for communication with the Flash module, the second one we could utilize. The pinout is as below:

According to our development schematics, we are connecting clock to clock, output to input, and vice versa, but for selecting a chip (CSB pin on the BME280), we use D1 (GPIO5). Why? No particular reason. The CS pin in the SPI connection is the only one that is unique per connected device; the rest are shared between al peripherals.

How to read measurements?

From a code perspective, the only things we need to do are:

Of course, we will never know what the readings are if we do not show them somehow. For the development process, let’s use the internal Arduino Serial object for communication over USB to our computer. The whole code looks like below.

#include <Arduino.h>
#include <Adafruit_BME280.h>

Adafruit_BME280 bme(D1);

void setup()
{
    Serial.begin(74880);
    while (!Serial) {
        yield();
    }
    if (!bme.begin()) {
        Serial.println("Cannot connect to BME280!");
        while (1) {
            yield();
        }
    }
}

void loop()
{
    Serial.print("T = ");
    Serial.println(bme.readTemperature());
    Serial.print("H = ");
    Serial.println(bme.readHumidity());
    Serial.print("P = ");
    Serial.println(bme.readPressure() / 100.0f);
    Serial.println();
    delay(10000);
}

There is one comment required for the code above. For the while loops when we are waiting for serial communication is ready or to “stop” after failed connection with the BME280, we use yield. For the time of our processor, our program competes with a Wi-Fi stack. We need to let the timers finish their jobs.

How to connect to Wi-Fi AP from the ESP8266 module?

To connect our device to an accessible Wi-Fi network using ESP8266WiFi, we need to:

The source code, plus some messages on the serial for debugging purposes, you can find below.

#include <Arduino.h>
#include <ESP8266WiFi.h>

#define WIFI_SSID "ssid"
#define WIFI_PASS "password"

void setup()
{
    Serial.begin(74880);
    while (!Serial) {
        yield();
    }
    uint8_t wifiTimeout = 0;

    WiFi.mode(WIFI_STA);
    WiFi.begin(WIFI_SSID, WIFI_PASS);
    while (!WiFi.isConnected()) {
        delay(100);
        yield();
    }
    Serial.print("IP: ");
    Serial.println(WiFi.localIP());
}

void loop()
{
}

How to send JSON encoded data using the ESP8266?

The HTTP connection we will handle using the ESP8266HTTPClient library, JSON encoding we will handle through the ArduinoJson library. Besides adding required headers, let’s define an URL from our server-side solution, we will be using to post data.

#include <ESP8266HTTPClient.h>
#include <ArduinoJson.h>
#define API_PATH "192.168.1.1:8080/reading"

Working with HTTP client is relatively straightforward. We begin a connection, set headers, and post a payload. The JSON part is only a little bit more complicated, but only because of the need to convert data between an object and string.

The JSON object we will populate with data from the BME280 readings. Let’s not forget about adding some debugging messages to the serial port for confirmation that everything is working fine.

void sendReadings()
{
    const uint16_t szBuffer = JSON_OBJECT_SIZE(4);
    StaticJsonDocument<szBuffer> document;
    char jsonMessage[szBuffer];
    WiFiClient client;
    HTTPClient clientHttp;

    document["sensor"] = ESP.getChipId();
    document["temperature"] = bme.readTemperature();
    document["humidity"] = bme.readHumidity();
    document["pressure"] = bme.readPressure() / 100.0f;
    serializeJson(document, Serial);
    serializeJson(document, jsonMessage);
    clientHttp.begin(client, API_PATH);
    clientHttp.addHeader("Content-Type", "application/json");
    clientHttp.POST(jsonMessage);
    Serial.println(clientHttp.getString());
    clientHttp.end();
}

What about power saving with the ESP8266 and BME280?

One of our initial goals was low power usage. A curious reader will notice that our device run on full power for full time, not only when we do readings or send data over Wi-Fi. The average current is about 170 mA, with a 500 mA peak during the Wi-Fi link setup.

It is way too much if we would like to run our IoT device on battery. Of course, current usage is higher than the production product because we have many additional elements on the development board than we need. Nevertheless, some code changes would help us to reduce power usage.

The BME280 power usage is already low, but we could change the setup to do measurements only on request and reduce sampling to one. The sampling configuration should look like that.

bme.setSampling(Adafruit_BME280::MODE_FORCED,
                Adafruit_BME280::SAMPLING_X1,
                Adafruit_BME280::SAMPLING_X1,
                Adafruit_BME280::SAMPLING_X1);

The current consumption will drop to 0.1 μA over the sleeping period. We will force the reading with the bme.takeForcedMeasurement() function.

To further reduce the current usage, we should do all the logic during setup and turn off the ESP8266 after a successful run or encounter an error. There is no need to continue if we cannot connect to the BME280 or, after some time, to the Wi-Fi network. Only then do a reading and send data.

To turn off the device but still to be able to wake it up, we would need to use a deep sleep mode of the ESP8266EX. To do it from code we need to call ESP.deepSleep(60e6) function. This example will set the timer to one minute. During the deep sleep, only the real-time clock is running.

What will happen after one minute? The GPIO16 / D0 pin will be pulled down. If we connect it to the RST pin, which is normally pulled up, we will trigger the device’s reset. This way, we will go through the bootloader to our setup function, eventually wake up the device.

Another improvement which we could do is disable Wi-Fi persistent function using WiFi.persistent(false) command. There is no need to save connection status as we are rebooting the device every minute; this will reduce time and some current usage.

The End.

We should also make all debugging messages optional and do not initialize the Serial object to reduce the size of the production code. It will also make it slightly faster.

The complete source code, both embedded and server-side solutions, are available on https://github.com/kiesiu/iot-weather-station. It comes with PlatformIO configuration for faster development.

electronicsdevelopmentcppgolangesp8266edaarduinoiotbme280
See Also
This site uses cookies to analyze traffic and for ads measurement purposes according to your browser settings. Access to those cookies is shared with Google to generate usage statistics and detect and address abuse. Learn more about how we use cookies.
I understand