Single board computer projects

Single board home server with Debian, openHAB 2 and MQTT

last updated: 29/01/18

My beaglebone was for some years a faithful server for my home data (photo-voltaic, heat-pump ...).

With more and more python scripts it had his problems and I got some times out of memory errors from the kernel. As I couldn't find the real cause of the errors (gnuplot very slow with big data files and many instances of gnuplot running at the same time?), I decided get another server for generating graphics.

In the meantime I got an Udoo X86 from a kickstarter campaign, and I will try to configure it as home server using MQTT and openHAB 2.

The Udoo advanced is a fan-less bard with an Intel Curie (x86 no arm) processor and 4GB RAM and a SATA port. Its much faster than the beaglebone or raspi.

Make a bootable USB-Stick from an .iso-file

The operating system will be debian stretch (this will also work for other distributions). Download the .iso file for net install (in my case: debian-9.3.0-amd64-netinst.iso).

To burn an .iso-file to a USB drive, you can use UNetbootin. UNetbootin allows to create bootable Live USB drives for Debian, Ubuntu, and other Linux distributions without burning a CD. It runs on Windows, Linux, and Mac OS X. You can either let UNetbootin download one of the many distributions or supply your own Linux .iso-file. The user interface is easy to use. Install in Linux with:

    sudo apt install unetbootin

In Linux you can also use the dd-command on the command line. Insert the USB drive (do not mount the drive), open a terminal and type dmesg. Now you see the name of your your drive (e.g. sdb, sde, ... ).

Now type the following command to burn the .iso-file: bash sudo dd status=progress if=debian-9.3.0-amd64-netinst.iso of=/dev/sde bs=4M

To test the burning, we can use QEMU, a machine emulator and virtualizer to boot into the usb drive. Install qemu (apt-get install qemu) and run the following command:

    sudo qemu-system-x86_64 -hda /dev/sde

qemu

Connect the external SSD

The power header of my SSD adapter has just two pins (5V and GND) in 2.54mm raster. Udoos header is 2mm raster with 4 pins (NC, 2 GND and 5V). A little hack was quickly done:

Udoo<em>SSD</em>circuit Udoo_SSD

Connect a mouse, keyboard and HDMI display to Udoo to proceed. Udoo needs a 12V, 1A power supply (5-6W).

Install the operating system

Without a Stick Udoo boots in a boot shell. The Bios is configured to boot from USB, so power it up with the USB Stick inserted. If you need to change things in Bios, press ESC at boot.

After booting from the USB-Stick, choose the graphical install. Choose the right language, localization, configure your keyboard and enter a hostname (e.g udoo-server) and a domain name. If you leave the root password empty, debian will disable root and the normal user account will be given the power to be root with sudo.

After this create your user account.

I do the partitioning manually (Partition disks: Manual), and want to use the efi mode. All files are stored on the external SSD.

nr size device mount point file system partition
1 200 MB /dev/sda1 /boot/efi esp efi system partition
2 16 GB /dev/sda2 - swap swap partition
3 128 GB /dev/sda3 / ext4 root partition
4 106 GB /dev/sda4 /home ext4 home partition

After choosing a download server the installer asks for choose the software to install. As a server, I don't need a GUI, so I install debian headless only with an SSH server.

Update and upgrade

After reboot we log in and get the latest versions of all programs:

    sudo apt update
    sudo apt upgrade
    sudo apt dist-upgrade

It is a good idea to install the terminal file manager "midnight commander" (mc) to search and edit files as root in terminal (sudo mc), and htop an interactive process viewer.

    sudo apt install mc  htop

Set timezone

    sudo timedatectl set-timezone Europe/Luxembourg

Control with date. (to list all timezones : sudo timedatectl list-timezones)

Add sudo

If you have a root account and want to use sudo for your default user:

    apt install sudo
    addgroup username sudo

Static IP address

The IP address of your server is given with DHCP. To get it use the following command:

    ip a

To get a static IP adress is easy in Stretch. Use the editor nano or mc (F4) to append the following to the file /etc/network/interfaces.

# The primary network interface
allow-hotplug enp2s0
iface enp2s0 inet static
    address 192.168.1.100
    netmask 255.255.255.0
    gateway 192.168.1.1

Save with CTRL+O and exit with CTRL+X and reboot.

Create a backup

After all this work we want to create a backup of the installation. To do so we use a Live USB Stick (e.g. Ubuntu) and boot the server from the Stick (ESC at reboot, Boot Manager).

We connect an external USB-HDD with more space then our SSD, open it in a filemanager (mounting) and locate the mount point (e.g. /media/ubuntu/label). After this we use dd to get an image. A pipe through gzip reduces the size.

    dd bs=1M if=/dev/sda | gzip > /media/ubuntu/bu/udooServerDeb.img.gz

Log in with SSH

Now we can remove the display, keyboard and mouse an login from another computer with SSH:

    sudo ssh username@192.168.1.100

or as root:

    sudo ssh root@192.168.1.100

If we don't know the IP address of the server after a reboot, because it was given by DHCP we can use the nmap-command. If nmap is not installed (for Windows look at ):

    sudo apt install nmap
    sudo nmap -sP 192.168.1.*

Look for the IP address of your server (e.g. 192.168.1.199) and log in with SSH (putty on windows):

    sudo ssh username@192.168.1.199

Install openHAB 2

More infos in https://docs.openhab.org/installation.

OpenHAB 2 is written in Java, so we need a JAVA platform (For ARM sb-computers install the 32 bit version!).

As suggested we use Zulu:

    sudo apt install dirmngr
    sudo apt install software-properties-common python-software-properties
    sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 0x219BD9C9
    sudo apt-add-repository 'deb http://repos.azulsystems.com/debian stable main'
    sudo apt update
    sudo apt install zulu-8

Now we will install openHAB 2 stable release:

    wget -qO - 'https://bintray.com/user/downloadSubjectPublicKey?username=openhab' | sudo apt-key add -
    sudo apt-get install apt-transport-https
    echo 'deb https://dl.bintray.com/openhab/apt-repo2 stable main'\
    | sudo tee /etc/apt/sources.list.d/openhab2.list
    sudo apt update
    sudo apt install openhab2

The backslash in line 3 means that the command from line 3 continues in line 4. If you type the command in one line, remove the backslash.

Now we can start openHAB 2 by and test if its running:

    sudo systemctl start openhab2.service
    sudo systemctl status openhab2.service

To start on installation, execute the following commands to configure openHAB 2 to start automatically using systemd:

    sudo systemctl daemon-reload
    sudo systemctl enable openhab2.service

Install mosquitto

Install the mqtt server with:

    sudo apt install mosquitto

Paths for openHAB 2 files

In /etc/default we find the config file openhab2. In this file we find the links to the openHAB 2 default paths:

OPENHAB_HTTP_PORT=8080
OPENHAB_HTTPS_PORT=8443
OPENHAB_BACKUPS=/var/lib/openhab2/backups
OPENHAB_HOME=/usr/share/openhab2
OPENHAB_CONF=/etc/openhab2
OPENHAB_RUNTIME=/usr/share/openhab2/runtime
OPENHAB_USERDATA=/var/lib/openhab2
OPENHAB_LOGDIR=/var/log/openhab2

Set up the MQTT Binding for openHAB 2

First we open openHAB in a browser with the url: serverip:8080 (e.g 192.168.1.100:8080). Then we select Paper UI to install the MQTT binding. Search in Add-ons Bindings for MQTT Binding (Binding-mqtt1) and install it.

Because it is a openHAB 1.x Binding, we need to configure the Binding settings through a configuration file. You find the configuration file mqtt.cfg among the service files (etc/openhab2/services/mqtt.cfg). Replace the #<broker>.url... and the #<broker>.clientId... lines with:

mosquitto.url=tcp://localhost:1883
mosquitto.clientId=openHAB2

It is important to restart openHAB 2, to take the changes in account:

    sudo systemctl restart openhab2.service

If all went well you will see the following in /var/log/openhab2/openhab.log:

MQTT Service initialization completed.
Starting MQTT broker connection 'mosquitto'

More infos can be found at: https://www.openhab.org/addons/bindings/mqtt1

Configuring openHAB 2 for the smartyreader

OpenHAB 2 defines the following base components:

Bindings: the numerous add-ons to communicate with your devices
Things: your physical devices represented in openHAB 2
Items: properties and capabilities linked to Channels of your Things
Groups: collections or categories containing Items
Sitemaps: user-defined front-end interfaces to arrange Groups, Items, ...
Transformations: helper functions to transform your data
Persistence: services to store data over time
Rules automation logic, the “smart” in your Smart Home!

Defining the Thing (our smartyreader :))

We need a configuration text file for the smartyreader (our Thing) in /etc/openhab2/things/.

The syntax for Things is defined as follows (parts in <..> are required):

Thing <binding_id>:<type_id>:<thing_id> "Label" @ "Location" [ <parameters> ]

So we type:

    Thing mqtt:device:smartyreader "lamsmarty" @ "school" [ip="192.168.1.114"]

and save this in a text file named lamsmarty.things (directory /etc/openhab2/things/).

The ip address is the fixed ip of our smartyreader (see smartyreader arduino code).

Link to openHAB doc on Things: https://www.openhab.org/docs/configuration/things.html

Defining the Items

The syntax for Items is defined as follows:

itemtype itemname "labeltext [stateformat]" <iconname> (group1, group2, ...)
["tag1", "tag2", ...] {bindingconfig}

Fields must be entered in the order shown, itemtype and itemname are mandatory and all other fields are optional. Fields may be separated by one or more spaces, or tabs and an Item definition may span multiple lines.

For the bindingconfig we use our mqtt binding (more info here):

{mqtt="<direction>[<broker>:<topic>:<type>:<transformation>:<regex_filter>],
 <direction>[<broker>:<topic>:<type>:<transformation>:<regex_filter>], ..."}

Here a line for our smartyreader:

String Smarty_var1 "Smarty Text: [%s]" <smarty> (gSmartys )
{ mqtt = "<[mosquitto:basement/smarty1:state:default]" }

We create a text file with this line and name it lamsmarty.items in the directory /etc/openhab2/items/).

Link to openHAB doc on Items: https://www.openhab.org/docs/configuration/items.html

Defining the Sitemap

We create a text file with the following text and name it lamsmarty.sitemap, It is the saved in the sitemaps directory /etc/openhab2/sitemaps/).

sitemap lamsmarty label="Smarty" {
  Frame label="Smarty" {
    Text item=Smarty_var1 icon="switch"
  }
}

You don't need to add the "label" field to your sitemap if you already define a text label in the Items file.

Link to openHAB doc on Sitemaps: https://www.openhab.org/docs/configuration/sitemaps.html

Extracting JSON data

Ok Here we got the text, but it would be nicer to get the different values from our JSON Text. This can be done with the JSONPath Transformation add-on.

Install JSONPath Transformation add-on with Paper UI (Add-ons Transformations).

    String Smarty1__Consumption "Smarty1 Cons. [%s]" <smarty> (gSmartys )  { mqtt = "<[mosquitto:basement/smarty1:state:JSONPATH($c1):]" }

Link to openHAB doc on JSONPath: https://www.openhab.org/addons/transformations/jsonpath/

A restart is smart

Sometimes openHAB is not reacting on changes in the text files (perhaps because of the mqtt binding). It is good to restart openHAB 2, to take the changes in account:

    sudo systemctl restart openhab2.service

Create graphs in openHAB2

Persistance

To draw graphs, we need the possibility to store item states over time. This is called persistence. As I want to use the complete data later for documentation a round-robin database like rrd4j is not a good idea. So I will use a time series database named InfluxDB.

InfluxDB

Installation and starting the server

Here the installation instructions:

https://docs.influxdata.com/influxdb/v1.6/introduction/installation/

First we install the package:

    mkdir ~/downloads
    cd ~/downloads
    wget https://dl.influxdata.com/influxdb/releases/influxdb_1.6.2_amd64.deb
    sudo dpkg -i influxdb_1.6.2_amd64.deb

Then we set an environment variable (control with printenv): bash export INFLUXDB_CONFIG_PATH="/etc/influxdb/influxdb.conf"

The directories under /var/lib/influxdb are owned by root instead of influxdb, so we have to change this:

    cd /var/lib
    sudo chown -R influxdb:influxdb influxdb

To start the influxdb server:

    sudo systemctl start influxdb

Control if everything's alright (quit with q):

    sudo systemctl status influxdb
Create a Data Source

Before openHab can start persisting data in InfluxDB we have to create a database. Type Influx to access to the InfluxDB shell: bash influx

Paste the following lines to create the InfluxDB database. Memorise your own passwords!:

CREATE DATABASE openhab_db
CREATE USER admin WITH PASSWORD 'Password1' WITH ALL PRIVILEGES
CREATE USER openhab WITH PASSWORD 'Password2'
CREATE USER grafana WITH PASSWORD 'Password3'
GRANT ALL ON openhab_db TO openhab
GRANT READ ON openhab_db TO grafana
exit
Installing the openHAB 2 binding for InfluxDB

Click on the Paper UI icon (serverip:8080) to install the binding. Choose in Add-ons Persistance the InfluxDB (v 1.0) Persistance (Binding-mqtt1) and install it.

Edit the file influxdb.cfg (/etc/openhab2/services/) by changing the following lines (remove the #-sign!):

url=http://localhost:8086
user=openhab
password=Password2
db=openhab_db

Other things for our home server

Install paho-mqtt for python programs

    sudo apt install python3-pip
    sudo pip3 install paho-mqtt
    sudo apt install gnuplot

Install ssmtp for mailing

For mailing you have to install ssmtp (needed), mpack (for attachments) and berhaps mailutils (not mandatory):

    sudo apt install ssmtp mailutils mpack

With your editor you have to set up the defaults for SSMTP in /etc/ssmtp/ssmtp.conf. Edit the fields:

    root=my@mail.adr
    mailhub=smtp.xxx.xx:587
    hostname=localhost
    rewriteDomain=xxx.com
    FromLineOverride=YES
    AuthUser=youruserid
    AuthPass=xxxxxxxxxxxx
    UseSTARTTLS=YES

Test your mail with:

    echo "Hello world email body" | mail -s "Test Subject" my@mail.adr

https://github.com/airix1/openhabnano/blob/master/openhab.nanorc /usr/share/nano/openhab.nanorc

For troubleshooting:

https://www.openhab.org/docs/administration/logging.html