Setting up MultiTech LoRaWAN gateway on Ubuntu

As the convener for the Adelaide community of The Things Network, I am frequently setting up Multitech Conduit Gateways. Depending on your PC or notebook hardware you might have some problems with the Exar USB-UART driver on Linux.

Here are the steps to getting this unit setup from an Ubuntu (should work for any other Linux distro) machine.

lsusb

Should show something like this:

Bus 002 Device 006: ID 04e2:1410 Exar Corp. XR21V1410 USB-UART IC

Download the driver for the Exar site: https://www.exar.com/design-tools/software-drivers

unzip xr_usb_serial_common_lnx-3.6-and-newer-pak.zip
cd xr_usb_serial_common_lnx-3.6-and-newer-pak
make
sudo insmod ./xr_usb_serial_common.ko

Ensure driver is loaded at startup

sudo vim /etc/modules
#Add the following
xr_usb_serial_common

ls /dev/tty*

should now show another USB port ttyXRUSB0

For the rest you just need to follow the excellent instructions by Jac Kersing who maintains the Multitech TTN installer and documentation here:
https://www.thethingsnetwork.org/docs/gateways/multitech/mlinux.html

Get EUI of your gateway

mts-io-sysfs show lora/eui 2> /dev/null | sed 's/://g'

Moving to KVM virtual machines

Installing VirtualBox is getting increasingly painful on Ubuntu due to the problems with UEFI Secure Boot and the VirtualBox kernel modules. Another reason for an alternative is that running VirtualBox VM’s completely in the background is not as straightforward as it could be.

From the available alternatives I looked into (VMWare, Xen & KVM) it was KVM that fitted my needs (casual VM usage with mostly headless VM’s for testing purposes). Main reasons:

  • Well supported by Ubuntu
  • Easy, straightforward install
  • Background VM’s are simple as
  • Moving VM’s from one host to another is a breeze

Checking system

To check if the CPU can actually support

egrep -c '(svm|vmx)' /proc/cpuinfo

If the number returned is > 0 your systems should be capable to run.

You will also enable your BIOS for virtualisation (in Security settings of most BIOS’s) if that has not already be done. You will get an error if not enabled if you are trying to run an install. The Install of KVM will work fine.

Installation

sudo apt-get install qemu-kvm libvirt-bin bridge-utils virt-manager
sudo addgroup libvirtd
sudo adduser libvirtd
sudo service libvirtd start
sudo service libvirtd status
sudo virt-manager

Moving VM’s to another host

Source Host

virsh shutdown VMNAME
virsh dumpxml VMNAME > /tmp/VMNAME.xml
scp /tmp/VMNAME.xml TARGETHOST:/tmp/VMNAME.xml
scp /var/lib/libvirt/images/VMNAME.qcow2 TARGETHOST:/var/lib/libvirt/images/VMNAME.qcow2

Target Host

virsh define /tmp/vm.xml
virsh start vm

Once you have confirmed operation you probably want to remove the source VM from the Source Host.

virsh undefine VMNAME
rm /var/lib/libvirt/images/VMNAME.qcow2

Microchip LoRaWAN Development Utility on Ubuntu

Having just wasted a few hours on getting this Java software running on Linux I am documenting this for future reference and hopefully saving other LoRa / TTN folks some time.

Prerequisites

Install a Java JDK + JavaFX. This should work with the default OpenJDK 8 or 9 which comes as part of the Ubuntu repositories. I ended up installing Oracle JDK 8 as well as I thought the error might be related to OpenJDK.

sudo apt install openjdk-8-jre-headless openjfx

Download & install utility

Download location:
LoRa® Technology Evaluation Kit

cd ~/Downloads/
chmod +x LoRaSuite-linux-1.0.run
./LoRaSuite-linux-1.0.run

Fix User Preferences

This step is required for the Utility to run. Unfortunately, this is documented NOWHERE…

Change the following files to include the FilePath entry. The map is empty by default.

  • /home/USERID/.java/.userPrefs/dfu/prefs.xml
  • /home/USERID/.java/.userPrefs/fed/prefs.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE map SYSTEM "http://java.sun.com/dtd/preferences.dtd">
<map MAP_XML_VERSION="1.0">
  <entry key="FilePath" value="/home/USERID/"/>
</map>

If you chose the default install location you can now start the utility

java -jar ~/Microchip/LoRaSuite/Applications/LoRaDevUtility/LoRaDevUtility.jar

A big thank you goes to The Things Network user JBI – who provided this answer in a TTN forum post.

No thanks to Microchip as their forum and firmware release policy is a bit of a shocker. There are several reports of such problems in the forum with no answers. Unfortunately, this seems to be no exception with electronics manufacturers (Hello Kerlink!).

Display your Flickr Favourites as Screensaver Slideshow

Install XScreenSaver and remove Gnome default

sudo apt remove gnome-screensaver
sudo apt install xscreensaver xscreensaver-gl xscreensaver-gl-extra

Run the Screensaver UI and configure

In the “Advanced” section enter your Flickr RSS URL in “Choose Random Image”

XScreenSaver Ui
XScreenSaver Ui

https://api.flickr.com/services/feeds/photos_faves.gne?id=YOURFLICKRUSERID #replace with your Flickr User ID

Create a systemd user service to autostart

mkdir -p ~/.config/systemd/user/
vim ~/.config/systemd/user/xscreensaver.service

Past the following
[Unit]
Description=XScreenSaver
[Service]
ExecStart=/usr/bin/xscreensaver -nosplash
[Install]
WantedBy=default.target

Start and enable systemd user service

systemctl --user enable xscreensaver
systemctl --user start xscreensaver

To copy the settings (including RSS URL) onto other PC’s or re-install it might be a good idea to backup or copy the contents of ~/.xscreensaver

GrovePi Zero – connecting your IoT sensors

I recently purchased a GrovePi Zero and expected this to be a reasonable straight forward way to connect Grove sensors to your Raspberry Pi, read sensor values via Python and pushing them upstream via MQTT. However the software side of things turns out anything but straight forward. Most of the suggestions on the Dexter Industries forum suggest to download some custom OS image – WTF? Hopefully this will save some people time to chase down the same rabbit holes…..

The problem starts with the install scripts which (a pet hate of mine) assume the /home/pi account to be present. There is a variable in at least the 2 of the install scripts (another Github repository – one calls another from a Github URL – like that will ever work ???) which allows you to change the user account but there is also hard-coded references all over the place that still point to that user account. Trying to step manually through the dependency hell in these install scripts that call other scripts ended to be a complete nightmare. It seems to download specific (old !?!) versions of libraries all of which have perfectly fine binaries in the Raspbian repos. Most of the troubleshooting tips use the UI which is not much good on a headless system.

Since all I wanted was to read the sensors from Python I found that there was a library on PyPi (https://pypi.python.org/pypi/grovepi) which should actually do this without installing all this redundant dependencies on what is essentially a remote sensor aggregator. Got hopes up too quickly. Unfortunately turns out that the firmware on these devices is really old and riddled with bugs (could not read from most sensors I tried and all were known issues fixed in later firmware releases. Trying to manually install some of the dependencies (such as avrdude which are all in the Raspbian repositories) for the simple firmware update script turned out useless as they use a custom ‘gpio’ programmer not present. If you are splitting things into multiple Github repositories (which is in itself a good idea) why don’t be consistent about it and create one for the firmware updater alone? You shouldn’t have to compile half of Github on a low-powered device just to flash a firmware file.

Looks like the only solution is to bite the bullet and get another SD card with a default Raspbian image and go through the whole painful experience of installing the whole dependency hell and compiling everything on a RaspberryPi Zero which is like watching paint dry. Hopefully this should then just work on another device that only needs the Python library as a dependency.

Let’s hope so… [will update once finished jumping through all the hoops]

Set up a Raspberry Pi Zero headless

If you are using the GUI (Raspian full download) and want to connect your RPi Zero to a keyboard and monitor there are probably easier ways to do this. These notes are for people that want to use a headless (no monitor and GUI) setup ready to connect to your RPi after first boot via SSH from another terminal.

Download Raspbian Lite

wget -O raspbian-lite-latest.zip https://downloads.raspberrypi.org/raspbian_lite_latest

Download link: https://www.raspberrypi.org/downloads/raspbian/

Write Image to SD Card

dd bs=4M if=2017-08-16-raspbian-stretch-lite.img of=/dev/sdb

After this step there should be 2 additional mounts (if not mount the 2 SD card partitions manually).

Set up network interfaces

Create a new config file for the wireless interface.

cd etc/network/interfaces.d/
vim wlan0.conf

Add the following to the new file (if you want to use DHCP – change to static if you want to fix the IP):

auto wlan0
allow-hotplug wlan0
iface wlan0 inet dhcp
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

Set up connection details for local wireless network

vim etc/wpa_supplicant/wpa_supplicant.conf

Check the file and ensure the network settings are as per the Wireless Network you are connecting to.

country=AU
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
network={
ssid="SSID"
psk=HASHED_PSK_VALUE
}

You could use the clear-text PSK passphrase but I prefer not to do that in configuration files. Create a PSK hash using ‘wpa_passphrasethis older article

There could be other config values you might need depending on the network you are connecting to. More info: https://linux.die.net/man/5/wpa_supplicant.conf.

Ensure there are DNS Servers configured

sudo vim etc/resolv.conf

Add the following to the new file (or your own nameservers):

# Google's public DNS servers
nameserver 8.8.8.8
nameserver 8.8.4.4

Enable SSH Access

Create an ssh empty file in the mounted boot partition. This is necessary to enable SSH access as SSH is by default disabled on more recent versions of Raspbian.

cd boot/
sudo touch ssh

Boot RPi

Boot the Rpi by powering up and after ~30secs you should be able to connect to the IP assigned. Check your router or ise ‘nmap’ or similar to check the assigned IP:

sudo nmap -p22 -sV 10.1.1.0/24

Default login details are
UID: pi
PWD: rasbperry

My normal further setup steps for RPi 3 are here.

EDIT [2017-08-21]: One of the things necessary on later RPi’s is to turn off power saving mode on the WLAN interface.

sudo iw dev wlan0 set power_save off

Connecting your LoPy to The Things Network in Australia

EDIT [2018-06-05]: I have updated the code with the Firmware 1.18.+ releases. The code is available at our Growing Data Foundation Github.

These notes are to assist Australian IoT enthusiasts to get started in connecting a LoPy to The Things Network as it is unfortunately (not yet) straight forward to make them work with the current AU-915 TTN Channel plans. As the initiator of the local Adelaide Community of The Things Network I have been experimenting with a number of devices to connect sensors to #TTNADL. One of my personal favourites is the Pycom LoPy as a nice middle-ground between capabilities and technical complexity.

The Things Network Adelaide

However I ran into a problem where the LoPy would not get a signal back from the TTN network when joining over OTAA even though the TTN Console (http://console.thethingsnetwork.org/) the device showed as connected. With some friendly help from Jose Marcelino at the Pycom Forum this turned out to be an issue with the 915MHz frequency regions. Since there is no actual a standard governing which of the channels are used by TTN gateways, what is outlined here is what is implemented by most TTN communities down under (I know that at least ADL, BNE, SYD and WOL adhere to those). This is the typical Sub-band 2 (Channel 8-15) implementation of AU ISM 915 with TTN Gateways.

This channel plan is also implemented by the install script for the MultiTech Conduit Gateways which are currently the most common gateways installed across Australia (see https://github.com/TheThingsNetwork/gateway-conf/blob/master/AU-global_conf.json)

Channel# Direction Frequency MHz Bandwidth kHz Data rate
8 up 916.8 125 DR0 – DR3
9 up 917.0 125 DR0 – DR3
10 up 917.2 125 DR0 – DR3
11 up 917.4 125 DR0 – DR3
12 up 917.6 125 DR0 – DR3
13 up 917.7 125 DR0 – DR3
14 up 918.0 125 DR0 – DR3
15 up 918.2 125 DR0 – DR3
65 up 917.5 500 DR4
0 down 923.3 500 DR8 – DR13
1 down 923.9 500 DR8 – DR13
2 down 924.5 500 DR8 – DR13
3 down 925.1 500 DR8 – DR13
4 down 925.7 500 DR8 – DR13
5 down 926.3 500 DR8 – DR13
6 down 926.9 500 DR8 – DR13
7 down 927.5 500 DR8 – DR13

To set the LoPy unit up it needs to have the set frequency plan removed and the region specific frequency plan loaded (even though they were bought as 915Mhz units). I have created some quick Python code to set up the LoPy with the above channel structure as the default settings for those devices do not work. Note that you have to remove the default channel settings and add the correct ones before you can successfully register and send data.

Below is my adaptation of the LoPy LoraWAN example to work with TTN in Australia. Please note that the LoPy Lora class only accepts DR values from 1-7. Suggestions more than welcome.

Hope this helps other LoPy owners in Australia to connect to The Things Network. And if you are in Adelaide why don’t you come along to our OpenData and IoT Meetup ? Let’s build this thing together.

Using DNSMadeEasy as Dynamic DNS provider on Synology Diskstations

Since Synology (despite requests) still has not added DNS Made Easy as a listed provider (despite listing some really obscure services – go figure!) here is the steps to add a custom provider.

DNS Made Easy Setup

Create a new A-Record

  1. Set the name
  2. Set the IP (initial – any valid IP)
  3. Tick the “Dynamic DNS” tickbox
  4. Enter your chosen Dynamic DNS Password
  5. Save the new record

When saving the record you will see a “Dynamic DNS ID” – note down this number. This will become the hostname on the Synology setup.

Synology Setup

Click “Customize” to add a new DDNS provider

Name: DNSMADEEASY
Query URL
http://cp.dnsmadeeasy.com/servlet/updateip?username=__USERNAME__&password=__PASSWORD__&id=__HOSTNAME__&ip=__MYIP__

Click “Add” to add a new DDNS service

  1. Service Provider: *DNSMADEEASY
  2. Hostname: Dynamic DNS ID from DNS Made Easy
  3. Username/Email: your DNS Made Easy email
  4. Password/Key: your chosen DNSMadeEasy DDNS password

Once you save the new DDNS provider you should see the status to go “Normal” in a green color. This means the update was successful. You should now be able to PING the DNS record or if you log in to DNS Made Easy the IP address should have changed to the external IP of your DiskStation.

Getting Fujitsu ScanSnap S1300i to work on Ubuntu 16.04LTS

ScanSnap 1300i

Upgrade or install SANE backends

Since the version of SANE in the Ubuntu 16.04LTS repos is not working for this scanner you either need to install from sources (see this blog) or from this PPA.
sudo add-apt-repository ppa:rolfbensch/sane-git
sudo apt update
sudo apt install sane-backends tesseract-ocr gscan2pdf

Security

Add yourself to the ‘scanner’ group to be able to use the scanner.

sudo usermod -a -G scanner USERNAME

Checking SANE

Check for the libsane version (needs to be at least libsane.so.1.0.26 not libsane.so.1.0.25 which is in the Ubuntu repos)

sudo ldconfig -v | grep libsane

This should show something like: libsane.so.1 -> libsane.so.1.0.26

Add udev rules for the scanner

sudo vim /etc/udev/rules.d/79-scanner.rules
#add the following
Fujitsu ScanSnap S1300i
ATTRS{idVendor}=="04c5", ATTRS{idProduct}=="128d", MODE="0664", GROUP="scanner", ENV{libsane_matched}="yes"

Get the firmware for the FUJITSU ScanSnap S1300i

sudo mkdir /usr/share/sane/epjitsu
cd /usr/share/sane/epjitsu
sudo wget https://github.com/ckunte/scansnap-firmware/raw/master/1300i_0D12.nal

Testing

scanimage -L

>> device `epjitsu:libusb:002:027′ is a FUJITSU ScanSnap S1300i scanner

You can now use gscan2pdf to scan, clean and OCR from the ScanSnap S1300i.

RaspberryPi Version 3 SOE

Since there is now a supported Raspbian version without GUI and other unneeded add-ons available as Raspbian Lite the need to use other installers (with sometimes some downsides) is now not a necessity anymore. Below is a list of steps I like to perform before using them for any purpose as my Standard Operating Environment.

Download Raspbian Lite

Download link: https://www.raspberrypi.org/downloads/raspbian/

Write to SD Card

dd bs=4M if=2016-05-27-raspbian-jessie-lite.img of=/dev/sdb

Boot RPi

Default login details are
UID: pi
PWD: rasbperry

Regional settings

locale-gen en_AU.UTF-8
dpkg-reconfigure locales
ln -sf /usr/share/zoneinfo/Australia/Adelaide /etc/localtime

System update

apt update && apt upgrade
apt install vim

Setup SSH keys

As a security precausion it’s a good idea to disable password authentications
ssh-keygen -t rsa
vim /root/.ssh/authorized_keys
# --> add RSA public key

Setup Wireless LAN

iwlist wlan0 scan
wpa_passphrase SSID WPA_KEY
vim /etc/wpa_supplicant/wpa_supplicant.conf

Copy the resulting hash from the previous command

Example config

country=AU
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
network={
ssid="YOURSSID"
psk=123456789012345678901234567890
}

Rename default user account

The default Raspbian images use the default user ‘pi’ – I prefer to change this user to another account for security purposes and to be in line with other Linux system I use.

Reboot and log in to the RPi as root – you need to run the following commands as root.

usermod -l NEW_USER_ID pi
usermod -m -d /home/NEW_USER_ID NEW_USER_ID
groupmod --new-name NEW_USER_ID pi
mkdir /home/NEW_USER_ID/.ssh/
vim /home/NEW_USER_ID/.ssh/authorized_keys
# --> add RSA public key
# change owner and permissions on key files
chown -R NEW_USER_ID:NEW_USER_ID /home/NEW_USER_ID/.ssh/
chmod 700 /home/NEW_USER_ID/.ssh/
chmod 600 /home/NEW_USER_ID/.ssh/authorized_keys
rm /etc/sudoers.d/010_pi-nopasswd
echo 'NEW_USER_ID ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/010_NEW_USER_ID-nopasswd

If you are using the desktop version not a headless version you want to make sure you don’t boot into the desktop and if you want to change the autologin ensure you update the username there as well.

/etc/systemd/system/autologin@.service
/etc/lightdm/lightdm.conf

Configure Raspberry Pi settings

raspi-config
I generally need to enable SPI, change the hostname and others depending on the purpose of the unit.

EDIT [2017-08-21]: With the update of Raspbian to Stretch you now have the firmware updater as part of the Raspbian repos:

sudo rpi-update

This will update the Pi Firmware to the latest version as well – which is worth it if you use it for later cameras and peripheral usage.

EDIT [2017-12-27]: change annoying vim defaults in Debian Stretch onwards. Visual mode.

sudo vim /usr/share/vim/vim80/defaults.vim

Comment out the following lines:
“if has(‘mouse’)
” set mouse=a
“endif