Build SOLVCON development environment

There are several ways to build its development environment. We could use miniconda directly and only. Or, we could follow the travis.yaml to build the isolated development environment.


Everything comes from miniconda:

export PATH="<YOUR_MINICONDA_ROOT>/bin:$PATH"

Then further configuration to be more automatic. This step is/should be optional:

conda config --set always_yes yes --set changeps1 no
conda update -q conda
conda info -a

Change PS1 option is not necessary. You could decide if you refer it or not.

Create a conda isolated python environment <YOUR_SOLVCON_SRC>/build/env/

<YOUR_SOLVCON_SRC>/contrib/devenv/create.sh

Go to this conda isolated python environment and activate the associated virtualenv

source <YOUR_SOLVCON_SRC>/build/env/start

Please note we won't see the prompt change as what we always get via usual conda activate. Check if we have already been in the conda virtual environment by:

which python

And this python should be the one in the conda virtual environment.

Now it is time to install everything in the conda virtual environment!

<YOUR_SOLVCON_SRC>/contrib/conda.sh

If you want to use the latest pybind11 (optional):

pip install -U https://github.com/pybind/pybind11/archive/master.zip

This month I found an issue and YYC identified it is a real issue of pybind11[1].

Now all packages and libraries are ready to build SOVLCON. Let's build by[2]:

python setup.py build_ext --inplace

That is it! Let's test it a bit:

Unit tests with doctest

nosetests --with-doctest -v

Function tests. Your test machine should be a SSH server to test parallel tests locally.

nosetests ftests/parallel/* -v

In the latest step of travis.yaml, there is a release.sh.

<YOUR_SOLVCON_SRC>/contrib/release.sh

This step is optional if you run SOLVCON locally. The release.sh is for ephemeral build/test nodes. The script strips the build files and packs the necessary libraries with the SOLVCON source to distribute to the test/biuld nodes.

Time to "march" something!! : D

[1] https://github.com/pybind/pybind11/pull/836 No one but me noted the issue for one month!

[2] You may not want to perform "python setup.py install" because this will install your SOLVCON code into the conda environment. We want to keep conda and build/env virtual environment separately.

Summary

In this article we separate the conda and env virtual environment. The advantage to separate them is that we could install common packages via <YOUR_SOLVCON_SRC>/contrib/conda.sh and share them over different build/env virtual environments.

extract Debian pakcages

We can use not only ar to extract data from a Debian package, but also the following commands:

dpkg-deb -x <a-deb> <a-folder>
dpkg-deb -e <a-deb> <a-folder>/DEBIAN

then pack it back by

dpkg-deb -b <the-folder> <the-deb>

use docker to provide nfs service.

First, fetch the docker file. e.g. https://github.com/tai271828/dockerized_nfs_server

And then build the docker image by

docker build -t <the container name you want to have> .

Then fix this source code start.sh to rename mynfs to be <the container name you want to have>.

Follow the README to start the container to have the nfs service. Before starting the container, make sure you native host has the kernel which supports NFS service. Take Ubuntu 16.04 as an example, NFS service still needs the support from kernel. Use ps aux | grep nfsd on your native host to see the NFS support from kernel is already there or not. If no, you may insert the nfsd kernel module to have the NFS support from kernel:

sudo modprobe nfsd

Otherwise you may have the following error message when trying to mount the NFS folders provided by the NFS container:

mount.nfs: timeout set for Sun Nov  6 13:54:20 2016
mount.nfs: trying text-based options 'proto=tcp,port=2049,vers=4,addr=172.17.0.2,clientaddr=172.17.0.1'
mount.nfs: mount(2): Connection refused

use python to write a GUI application based on GTK+3 framework

Recently I want to have a little application and I chose this solution:

python - the programming language
GTK+3 - the GUI framework
glade - the IDE to design the layout of the GUI application

glade could dump the layout drawn with glade IDE to a file in XML, which could be understood by GTK3+, so we could speed up our designing process.

The reason I chose GTK+3 is simple. It is because the glib on my OS** requires my glade should be 3.x instead of 2.x.***

** I use Ubuntu 14.04 64 bit
*** the syntax, "import gtk", is used to support GTK+2.x, and you could refer to this stackoverflow link for the details.

---

Step 1: use glade to draw the layout and dump it to XML.

Firstly launch glade, and then create a window by clicking the button "window" of the tool panel. Please note we should give the window a name, which its default is "window1", when creating it. This name will be also used for python gtk3+ program when the program tries to access the window object.

Save after drawing and the file will be in XML format.

Step 2: Write the python code

The overview flow of the python gtk+3 framework roughly looks like the following:
Use builder to get the window object, register signal and set up configuration etc. Finally executing Gtk main loop and waiting for events.

This is a snippet code of an example:

from gi.repository import Gtk
class HellowWorldGTK(object):
    """This is a Hello World GTK application"""
    def __init__(self):
        # layout generated by glade
        layout_filename = "layout.xml"
        builder = Gtk.Builder()
        builder.add_from_file(layout_filename)
        #builder.connect_signals(self)
        self.window = builder.get_object("window1")
        if self.window:
            self.window.connect("destroy", Gtk.main_quit)
        self.window.show_all()
if __name__ == "__main__":
    hwg = HellowWorldGTK()
    Gtk.main()

layout.xml is the file name saved in step1 with glade and "window1" is the window name mentioned in step1. Please pay attention to this line to quit the GTK main loop as destroying the window.

self.window.connect("destroy", Gtk.main_quit)

Without this line, the program will keep running after launching it and there is no way to use OS signal to terminate the process.

Step 3: executing

Save the snippet in Step2 as go.py. Execute go.py.

---

Please refer to Python GTK+3 Tutorial for more details.

use RPi2 and Snappy Ubuntu Core to make a MIDI parser

Snappy Ubuntu Core is a system specific to IOT clients based on Ubuntu. There is unofficial image supporting Raspberry Pi 2. I want to use both of them, Snappy Ubuntu Core and RPi2, to make a MIDI parser and connect it to any machine. So those machines could receive MIDI signal and even more, use a instrument to output MIDI signals and control the machines via this Snappy Ubuntu Core + RPi2 MIDI parser.

Firstly, this parser must be able to receive MIDI signals from MIDI instruments. Secondly, this parser could dump the MIDI signals it received to the machine we want to control. Let's call it target machine.

get the image

First of all, of course, we need to download the Snappy image. Currently it is unofficial so it is very welcome to play around with it and, uhmm, report a bug if we find any of them. For example, I found a bug and report is as this one (LP:#1461878) which the apps will disappear by no reason. If you think you find out a bug, please have a look of the mailing list first. The mailing list is very active so it is very useful. Once you make sure there is no one has encountered the same issue, please feel free to file a bug on Launchpad and mention it on the mailing list.

After you get the image from this link, un-compress the image you downloaded and put it into the micro SD by this command.

sudo dd <the image> of=/dev/sdX bs=32M

Please note you should replace /dev/sdX with the device name of your micro SD card.

receive MIDI signals - enable the hardware

If we want to receive MIDI signals from the hardware, which is something like a MIDI instrument, we have to make sure two things work. The first one is the kernel could detect and enable to use this instrument. It is very easy to know about this information by applying this command right after you plugin your instrument into your Pi.

dmesg

we should see something like this

[44934.115258] usb 2-2: new full-speed USB device number 9 using xhci_hcd
[44934.244564] usb 2-2: New USB device found, idVendor=07cf, idProduct=6803
[44934.244573] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[44934.244578] usb 2-2: Product: CASIO USB-MIDI
[44934.244581] usb 2-2: Manufacturer: CASIO

This means our kernel found the hardware, a CASIO electronic piano, and got ready to use it. Fortunately, modern kernels support many MIDI instruments. Thanks to people contributed to kernels!

receive MIDI signals - dump the signals

Now we know our hardware/instrument is enabled and get ready to use. We are now trying to access the signals and to dump it to somewhere we could manipulate it later.

Again, thanks for the modern mature Linux kernels, ALSA has been part of the modern Linux kernels and supports MIDI handling very well. I will illustrate my study to use ALSA stack and tools in the other articles.