Running Wacom Bamboo Slate client application on Ubuntu Bionic

I have several Wacom Bamboo Slate, which is a kind of graphics tablet in different size. The most special feature of Bamboo Slate is that you could use the customized pen to draw on a real paper on the top of the tablet, and you could get the digital image at the same time. It also supports "Live mode" so the tablet could behave like a normal graphic tablet.

Unfortunately, the corresponding client application only supported on Android, iOS, Windows, and OSX. Here shows how to make it work on one of the Linux distribution, Ubuntu Bionic.

Tuhi project

There is an open source project Tuhi https://github.com/tuhiproject/tuhi is trying to support the client application of the device.

Build Tuhi

The biggest issue that may happen to you on Ubuntu Bionic is the one of the build prerequisites, pygobject-3.0 should be 3.30 or higher. The corresponding debian package on Ubuntu Bionic is
python-gi-dev, and the latest version Bionic provides is 3.26.x (2020 March). 3.30 is only available on Eoan or later.

There are two solutions. First, you may build the code in a python virtual environment, which currently provides PyGObject 3.34.0. Or tweak the source code to unlock the dependency check to allow to use elder version of pygobject. I tried the latter solution with 3.26, and both of the normal and live modes work well for me.

Run Normal Mode in a Python Virtual Environment

After building the code, you will get several runner with the suffix .devel to run the application in development mode. You may execute tuhi.devel directly.

Run Live Mode in a Python Virtual Environment

Regarding live mode, the application needs more system permission because it will create a HID device, and need to communicate with the Linux kernel to create device nodes. You may use the following command to execute the live mode runner:

sudo <your python interpreter of your virtual environment> tools/tuhi-live.py

so you could get enough permission to run the live runner.

The Difference Between Normal and Live Mode From System Perspective

The normal mode is mainly based on the session bus of dbus mechanism and GTK3. The live mode is mainly based on being a HID device of Linux kernel.

Both of normal and live mode are established by communicating with the rule of the device firmware. Check the Protocol and Interactions class of the protocol module.

Get the Correct Dimension in Live Mode

If you are using the device in the normal mode, it works like a charm. If you are using the device in the live mode, you may be aware of the distortion of what you are drawing. This is caused by the mismatch of the ratio between your monitor dimension and your bamboo slate. By setting the slate dimension it will help.

There are several ways to match the ratio. Here is mine:

• Constrain the tablet in one monitor only. This may be optional for you because I use multiple monitors.
• Remove the out-of-range part of the table panel to match the monitor dimension.

Thanks for FLOSS. We already have the corresponding tools to achieve the above tasks. I will illustrate what I used in the following sessions.

Constrain the Tablet in One Monitor Only

Firstly let's check if the device has been regarded as one of the input device of your X.

$ xinput
⎡ Virtual core pointer                    id=2 [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer              id=4 [slave  pointer  (2)]
⎜   ↳ Logitech M310                            id=10 [slave  pointer  (2)]
⎜   ↳ Logitech K520                            id=11 [slave  pointer  (2)]
⎜   ↳ SynPS/2 Synaptics TouchPad              id=15 [slave  pointer  (2)]
⎜   ↳ Wacom A4 - Office - Slate Pen stylus    id=17 [slave  pointer  (2)]
⎣ Virtual core keyboard                    id=3 [master keyboard (2)]
    ↳ Virtual core XTEST keyboard              id=5 [slave  keyboard (3)]
    ↳ Power Button                            id=6 [slave  keyboard (3)]
    ↳ Video Bus                                id=7 [slave  keyboard (3)]
    ↳ Power Button                            id=8 [slave  keyboard (3)]
    ↳ Sleep Button                            id=9 [slave  keyboard (3)]
    ↳ Chicony USB2.0 Camera: Chicony          id=12 [slave  keyboard (3)]
    ↳ Intel HID events                        id=13 [slave  keyboard (3)]
    ↳ AT Translated Set 2 keyboard            id=14 [slave  keyboard (3)]
    ↳ Logitech K520                            id=16 [slave  keyboard (3)]

Neat, we have "Wacom A4 - Office - Slate Pen stylus" as one of the input device now. The let's check with a more specific tool by:

xsetwacom list
Wacom A4 - Office - Slate Pen stylus id: 17 type: STYLUS

The listed name is exactly the input device name.

Man xsetwacom will Tell You a LOT

You may use "xsetwacom list parameters" to check your Wacom device status, and then setup them via "xsetwacom set <device> <parameter> <value>".

For example, inquiry by

$ xsetwacom get "Wacom A4 - Office - Slate Pen stylus" Mode
Absolute

Tip: "man setwacom" to know how many parameters are available.

If it is not in the absolute mode, change it by

$ xsetwacom set "Wacom A4 - Office - Slate Pen stylus" Mode Absolute

Inquiry the monitor output source:

$ xrandr -q

According to its output, we could constrain the tablet in the target monitor via

$ xsetwacom set "Wacom A4 - Office - Slate Pen stylus" MapToOutput eDP-1

You may be aware of the change of the value of "Coordinate Transformation Matrix" output by "xinput list-props <your wacom device name>" before and after the MapToOutput setting.

Finally, make sure the value of "Wacom Tablet Area" is in the 1:1 ratio of your monitor.

$ xinput list-props "Wacom A4 - Office - Slate Pen stylus"
Device 'Wacom A4 - Office - Slate Pen stylus':
Device Enabled (169): 1
Coordinate Transformation Matrix (171): 0.545455, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
Device Accel Profile (300): 0
Device Accel Constant Deceleration (301): 1.000000
Device Accel Adaptive Deceleration (302): 1.000000
Device Accel Velocity Scaling (303): 10.000000
Device Node (292): "/dev/input/event18"
Wacom Tablet Area (732): 2500, 6362, 28700, 21100
Wacom Rotation (733): 0
Wacom Pressurecurve (734): 0, 0, 100, 100
Wacom Serial IDs (465): 1, 1, 2, 0, 0
Wacom Serial ID binding (735): 0
Wacom Pressure Threshold (736): 26
Wacom Sample and Suppress (737): 2, 4
Wacom Enable Touch (738): 0
Wacom Hover Click (739): 1
Wacom Enable Touch Gesture (740): 0
Wacom Touch Gesture Parameters (741): 0, 0, 250
Wacom Tool Type (742): "STYLUS" (731)
Wacom Button Actions (743): "Wacom button action 0" (744), "Wacom button action 1" (745), "Wacom button action 2" (746), "None" (0), "None" (0), "None" (0), "None" (0), "Wacom button action 3" (747)
Wacom button action 0 (744): 1572865
Wacom button action 1 (745): 1572866
Wacom button action 2 (746): 1572867
Wacom button action 3 (747): 1572872
Wacom Pressure Recalibration (748): 1
Wacom Panscroll Threshold (749): 1300
Device Product ID (293): 1386, 1
Wacom Debug Levels (750): 0, 0

That's it. Enjoy your drawing!

Enable EDAC debug mode on Ubuntu bionic kernel

When developing Ubuntu Bionic kernel, you probably notice the EDAC, Error Detection and Correction, is not enabled by default. You may want to enable it for development. This is how it works.

Go and fetch Ubuntu bionic source code via Launchpad, and make a change as the following:

diff --git a/debian.master/config/annotations b/debian.master/config/annotations
index d4ba76f3a350..bf220bf6e729 100644
--- a/debian.master/config/annotations
+++ b/debian.master/config/annotations
@@ -1259,7 +1259,7 @@ CONFIG_OF_UNITTEST                              flag<DEBUG>
 # Menu: Device Drivers >> EDAC (Error Detection And Correction) reporting
 CONFIG_EDAC                                     policy<{'amd64': 'y', 'arm64': 'y', 'armhf': 'y', 'i386': 'y', 'ppc64el': 'y'}>
 CONFIG_EDAC_LEGACY_SYSFS                        policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'i386': 'n', 'ppc64el': 'n'}>
-CONFIG_EDAC_DEBUG                               policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'i386': 'n', 'ppc64el': 'n'}>
+CONFIG_EDAC_DEBUG                               policy<{'amd64': 'n', 'arm64': 'y', 'armhf': 'n', 'i386': 'n', 'ppc64el': 'n'}>
 CONFIG_EDAC_DECODE_MCE                          policy<{'amd64': 'm', 'i386': 'm'}>
 CONFIG_EDAC_GHES                                policy<{'amd64': 'y', 'arm64': 'y', 'i386': 'y'}>
 CONFIG_EDAC_AMD64                               policy<{'amd64': 'm', 'i386': 'm'}>
diff --git a/debian.master/config/arm64/config.flavour.generic b/debian.master/config/arm64/config.flavour.generic
index bb7773a235d2..b6d9b685a5a7 100644
--- a/debian.master/config/arm64/config.flavour.generic
+++ b/debian.master/config/arm64/config.flavour.generic
@@ -1,3 +1,4 @@
 #
 # Config options for config.flavour.generic automatically generated by splitconfig.pl
 #
+CONFIG_EDAC_DEBUG=y

For now, if you check the debugfs of EDAC, you should have /sys/kernel/debug/edac folder is generated by default.

Using command line tools proactively

I still remember the day that I firstly saw the books like "the Linux command manual". Most of them are just (or similar to) the collection of the contents from the "man" command output. I was surprised at "How a person could memorize or know so many commands and their usage." There are hundreds of commands, and thousands of their corresponding options and usages.

Many people, at least myself, figure out which command we should know and how to use it by googling a specific question. For example, google "How do I list files in a directory with Ubuntu Linux", and pick up (randomly) the first few searching results to follow. As the time goes by, I know more and more commands until I could handle most of the issues in my life.

Then the learning curve reach a plateau. I won't learn new command or new option of a known command for longer and longer period.

"How the people on the Stack Overflow know so many variant ways or commands to handle a similar problem?" I seldom (and can't most of the time) walkthrough the manual of a command, and I suppose many people don't as well. For example, the "dd" command has a lot of fancy and useful options and features to use, but the default value should work like a charm in more than 80% of life problems. If I don't know there is such a option or an extra feature to use, how am I aware that I could use them?

Besides from randomly googling and waiting for someone's answer on Stack Overflow, there is another way to think outside the box: think proactively.

To think proactively here means the following points of view have been considered in our mind:

• What problem I am going to resolve?
• What's the essential property of this problem? Does this property has similar problems as well?
• What feature this tool should has to resolve this problem essentially?
• What kind of the tool to resolve this problem may be? How this kind of tool to resolve the problem?
• What is the result after I applied the tool to the problem?

Let's take the "dd" command as an example again. Assume our problem is "to clone one disk". Then the essential property of this problem could be:

• How to clone this disk faster? - Is there any option to make it faster?
• How to clone the disk reliably? - Is there any option for me to check the status frequently?
• It is an I/O problem - There are very likely to be input and output related features.
• It is an operation on block device - I have to think form the block device point of view.

And then it could be:

• Speed: what is the potential features to make the read/wirte IO of block devices faster? - read/wirte chunk size. Error handling.
• Reliability - Is there any process status reporter? Is there any error handler?

So I might figure out the "bs" option may play a role for the speed, and there are "sync" and "noerrors". I could suppose there should be a option for progress. It is status=progress then.

This mind is similar to the mind when trying to find out an solution to a problem, answer to a question, or debug code. Essentially they are just to figure out the goal, collect the associated 

information, apply it and review the result consciously.

For example, I am working on SGE (Sun Grid Engine) infrastructure recently. I was prototyping a solution to build the infrastructure automatically with LXD/LXC. When I complete the prototype I move the solution Juju/Charms to MaaS. I was blocked then. However I could soon find the root cause out by thinking proactively, like:

• The error looks like a network issue and permission issue.
• To build a SGE scheduler is a question about communication between nodes.
• When I setup the prototype successfully, which part relates communication/networking/permission.
• Is the same step applied to the new infrastructure flow?

Then I could understand what kind of commands (qconf for example) and the associated options I may look into. : )

In conclusion, when you have some background knowledge already, try to think "the tool that I has known would be great if it could has this feature. Does it have this feature?" rather than "google the problem directly." To google a problem on the internet you could most of the time just get an entry level answer, none, or noise.

Modify casper/initrd of Ubuntu 18.10 Cosmic Cuttlefish

The article is firstly posted here https://askubuntu.com/questions/1094854/how-to-modify-initrd-initial-ramdisk-of-ubuntu-18-10-cosmic-cuttlefish/1094855 because this change is pretty new and it seems that nobody has asked on the internet. To post there should help many people in the follow months after 18.10 release.


Besides, the quote from Debian wiki is also useful as background knowledge.

• If an uncompressed cpio archive exists at the start of the initramfs, extract and load the microcode from it to CPU.

• If an uncompressed cpio archive exists at the start of the initramfs, skip that and set the rest of file as the basic initramfs. Otherwise, treat the whole initramfs as the basic initramfs.

• unpack the basic initramfs by treating it as compressed (currently gzipped) cpio archive into a RAM-based disk.

• mount and use the RAM-based disk as the initial root filesystem.

Troubleshooting - curtin version is incorrect on a MaaS region server

Few weeks ago an weird MaaS issue happened to me. When I tried to commission or deploy node with ga-18.04 kernel. The deployment cycle always stops at the grub entry, which shows "Commissioning".

After fighting for few days by stopping in the ephemeral environment when dd the customized image to the hard disk. I noticed that the well-functioned MaaS region server updates the kernel in the ephemeral environment, and the malfunctioned MaaS region server doesn't. To use the new kernel is very important for me to deploy my customized images because I need nls_iso8859-1.ko module to deal with my recovery partition. This code snippet shows how a recent curtin (18.1) updates the kernel

ubuntu@breckenridge-dvt2-201802-26115:/curtin$ grep linux-image -r *
Binary file curtin/deps/pycache/init.cpython-36.pyc matches
curtin/deps/init.py: # linux-image package for this environment
curtin/deps/init.py: kernel_pkg = 'linux-image-%s' % os.uname()[2]
def check_kernel_modules(modules=None):

if modules is None:

modules = REQUIRED_KERNEL_MODULES

# if we're missing any modules, install the full
# linux-image package for this environment
for kmod in modules:

try:

subp(['modinfo', '--filename', kmod], capture=True)

except ProcessExecutionError:

kernel_pkg = 'linux-image-%s' % os.uname()[2]
return [MissingDeps('missing kernel module %s' % kmod, kernel_pkg)]

return [] 

Thus I went to dig in curtin, which takes care of the installation/dd of images, and noticed the version of curtin differs in two different MaaS region server which are installed the same version of MaaS. By updating the curtin I fixed the issue. The mulfunctioned one uses 0.1.0 curtin, and the good server uses 18.1.

In conclusion, the curtin version of the MaaS region server matters. It seems that the curtin will map into the ephemeral environment and be leveraged. Interesting!

Summary of the Debugging Tips

• Summary of the debugging flow of this case
• stop at the grub entry
• check the previous stage and found errors in curtin stage
• compare good and bad environment to use curtin (ephemeral environment)
• identified the root cause is lack of nls_iso8859-1.ko
• notice good environment updates its kernel
• figure out the curtin source differs
• found the curtin version differs
• curtin log is valuable. Read it carefully. Check if it triggers the very first error.
• Effective Debugging: 66 Specific Ways to Debug Software and Systems by Diomidis Spinellis suggests to compare the buggy system with a well-functioned system may help. So true!

How "source activate conda-virtual-environment" works?

When using conda of Anaconda  or Mini-conda to create and manage a Python virtual environment, this kind of command is commonly used to activate and deactivate the target virtual environment:

source activate <conda-virtual-environment-name>

How does this work? Firstly we need to know:

• source is a feature of bash shell. It is equivalent to . (a dot) of dash shell.
• bash manual page says
• ... filenames in PATH are used to find the directory containing filename. ...

If you could use the command, conda, your conda bin folder must be included in the  environment variable PATH to make the command conda available to be searched and used. If you go to the same bin folder of conda path, you could see the files, activate and deactivate are in the same folder.

Thus, the command, source activate conda-virtual-environment, is actually

source <path-to-conda-bin-folder>/activate <conda-virtual-environment-name>

<conda-virtual-environment-name> is just the argument of the executable file, activate. Read the file activate would help you to understand how the virtual environment is launched/activated.

By the way, the recent conda is going to use conda activate to replace the conventional source activate.

Installer nightmare

To develop an installer or debug it could be very challenging for the sake of limited resource. Besides, long turn around time is another big big challenge as well. It could be a nightmare.

The limited resource here means:

• You have no idea where the log will be.
• You may not fetch the log you want.
• You don't manage to access the log even you know where it is.

The long turn around time here means:

• You can't reproduce the breakpoint within 5 minutes because you have to restart the machine and wait for image-level copying.

I will take an example below to elaborate the essence of an installer challenge. LAVA is a tool for debian, and I will talk about Ubuntu.

Ubuntu Desktop installer

To develop Ubuntu Desktop installer on a real machine in OEM mode. I often turn on debug mode by injecting debug parameters in the kernel parameter line and go to /var/log/installer/debug. To hardcode the frequently used parameters in the bootloader (say grub.cfg of grub) may be a good idea, because it takes much attention to input the parameters. The following parameters are the ones I used very much:

• debug -- automatic-oem-config debug

I tweak casper/filesystem.squashfs sometime as well to dump more special messages at the stage 1 of recovery. Besides, to install useful tools by choort/dpkg may be a good idea as well. A better text editor and the ability to ssh connect remotely may help me to interact with the installer and monitor the log in run time.

To leverage tweaking squashfs is useful, however it also pays off. A typical Ubuntu desktop squashfs could be 1 ~ 2G. If you are not using solid state disks, it takes a log of time to extract the squashfs, modify it, and then re-pack it back to a squashfs. Frequent flow looks like:

• sudo unsquashfs -d ./fs filesystem.squashfs (extract files)
• sudo mksquashfs ./fs/ filesystem.squashfs.mod (pack modified files)
• sudo cp filesystem.squashfs.01 ./<somewhere of your installing media>/casper/filesystem.squashfs (deploy)

An auxiliary could be an ftp server to download debug tools.

Make a unattended installation of Ubuntu server image

In this post we see how I investigated to find the target preseed file matches my requirement. This post shows the step-by-step commands in summary.

Firstly let's fetch the iso contents from the original iso image.

$ sudo mount -o loop ~/Desktop/images/ubuntu-16.04.1-server-amd64.iso ./160401-base-iso/
mount: /dev/loop6 is write-protected, mounting read-only
$ cp -rT ./160401-base-iso/ ./160401-target-iso/
$ sudo umount ~/Desktop/images/ubuntu-16.04.1-server-amd64.iso

Then let's tweak isolinux, which is used to boot the system at the very first time, a bit:

$ chmod u+w 160401-target-iso/isolinux/txt.cfg
$ vi 160401-target-iso/isolinux/txt.cfg

You may want to use the txt.cfg directly from here https://gist.github.com/tai271828/cbe426c158c68ae8f51a18b0ad26af52#file-txt-cfg

and

default install
label install
  menu label ^Install Ubuntu Server
  kernel /install/vmlinuz
  append  file=/cdrom/preseed/unattended-ubuntu-server.seed vga=788 initrd=/install/initrd.gz quiet languagechooser/language-name=English debian-installer/locale=en_US keyboard-configuration/layoutcode=us ---

You may also want to use the isolinux.cfg here as well https://gist.github.com/tai271828/cbe426c158c68ae8f51a18b0ad26af52#file-isolinux-cfg

$ chmod u+w 160401-target-iso/isolinux/isolinux.cfg
$ vi 160401-target-iso/isolinux/isolinux.cfg

set timeout as 1 (or any positive integer. 0 means waiting forever)

Lastly let's put the preseed, unattended-ubuntu-server.seed

,  pointed by txt.cfg. Please check the details of the preseed file here: https://gist.github.com/tai271828/cbe426c158c68ae8f51a18b0ad26af52#file-unattended-ubuntu-server-seed

Everything is ready! Let's generate the iso. Under the target iso folder

sudo mkisofs -r -V "UBUNTU160401" -cache-inodes -J -l -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o ~/Desktop/images/ubuntu-16.04.1-server-amd64-autoinstall.iso .

The above steps are wrapped up here https://gist.github.com/tai271828/cbe426c158c68ae8f51a18b0ad26af52#file-prepare-image-sh Tweak it to match your file hierarchy.

PS Ubuntu server by default has no tty7 (used for X to provide GUI conventionally) and use tty1.

What is Next...

The final product is an iso. You could boot it from virt-manager easily by using it as a virtual CD-ROM disk. However, in the modern world, we use live USB much more often now. To make a bootable live USB of this iso, you may achieve the goal by

sudo isohybrid ubuntu-16.04.1-server-amd64-autoinstall.iso
sudo dd bs=4M if=./ubuntu-16.04.1-server-amd64-autoinstall.iso of=/dev/sdX conv=fdatasync

How and why isohubrid works you could refer to my another blog post in Chinese http://zh-tw-tai271828.blogspot.tw/2017/07/hack-iso-usb-isohybrid.html

To investigation process to make a unattended installation of Ubuntu server image

If you are trying to find a step-by-step solution post, this post is NOT for you. Please go to here to have a step-by-step solution http://tai271828.blogspot.tw/2018/04/make-unattended-installation-of-ubuntu.html .


The longevous and respectable installer, debian-installer (d-i), provides powerful feature to customize and automate your installation by preseed, which is a configuration file to answer the questions of the installation prompt. The core question is: how do I know which question regarding which prompt, and what answer is acceptable or understandable by d-i?


If you check the d-i manual to try to answer the above question, you will find (1) the document enlists basic question-answers grouping by features (2) the groups collect basic description and may not elaborate the details for each question-answer item (3) you may customize your installation but you don't find the question-answer matches your requirement.

To overcome the lack of information, I did

(1) (default, RTFM ;) ) check the manual.[1]
(2) search example preseed file developed by others. google or check others' open source projects.
(3) find a workable benchmark. (debconf-get-selections is a useful and promising solution)


Regarding (1), I read the manual in this way very often: (a) confirm my goal ("What question I want to solve? Describe it in technical action item words.") (b) imagine what the solution may look like. What the design of the solution may be. (c) find the possible design from the manual (d) if nothing was found, skim the sub-titles of the  manual, and go back to (a)(b)(c). Read the document line-by-line is the final action which may or may not be considered to adapt.

Regarding (2), google may be useful (google "unattended ubuntu server preseed github", for example) or NOT. Targeting on open source projects and have a look of their source is more efficient in my experience, especially the project is still a working and alive project.

Regarding (3), this skill is also suggested by Effective Debugging: 66 Specific Ways to Debug Software and Systems: Item 5 Find the Difference between a Known good System and a Failing One [2]. To create the benchmark of a good system (with working answers to the questions), debconf-get-selections is a tool to dump the contents of the debconf database[3], which contains the question-answers in your system. You may want to append --installer when using debconf-get-selection to fetch the question-answers of installation stage.

A usual way looks like this:

1. Prepare a working system installed manually and answered all prompted as your wish. VM may be a good choice.
2. Login the working system. Dump the question-answers by debconf-get-selections --installer
3. Compare the question-answers or find possible question-answers against to the prompt you want to bypass automatically.

This is a bit trial-and-error flow. Setup an easy flow to repeat will be helpful very much. 

[1] Usually Appendix B. is recommended https://www.debian.org/releases/stable/amd64/apb.html.en

[2] https://books.google.com.tw/books?id=Fa6JDAAAQBAJ&pg=PT125&lpg=PT125&dq=effective+debugging+titles&source=bl&ots=moKocciySF&sig=HuoF5Dl3mJBoGf84YUXix-Hk5ic&hl=en&sa=X&ved=2ahUKEwjFwajc3c3aAhXNNpQKHa6dDocQ6AEwA3oECAAQSQ#v=onepage&q=effective%20debugging%20titles&f=false

[3] By default there is not debconf-get-selections, you may need to install it by "apt-get install debconf-utils" to get it.

Use grub to boot the target kernel after reboot

The key concept is that grub.cfg is the final product. The grub behavior will be performed according to grub.cfg (and the environment variables used by it). Thus please try to control everything with the tools of grub first. Let the tools control grub.cfg and environment variables for you.

Firstly, enable the default value is mutable: change the default value from 0 to be saved.

$ sudo vi /etc/default/grub
GRUB_DEFAULT=0

to be

GRUB_DEFAULT=0

and then give it a default value (0 here). The number "0" is the grub entry number. It could be 0, 1, 2, ...etc. depends on the order and number of your entries.

$ sudo grub-set-default 0

Update the grub.cfg and make all change effective.

$ sudo update-grub

Now your grub entries are ready to be selected. For example, select the 6th entry of a sub-menu (order number 1)

$ sudo grub-reboot "1>6"

And then reboot

$ sudo reboot

By the way titles should also work according to its GNU manual. https://www.gnu.org/software/grub/manual/grub/html_node/Simple-configuration.html

Previously it was documented the way to use entry title. While this still works it’s not recommended since titles often contain unstable device names and may be translated

Provision Ubuntu automatically with KVM and cloud-init

We usually want to prepare a fresh Ubuntu system to test our application or prepare a development environment. The following shows how to provision a fresh Ubuntu on your local machine by KVM virtualization technology.

Repository

https://github.com/tai271828/ubuntu-setup-automation

Pre-requirement

Ubuntu Xenial desktop

• libvirt.pc (provided by libvirt-dev)
• Python.h (provided by libpython3.5-dev)
• cloud-localds (provided by cloud-image-utils)
• optional: You may want to have virt-manager or virt-viewer in your system to see the provisioned system.

Setup

sudo apt-get install libvirt-dev
sudo apt-get install libpython3.5-dev
sudo apt-get install cloud-image-utils

virtualenv -p python3 venv

git clone https://github.com/tai271828/ubuntu-setup-automation.git

source venv/bin/activate

If you want to connect the KVM system later over SSH, you need to paste your public ssh key in this session (replace @@my_ssh_public_key@@ ). If you don't have a key or just want to use password to login, remove the whole session in order that cloud-init won't be confused by the invalid string @@my_ssh_public_key@@.

ssh_authorized_keys:
 - @@my_ssh_public_key@@

Run

In venv python virtual environment,

(venv) ubuntu-setup-automation/scripts⟫ cloud-localds /tmp/my-seed.img ../data/user-data && sudo cp /tmp/my-seed.img /var/lib/libvirt/images/ && sudo ./prepare-kvm-deployment && ../bin/create-instance-kvm

• cloud-localds /tmp/my-seed.img ../data/user-data
• This command prepares the image to inject user-data used by cloud-init later.
• sudo cp /tmp/my-seed.img /var/lib/libvirt/images/
• This command mv the user-data image, a.k.a my-seed.img, to the folder with libvirt accessible permission, so the user-data image could be used by qemu later.
• sudo ./prepare-kvm-deployment && ../bin/create-instance-kvm
• This command will
• Fetch the official Ubuntu iso
• Patch the iso so the iso has a preseed file to answer all questions prompted in the installation stage.
• Initialize the fresh installed Ubuntu system by cloud-init.

Check the configuration of your Jekins charm

Because of different working time zone, sometimes I have to take care of the Jenkins node deployed by Juju, which also maintained by another colleague, to support the colleagues in my time zone. What if I could not access the permission of the Jenkins node?

I take care of the MAAS server to make sure I could provide the hardware resource, e.g. physical nodes, so I could have the permission and credential of the MAAS server and its region controller. The Juju service is provided on the same server, so I could access the Juju profile to deploy the nodes in the lab for sure. The question is, what if I want to access the services deployed by the Juju?

Let's back to the Jenkins service, and take it for example. The answer is simple: check the deployment configuration yaml.

In the README  https://github.com/jenkinsci/jenkins-charm , the default user name is "admin" of the Juju-deployed Jenkins, and it is not recommended (and I think it means "not allowed") to change. Thus the rest of necessary information is to search the deployment configuration yaml hosts the password information, and pray for that the guy to deploy the service did not change its password manually by juju set jenkins password=mypassword.

Build SOLVCON development environment

There are several ways to build its development environment. We could use miniconda directly and only. Or, we could follow the travis.yaml to build the isolated development environment.


Everything comes from miniconda:

export PATH="<YOUR_MINICONDA_ROOT>/bin:$PATH"

Then further configuration to be more automatic. This step is/should be optional:

conda config --set always_yes yes --set changeps1 no
conda update -q conda
conda info -a

Change PS1 option is not necessary. You could decide if you refer it or not.

Create a conda isolated python environment <YOUR_SOLVCON_SRC>/build/env/

<YOUR_SOLVCON_SRC>/contrib/devenv/create.sh

Go to this conda isolated python environment and activate the associated virtualenv

source <YOUR_SOLVCON_SRC>/build/env/start

Please note we won't see the prompt change as what we always get via usual conda activate. Check if we have already been in the conda virtual environment by:

which python

And this python should be the one in the conda virtual environment.

Now it is time to install everything in the conda virtual environment!

<YOUR_SOLVCON_SRC>/contrib/conda.sh

If you want to use the latest pybind11 (optional):

pip install -U https://github.com/pybind/pybind11/archive/master.zip

This month I found an issue and YYC identified it is a real issue of pybind11[1].

Now all packages and libraries are ready to build SOVLCON. Let's build by[2]:

python setup.py build_ext --inplace

That is it! Let's test it a bit:

Unit tests with doctest

nosetests --with-doctest -v

Function tests. Your test machine should be a SSH server to test parallel tests locally.

nosetests ftests/parallel/* -v

In the latest step of travis.yaml, there is a release.sh.

<YOUR_SOLVCON_SRC>/contrib/release.sh

This step is optional if you run SOLVCON locally. The release.sh is for ephemeral build/test nodes. The script strips the build files and packs the necessary libraries with the SOLVCON source to distribute to the test/biuld nodes.

Time to "march" something!! : D

[1] https://github.com/pybind/pybind11/pull/836 No one but me noted the issue for one month!

[2] You may not want to perform "python setup.py install" because this will install your SOLVCON code into the conda environment. We want to keep conda and build/env virtual environment separately.

Summary

In this article we separate the conda and env virtual environment. The advantage to separate them is that we could install common packages via <YOUR_SOLVCON_SRC>/contrib/conda.sh and share them over different build/env virtual environments.

extract Debian pakcages

We can use not only ar to extract data from a Debian package, but also the following commands:

dpkg-deb -x <a-deb> <a-folder>
dpkg-deb -e <a-deb> <a-folder>/DEBIAN

then pack it back by

dpkg-deb -b <the-folder> <the-deb>

use docker to provide nfs service.

First, fetch the docker file. e.g. https://github.com/tai271828/dockerized_nfs_server

And then build the docker image by

docker build -t <the container name you want to have> .

Then fix this source code start.sh to rename mynfs to be <the container name you want to have>.

Follow the README to start the container to have the nfs service. Before starting the container, make sure you native host has the kernel which supports NFS service. Take Ubuntu 16.04 as an example, NFS service still needs the support from kernel. Use ps aux | grep nfsd on your native host to see the NFS support from kernel is already there or not. If no, you may insert the nfsd kernel module to have the NFS support from kernel:

sudo modprobe nfsd

Otherwise you may have the following error message when trying to mount the NFS folders provided by the NFS container:

mount.nfs: timeout set for Sun Nov  6 13:54:20 2016
mount.nfs: trying text-based options 'proto=tcp,port=2049,vers=4,addr=172.17.0.2,clientaddr=172.17.0.1'
mount.nfs: mount(2): Connection refused

use python to write a GUI application based on GTK+3 framework

Recently I want to have a little application and I chose this solution:

python - the programming language
GTK+3 - the GUI framework
glade - the IDE to design the layout of the GUI application

glade could dump the layout drawn with glade IDE to a file in XML, which could be understood by GTK3+, so we could speed up our designing process.

The reason I chose GTK+3 is simple. It is because the glib on my OS** requires my glade should be 3.x instead of 2.x.***

** I use Ubuntu 14.04 64 bit
*** the syntax, "import gtk", is used to support GTK+2.x, and you could refer to this stackoverflow link for the details.

---

Step 1: use glade to draw the layout and dump it to XML.

Firstly launch glade, and then create a window by clicking the button "window" of the tool panel. Please note we should give the window a name, which its default is "window1", when creating it. This name will be also used for python gtk3+ program when the program tries to access the window object.

Save after drawing and the file will be in XML format.

Step 2: Write the python code

The overview flow of the python gtk+3 framework roughly looks like the following:
Use builder to get the window object, register signal and set up configuration etc. Finally executing Gtk main loop and waiting for events.

This is a snippet code of an example:

from gi.repository import Gtk
class HellowWorldGTK(object):
    """This is a Hello World GTK application"""
    def __init__(self):
        # layout generated by glade
        layout_filename = "layout.xml"
        builder = Gtk.Builder()
        builder.add_from_file(layout_filename)
        #builder.connect_signals(self)
        self.window = builder.get_object("window1")
        if self.window:
            self.window.connect("destroy", Gtk.main_quit)
        self.window.show_all()
if __name__ == "__main__":
    hwg = HellowWorldGTK()
    Gtk.main()

layout.xml is the file name saved in step1 with glade and "window1" is the window name mentioned in step1. Please pay attention to this line to quit the GTK main loop as destroying the window.

self.window.connect("destroy", Gtk.main_quit)

Without this line, the program will keep running after launching it and there is no way to use OS signal to terminate the process.

Step 3: executing

Save the snippet in Step2 as go.py. Execute go.py.

---

Please refer to Python GTK+3 Tutorial for more details.

use RPi2 and Snappy Ubuntu Core to make a MIDI parser

Snappy Ubuntu Core is a system specific to IOT clients based on Ubuntu. There is unofficial image supporting Raspberry Pi 2. I want to use both of them, Snappy Ubuntu Core and RPi2, to make a MIDI parser and connect it to any machine. So those machines could receive MIDI signal and even more, use a instrument to output MIDI signals and control the machines via this Snappy Ubuntu Core + RPi2 MIDI parser.

Firstly, this parser must be able to receive MIDI signals from MIDI instruments. Secondly, this parser could dump the MIDI signals it received to the machine we want to control. Let's call it target machine.

get the image

First of all, of course, we need to download the Snappy image. Currently it is unofficial so it is very welcome to play around with it and, uhmm, report a bug if we find any of them. For example, I found a bug and report is as this one (LP:#1461878) which the apps will disappear by no reason. If you think you find out a bug, please have a look of the mailing list first. The mailing list is very active so it is very useful. Once you make sure there is no one has encountered the same issue, please feel free to file a bug on Launchpad and mention it on the mailing list.

After you get the image from this link, un-compress the image you downloaded and put it into the micro SD by this command.

sudo dd <the image> of=/dev/sdX bs=32M

Please note you should replace /dev/sdX with the device name of your micro SD card.

receive MIDI signals - enable the hardware

If we want to receive MIDI signals from the hardware, which is something like a MIDI instrument, we have to make sure two things work. The first one is the kernel could detect and enable to use this instrument. It is very easy to know about this information by applying this command right after you plugin your instrument into your Pi.

dmesg

we should see something like this

[44934.115258] usb 2-2: new full-speed USB device number 9 using xhci_hcd
[44934.244564] usb 2-2: New USB device found, idVendor=07cf, idProduct=6803
[44934.244573] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[44934.244578] usb 2-2: Product: CASIO USB-MIDI
[44934.244581] usb 2-2: Manufacturer: CASIO

This means our kernel found the hardware, a CASIO electronic piano, and got ready to use it. Fortunately, modern kernels support many MIDI instruments. Thanks to people contributed to kernels!

receive MIDI signals - dump the signals

Now we know our hardware/instrument is enabled and get ready to use. We are now trying to access the signals and to dump it to somewhere we could manipulate it later.

Again, thanks for the modern mature Linux kernels, ALSA has been part of the modern Linux kernels and supports MIDI handling very well. I will illustrate my study to use ALSA stack and tools in the other articles.