Don’t use SessionAsync

If you are trying to write a Cinnamon extension/applet and having this error when trying to parse some data that you have fetched via Soap.SessionAsync:

Failed to convert UTF-8 string to JS string: Invalid byte sequence in conversion input

Using Soap.Session instead of Soap.SessionAsync may solve your problem.

I’ve encountered this problem when working with StackExchange API which always returns data in compressed format regardless of request headers. It turns out Soap.SessionAsync doesn’t automatically decode data according to its encoding headers. You have to initiate decoding explicitly.

Or you could just use Soap.Session which handles decoding as you would expect.

I’ve wanted to write this because due to historic reasons all of the -already small number of- Javascript Soap library usage examples is written with SessionAsync class. For simple purposes though Session class can be used which is capable of handling both synced and asynced requests.

 

Patching a Debian Package

This is my experience on patching a debian package. Once you learn how to do it, its so simple and straightforward I wanted to demonstrate.

Problem

I develop an application named SerialPlot. It’s programmed with Qt SDK. It makes use of QSerialPort module of Qt for accessing the serial port devices. When I updated my system to Linux Mint 18 which is based on Ubuntu 16.04, SerialPlot got broken. When you remove a USB based serial port device while it is still open in SerialPlot, application goes into a loop and finally crashes. After some investigation I determined this was an issue with the Qt 5.5 . Problem was already fixed in the next version of the Qt.

Solution

I don’t want to update my complete Qt installation. More importantly I didn’t want to force people to update whole Qt library, just for my application. So I decided to port the fixes back to the 5.5 version of the QSerialPort library. This library is distributed in a debian package named libqt5serialport5.

Gathering Requirements

Change to an empty directory. Because below commands will liter the place a little.

mkdir qtserialport-fix && cd qtserialport-fix

Downloading the build dependencies:

apt build-dep libqt5serialport5

Downloading the source package of the libqt5serialport5. Remember, before this step you should have enabled the source package repositories.

apt source libqt5serialport5

Installing some tools that are required to build debian packages:

apt install build-essential fakeroot devscripts

Making the Fix

Now you should have 1 directory and 3 files in your working directory.

qtserialport-opensource-src-5.5.1/
qtserialport-opensource-src_5.5.1.orig.tar.xz
qtserialport-opensource-src_5.5.1-2build1.debian.tar.gz
qtserialport-opensource-src_5.5.1-2build1.dsc

The file *.orig.tar.xz is the original source code.  *.debian.tar.gz file is the packaged debian source code file. *.dsc file is required to upload package to a debian repository. The directory should contain our source package and debian/ directory required to patch and re-build the debian package already extracted.

To patch a debian package properly we will use a tool named quilt. This is a tool specifically created to manage patches to debian packages.

Change into the source directory:

cd qtserialport-opensource-src-5.5.1/

First should apply all the existing (if any) patches.

quilt push -a

Now we create a new patch:

quilt new fix-resourceerror-on-close.diff

Add the file we want to change to current patch:

quilt add src/serialport/qserialport_unix.cpp

Now it’s time to open the file in your favorite editor and make the necessary changes. After you save your file, tell quilt about the changes.

quilt refresh

To finish patching, un-apply all patches:

quilt pop -a

Building the Package

To build the package just call:

dpkg-buildpackage

If build succeeds you should have one or more *.deb files in the parent directory as result. Now you can install your fixed package.

sudo dpkg --install libqt5serialport5_5.5.1-2build1_amd64.deb

What Next?

If you want to distribute your fixed package to others, next steps would be to upload it to a PPA (or a regular debian repository, if you have one). For this you should change the version of the package. This is done by changing the debian/changelog file. But I won’t get into the any more details.

Finally

Please note that this is not a tutorial. If you are going to do this, you should read about debian packaging. Debian wiki is a good place to start. Unfortunately good information is scattered around and it can take some time until you understand what is exactly going on. Good luck!

makestmlib is Now A Part of KiPart

makestmlib.py script hat I’ve introduced in a previous post is now a part of KiPart.

KiPart is a tool to automatically create KiCad symbols from CSV files. It has its own format for this but support for other formats can be easily added. So I did that.

You can install KiPart via pip. And here is the documentation.

Here is an example of how to use kipart to create a library file from Stm32CubeMx output;

kipart -r stm32cube -o stm32.lib pinout.csv

It’s that easy.

And unlike my makeshift solution, KiPart produces a nice looking output taking into account text sizes etc. I hope its useful to someone.

Table for Selecting STM32 F4 Timers

If you didn’t know already, STM32 F4 MCU’s can have a lot of timers. And not all of them are the same. Some are feature packed, some are 32 bits, some doesn’t support certain interrupt types. Even the most feature-full one may not have a specific function you need. In software you are free to choose any one of them you want for any purpose. But if you need timers for external IO, say for interrupt timing, pwm measurement, pwm generation etc, you have to be careful when designing your hardware layout. Because not all the timers are the same and you can connect a timer to certain pins only. Sometimes you may have 2 different pins to choose, but thats about it.

So I’ve created a table to make it easier to select timers during hardware design. This table is prepared from STM32F4 reference manual (specifically manual for STM32F405/415, STM32F407/417, STM32F427/437 and STM32F429/439 series: RM0090). Note that this information may not be valid for all STM32F4 chips. Some information may be incorrect even. Please be cautious and check your reference manual as well.

TIMER # TIM1 TIM8 TIM2 TIM5 TIM3 TIM4 TIM9 TIM12 TIM10 TIM11 TIM13 TIM14 TIM6 TIM7
Resolution (bits) 16 16 32 32 16 16 16 16 16 16 16 16 16 16
Prescaler 16 16 16 16 16 16 16 16 16 16 16 16 16 16
Count Up
Count Down
Count Up Down
Input Capture
Output Compare
PWM
One Pulse
# IO Channels 4 4 4 4 4 4 2 2 1 1 1 1
Complementary Outputs
Sync Circuit
Interrupt (Counter overflow)
Interrupt (Counter underflow)
Interrupt (Counter Initialization)
Interrupt (Trigger)
Interrupt (Input Capture)
Interrupt (Output Compare)
Interrupt (Break Input)
Trigger Input
Driving DAC
DMA trigger (counter overflow)

You can download a slightly better looking version (PDF) of this table here.

Here is the source (ODS) file. You can open it with Libre Office Calc.

Fix the sticky top navigation bars on websites

I hate sticky top navigation bars (what are they actually called?)! I just hate them. If I wanted to reach your search bar or wanted to switch to homepage I can press the `Home` button and I’m at top. Or I can flick my smooth scrolling wheel and I’m there. Or I just click my mouse wheel, with a slight mouse movement; I’m up there.

If you really want to provide some sort of instant navigation, put it in the sidebar. That’s what it’s for! Leave my vertical space for the content! It gets even worse on the mobile! Not everyone owns a giant 5.5″ mobile beast. I like my 4″ tiny screen. It’s just enough. [UPDATE: now I own a 5.5″ mobile beast, it turns out, those bars still cover a good portion of screen estate, urghh] But those idiots, put not 1 but 2 layers of sticky top panels there and drive me crazy.

I’ve just noticed my angry English sucks. So I will keep it short.

Here is a greasemonkey script that I’ve forked from this one. It tries to disable sticky top panels. Be warned though, this is just an ugly hack. It may (will) potentially break the interface in some cases.

Script for creating KiCad symbols for STM32 MCUs

ST has recently released a tool for creating projects for STM32 mcu’s called STM32CubeMX . This tool also allows you to plan your pin layout for a selected MCU part. Using the interface you see below, you can select pin features and configuration. You can also label the pins as you desire.

After you plan your pin layout, you can create a list of pins in the form of a CSV file. For this from menu select: “Pinout > Generate CSV pinout text file”. This should create a CSV file similar to this:

"Position","Name","Type","Signal","Label"
"1","VBAT","Power","",""
"2","PC13-ANTI_TAMP","I/O","",""
"3","PC14-OSC32_IN","I/O","RCC_OSC32_IN",""
"4","PC15-OSC32_OUT","I/O","RCC_OSC32_OUT",""
"5","PH0 - OSC_IN","I/O","RCC_OSC_IN",""
"6","PH1 - OSC_OUT","I/O","RCC_OSC_OUT",""
"7","NRST","Reset","",""
"8","VSSA/VREF-","Power","",""
...
...
...

Using this file and my script you can create a KiCad symbol that is specific to this pin layout in a whim. Get the script here.

When you run the script from the command line:

./makestmlib.py --name symbol_name input_file_name.csv output_file_name.lib

It should generate a KiCad symbol library which contains a symbol similar to what you see below.

As you can see output isn’t perfect. It certainly requires some edits. But it’s still very very useful.

PCB back annotation in KiCAD with Python

KiCAD already provides a tool to automatically annotate your schematics before moving on to PCB. But I prefer to to annotate my components according to their PCB position. That makes it easier to locate a certain component when you have PCB on your hand and schematic is open in the computer screen.

Some tools such as Altium, provides such function. It works like this;

  • First you annotate your components in schematic (presumably according to their position in schematic sheet)
  • You switch to PCB side, complete the placing of your components
  • Re-annotate the components on PCB
  • Back-annotate name changes to the schematic

KiCad doesn’t have this feature. So I decided to give it a try at implementing this feature with KiCad’s python scripting interface. And I’m successful (I think..).

I’ve created a script that when you run it; re-names all components on your PCB according to their PCB position. Then updates your schematic files with these changes. You don’t have to lift a finger.

First of all my script doesn’t use the KiCad’s python API directly. Instead it uses a wrapper written in python which provides a more pythonic API. It’s called kicad-python and it’s on github. Unfortunately original project had some problems, so I had to fork it and add some missing features. So you have to use my fork instead.

Below is the script. You can also find it on my github repository. Before running it make sure you BACKUP YOUR FILES.

Note that you don’t have to install ‘kicad-python’ on your system. You can use it from the downloaded location. There is only 1 dependency at the moment (other than obvious kicad installation), which you can install via pip:

  • enum34

Update: now kicad-python comes with it’s copy of the ‘enum34’ module. I’ve added it because at the moment installing python modules on kicad’s windows version seems to be a pain.

To run the script, load your board with pcbnew. Start the python console from “Tools->Scripting Console”. Make sure ‘kicad-python’ is in your path (if you haven’t installed it):

import sys
sys.path.append("/home/user/path/to/kicad-python/")

Run the script with:

execfile("/home/user/path/to/kicad-python/examples/pcbannotate.py")

After you run a script, pcbnew doesn’t update the view immediately. You can force it to redraw by cycling canvases. For ex; switch to original canvas mode then switch to opengl canvas. Let me know if you know a better method for this.

And here is the script itself. Isn’t it beautiful?

from kicad.pcbnew.board import Board
from kicad.pcbnew.layer import Layer
import re
import glob
import os
import codecs

b = Board.from_editor()

mods = list(b.modules)

def sortkeys(mod):
    if mod.layer == Layer.Front:
        return (mod.layer, mod.y, mod.x)
    else: # Layer.Back
        # Components in the back are sorted from right to
        # left according to their canvas position. When you flip the
        # board at your hand, they will be sorted 'left to right'.
        return (mod.layer, mod.y, -mod.x)

mods = sorted(mods, key=sortkeys)

ref_counter = {} # dictionary of reference names
changes = [] # a list of tuples

for mod in mods:
    prev_ref = mod.reference
    m = re.match('^((?:[a-zA-Z_\d]+[a-zA-Z_])|(?:[a-zA-Z_]+))(\d+)$', prev_ref)

    if m:
        name, number = m.groups()  # for ex: R16 -> name:'R' , number: '16'
    else:
        print("Skipping: %s." % prev_ref)
        continue

    if name in ref_counter:
        next_ref = name + str(ref_counter[name]+1)
        ref_counter[name] += 1
    else:
        next_ref = name + str(1)
        ref_counter[name] = 1

    if next_ref == prev_ref:
        continue

    print('Re-naming %s -> %s' % (prev_ref, next_ref))

    mod.reference = next_ref
    changes.append((prev_ref, next_ref))

if changes:
    b.save()
    print("PCB annotated.")

    # PCB annotation completed, now back-annotate schematics

    # prepare replacer
    changes = dict(changes)

    def replacer(match):
        return changes[match.group(0)]

    regx = re.compile('|'.join(r'\b%s\b' % k for k in changes.keys()))

    # get a list of schematic files by globbing
    directory = os.path.dirname(b.filename)
    sch_files = glob.glob(directory + '/*.sch')

    for sch_file in sch_files:
        print("Updating %s..." % sch_file)
        # read file
        s = codecs.open(sch_file, mode='r+', encoding='utf-8').read()

        # make changes
        s = regx.sub(replacer, s)

        # update file contents
        codecs.open(sch_file, mode='w', encoding='utf-8').write(s)
else:
    print("No changes were made! This is normal if your components were already named in correct order.")

Important notes:

  • Backup your files before running the script.
  • Make sure you have a recent version of KiCad (at least 4.0)
  • Script won’t rename components if their names doesn’t end with a number. For ex. something like ‘D12X’ won’t be renamed!
  • Schematics are updated with simple string replacement. So it may not work in some cases. Unfortunately at the moment KiCad has scripting only for PCB
  • After you run the script, update the netlist files. When you read netlist from pcbnew, it shouldn’t change any component footprints. Otherwise something went wrong! It may update some (automatically given) net names though. That’s expected.
  • You should really backup your files!

If you have any problems/questions with the script please let me know. I will try to help as much as I can.

SerialPlot – Realtime Plotting Software for UART/Serial Port

SerialPlot is my new open source adventure. It allows you to plot data coming through your computers serial port. To be honest, nowadays it’s most likely this will be a USB port, emulating serial port.

It supports binary and ASCII data formats. Binary formats include uint8, int8, uint16, int8, uint32, int32 and float. There is also multi channel support. SerialPlot doesn’t require a sophisticated package format. It just assumes each channels data is sent consecutively. That means all channels should be synchronized. In case of ASCII it’s different. ASCII input actually can be in CSV (comma separated values) format. Each channel is sent as a column.

Below is a screenshot of the SerialPlot:

 

Currently there is installation package for debian/ubuntu only. I hope to add windows support soon, and some other linux distributions as well. You can download debian package from here.

Source code is released under GPLv3 license. You can obtain code from my git repository over at bitbucket. I also have created a project page at hackaday.io.

If you have any trouble installing/building, please let me know.

Emacs mu4e: auto-complete mail addresses

I love auto-complete package. I wish it was more commonly used by other emacs packages. But luckily auto-complete is very easy to use(*). Below is a small snippet to use auto-complete mode for completing mail addresses when writing an e-mail with mu4e.

(defvar ac-source-contacts
  '((candidates . mu4e~contacts-for-completion)
    (prefix . "^\\(?:To\\|CC\\|BCC\\): \\(?:.*, *\\)?\\(.*\\)")))

(add-hook 'mu4e-compose-mode-hook
          (lambda()
            (setq ac-sources '(ac-source-contacts))
            (auto-complete-mode)))

(*) provided that you already have a nice list of candidates to choose from : )

Update 3/01/2015: `company-mode` which provides a completion mechanism very similar to `auto-complete` mode works with mu4e out of the box. You don’t have to configure anything. Install and enable. Although there is a graphical glitch, I find its completion better.