Monthly Archives: November 2018

Onward with Postgres and Diesel – Rookeries v0.13.0

Posted on by

That took longer than expected… but I’ve finally migrated Rookeries to use PostgreSQL 10! I finished the majority of the raw migration work about a week before PyCon Canada. However I did not want to confuse my updates and tweets related to PyCon Canada with my work on Rookeries. Also I wanted to add a few bits of polish, create a mailing list for Rookeries and fix up any issues I found with the site. (Quite a few data and TLS certificate issues happened along with this release.)

I found that migrating Rookeries from CouchDB to PostgreSQL quite challenging. This adds yet another data point to those that argue that fixing data models and databases can be significantly harder than fixing applications. One of the challenges I stumbled in was mapping the various keys in a CouchDB document for a resource, into a table. Fortunately the JSONB columns allowed for the flexibility to let me resolve some of my earlier problematic data designs later on. I also needed to familiarize myself with diesel.rs, the amazing ORM/database engine library for Rust. I struggled with understanding how custom types worked in Rust, and why they make database mapping of columns easy for more advanced mappings (like optionally nullable, JSONB and Array). However I managed to figure out even the hard parts using a combination of the documentation and by peeking into the test cases for diesel. Having a migration tool in the companion diesel-cli utility, provides the nicest UX for building out database migrations and mapping them to code. Overall the effort paid off and I am convinced that Rust has one of the cleanest and maintainable ORM/database libraries I’ve seen.

As mentioned earlier, I created a mailing list dedicated to Rookeries updates. I will continue to blog about releases, but if you are interested in the internals and development of Rookeries, then this mailing list a good thing to follow. Consider it like a journal that you can also reply back to and enter in a conversation about the development of Rookeries. I added a panel to the side of the site, that makes subscribing to Rookeries easy.

This will hopefully will be the last of the major technological shifts for the project, at least on the backend. The next releases I plan on concentrating on getting Rookeries in a state where I can drive Gatsby powered static sites via REST APIs. Once that becomes a reality, I will work on replacing my WordPress sites with Rookeries powered sites.

Fixing Docker on Linode (Linux v4.18.16)

Posted on by

This week as I had some downtime after PyCon Canada, I started working on resolving all the issues that I postponed. One of these issues involved applying security updates to my Linode server and rebooting the server. However when I did so… I noticed that the Rookeries site went down. When I logged into the server, I quickly found the problem: Docker refused to start after the kernel updates.

As this bug report on Docker for Linux says, there is an issue with the latest Linode kernel when it comes to OverlayFS.

This causes the containerdservice that docker-ce is dependent on to not start. When looking at the logs (using sudo journalctl -xe), you’ll see an error along the lines of:

modprobe: ERROR: ../libkmod/libkmod.c:514 lookup_builtin_file() could not open builtin file '/lib/modules/4.18.16-x86_64-linode118/modules.builtin.bin'
modprobe: FATAL: Module overlay not found in directory /lib/modules/4.18.16-x86_64-linode118)

Thankfully there is a workaround to resolve this problem. From the instructions you need to an override configuration for the containerd service:

$ mkdir -p /etc/systemd/system/containerd.service.d/
$ cat << EOF > /etc/systemd/system/containerd.service.d/override.conf [Service]
ExecStartPre=
EOF
$ systemctl reload
$ systemctl restart docker

Anyways, I hope this helps if you run into the same situation.

Surfacing after PyCon Canada 2018

Posted on by

Uff! What a weekend! For the past few weeks, I did not write any updates because I helped organize PyCon Canada 2018! And present a talk… because apparently life is not exciting enough to do one or the other… Actually these next 2 days I am coordinating the coding sprint portion of the conference which I very much look forward to!

It was totally worth it. Even though I could of used more sleep, the conference went off without any real problems. I met a number of very interesting people, and while I didn’t get to see many of the talks, I thoroughly enjoyed helping out. Big thanks to all the organizers, sponsors and volunteers for making PyCon Canada 2018 a hit!

And I’ll be back to a (semi-)regular schedule of blogging in the next few days.