Category Archives: Web and Cloud Technologies

The Web and the cloud are the dominant features of the software world. This is their spot on my blog.

Rookeries v0.15.0 Released – Technical Preview of the Static Site Generator

Posted on by

It turns out that rewriting a CMS into a static site generator takes a bit of time. 😉 After 3 months of silence, I can finally announce the long awaited release of version 0.15.0 of Rookeries. This release is the first fully functional technical preview of the static site generator!

You can now create, build and develop a full site with Rookeries! In fact I rebuilt the Rookeries site, using Rookeries to prove out the concept. You start by using the generator to setup a new project. Then you can add Markdown files to build new pages to an existing site, change the HTML and CSS templates, and run a build which creates a modern site. A site that in essence is a server-side rendered single-page application. So if you are running Debian or Ubuntu Linux you can try it out Rookeries by installing the deb package.

Now, this release is a technical preview release to demonstrate the concept. I plan on making the template and plugin systems more flexible. Also I want to improve the usability by adding an automatic rebuild when a page is edited. And I need to write proper documentation as well. Support for other operating systems and distributions is also in the works. However already Rookeries is significantly faster than its nearest competitor: GatsbyJS and smaller in size. And you get a nicer, more maintainable site than from many other tools.

So please take Rookeries for a spin, and let me know how you like it!

Fixing Docker on Linode (Linux v4.18.16)

Posted on by

This week as I had some downtime after PyCon Canada, I started working on resolving all the issues that I postponed. One of these issues involved applying security updates to my Linode server and rebooting the server. However when I did so… I noticed that the Rookeries site went down. When I logged into the server, I quickly found the problem: Docker refused to start after the kernel updates.

As this bug report on Docker for Linux says, there is an issue with the latest Linode kernel when it comes to OverlayFS.

This causes the containerdservice that docker-ce is dependent on to not start. When looking at the logs (using sudo journalctl -xe), you’ll see an error along the lines of:

modprobe: ERROR: ../libkmod/libkmod.c:514 lookup_builtin_file() could not open builtin file '/lib/modules/4.18.16-x86_64-linode118/modules.builtin.bin'
modprobe: FATAL: Module overlay not found in directory /lib/modules/4.18.16-x86_64-linode118)

Thankfully there is a workaround to resolve this problem. From the instructions you need to an override configuration for the containerd service:

$ mkdir -p /etc/systemd/system/containerd.service.d/
$ cat << EOF > /etc/systemd/system/containerd.service.d/override.conf [Service]
ExecStartPre=
EOF
$ systemctl reload
$ systemctl restart docker

Anyways, I hope this helps if you run into the same situation.

Getting Started with Writing a Technical ebook

Posted on by

The early release of my ebook Juggling JSON with jq comes out tomorrow! However this post is more about the process of writing the book itself.

Getting started on an technical ebook, (such as Juggling JSON with jq), requires a bit of upfront setup. On the ebook side, I decided to go the route of writing the book in Markdown, and generating the various formats using Sphinx. While I feel most comfortable using Markdown, and yet Sphinx uses reStructedText by default. So I had to coax Sphinx to accept Markdown by using a project called m2r. Generating the PDF version of the ebook took a bit to get working. Sphinx uses LaTeX to generate PDFs, and LaTeX while powerful can be clunky to work with. I wrapped everything up with an invoke script, and now I can quickly generating new versions of ebook in the various formats I want to support.

Something unique to writing technical books, is the need to have actual working examples. You can learn by reading, but working through exercises and examples re-enforces that learning. In the case of Unjumbling JSON with jq, I needed an example REST API that readers play with. I searched for some nice open APIs, but nothing seemed very compelling. Many of the open APIs require some form of user registration and non-trivial authentication method that would complicate the examples in the book. So I setup a simple demo API for the book. Thankfully with Docker and Flask, that isn’t a particularly daunting task. (Dockerizing most of my webapps definitely made my live easier overall.)

Finally using Gumroad made marketing and selling the book a lot more approachable. Getting everything setup for e-commerce is a daunting job, if you plan on doing it yourself. Thankfully for ebooks, and similar digital products, Gumroad solves most of the problems one can encounter. I definitely recommend using them if you are planning to do something similar.

Book Announcement: Unjumbling JSON with jq

Posted on by

jq is an amazing tool for querying and manipulating JSON in command-line, that I learned about from one of my good colleagues, Eric Olsen. And I feel that jq deserves a good book describing how to use this tool. Hence I am writing a book called Unjumbling JSON with jq on the topic.

As mentioned in a previous post, I originally planned on writing a single book on both jq and httpie. I divided the original book in two, because there is only a small overlap between the two. I wanted to show examples of grabbing a REST API response via httpie, and parsing the JSON output with jq. However basic querying a REST API is something that could be covered in a short section. By writing the books separately, I will be able to release them faster, and the books will be much more focused.

I plan on selling early drafts of the ebook on August 10th. Buyers of the ebook will get regular versions of the evolving drafts of the ebook, and a free upgrade to the final version of the book. I want to release the early drafts to get early reader feedback. In addition readers of the book will have access to the REST API that accompanies the book.

You can order the early version of Unjumbling JSON with jq from here].

WordPress Password Resets via MySQL

Posted on by

I have not completed the switch away from WordPress completely, (as this site and the justCheckers site show). I might switch to using something like Django CMS or Wagtail. Unfortunately I have not had as much time as I’d like to, to work on Rookeries, my own Flask based CMS

Still in the meantime, I managed to accidentally lock myself out of my own WordPress sites. Thankfully there are a few ways to reset passwords in WordPress. The most surefire way I found was to reset the password using MySQL. (Using the wp-cli tools looks interesting, but I didn’t feel like setting up get another PHP tool.) Most of the time you don’t need to do this. However if you’ve managed to forget your only admin password… well this will get you out of that problem. So what does resetting a WordPress password via MySQL, you ask?

  1. Log into your server via SSH (or to MySQL via a SSH tunnel if you have that enabled)
  2. Login into MySQL: mysql -u $MY_DB_USER -p
  3. Get the username if you have not already. You can look those up in the ${MY_WP_SUFFIX}_wp_users table.
  4. Finally reset the password, using the MD5() function and an update SQL statement: UPDATE "wp_users" SET "user_pass" = MD5('new_password') WHERE "wp_users"."user_login"= "username";
  5. Log in with the user using the regular WordPress login (e.g. https://my-wordpress.example.com/wp-login.php). Remember to change the password, as that will use a stronger hash than MD5 internally and is more secure.

And there you go!

Approaching Inbox Zero in Gmail

Posted on by

Earlier this year (yes I meant to send this out much earlier) I went to a meetup hosted by my local PyLadies group. There Tracy Osborn of Wedding Lovely and Hello Web App fame gave an amazing talk about marketing for developers. I was truly inspired by the talk, and I feel it was very relevant for me, especially as I try to launch my first product and startup Amber Penguin Software. I could write a series of blog posts just on the content from this one meetup alone, and I probably will over time. But today I’ll focus on one thing in particular that I’ve learned from Tracy.

Getting to Inbox Zero

Nowadays I try to not organize my day to day tasks, by either my inbox or even one of my many, many Trello boards. Rather I try to bite off a few urgent and important tasks each day. Still I end up spending time on tasks initiated from emails in my email inbox. Unfortunately my inboxes seems to fill up faster than I can manage them at times. I would love to have a clean inbox as in Inbox Zero technique but more importantly I want to be much more responsive to the emails I get. Also opening up my email can be overwhelming when I see the number of emails in my inbox.

Using Multiple Inboxes in Gmail

One of the things I learned after the meetup, while browsing Tracy’s site was a technique to get my inbox under control at least in Gmail. In essence, you need to enable the “Multiple Inboxes” lab experiment in the Gmail Labs settings. Then you need to write a few filters such as is:drafts || label:follow-up (which happens to be my filter for follow-up emails) for each particular inbox. Et voilà! You have a much more manageable inbox that is subdivided into categories, and the actions you need to take.

Where it works and where it doesn’t work

Unfortunate this technique only works in Gmail at the moment. Some other webmail providers maybe have a similar multiple inbox solution, but unfortunately ProtonMail does not but it is a suggested feature. So my ProtonMail will probably lag behind in terms of how quickly I respond, unless I or someone else implements the multiple inbox feature in ProtonMail.

However where I can use multiple inboxes like in my Amber Penguin Software email (managed by Gmail) it has drastically improved my email experience and my own responsiveness. My Gmail still needs some love to get everything under control, but once I do I will be much better at replying to emails. Ultimately this technique helps you become more confident in categorizing your email, and then acting up on it when the time comes.

Encrypt All the Things!

Posted on by

This week has been a rough one for me, so I’ll keep this blog entry short.

However I am pleased to announce that after a bit of struggle, I have setup HTTPS on all my sites, including this one. Thanks to Let’s Encrypt and specifically the acme-nginx project to making that possible. Getting everything setup, and automated using Ansible took a bit of work, but in the end it was worth it.

In the future I am looking forward to using the new more secure TLS 1.3 standard once it is supported more widely by NGINX and various browsers.

Avoiding Blank PHP Responses after upgrading to Ubuntu 16.04, and php7.0-fpm

Posted on by

Earlier this week I finally made the plunge to upgrade my VPS to Ubuntu 16.04. With a minor hiccup surrounding supervisord (which I probably can avoid if I go the systemd route) not being enabled at boot, the upgrade was simple for both my WSGI and Node webapps.

I can not say the same thing about my WordPress/PHP installations. (Installations that I hope to transition off to Rookeries once that software becomes more stable.) It took me a few hours to track down and resolve the problems. Hence I am posting this article, to hopefully save someone else’s time when they do the same upgrade.

Upgrading to PHP 7.0

Ubuntu 16.04 makes the switch away from PHP 5 to PHP 7. So I had to switch to php7.0, php7.0-fpm, and php7.0-mysql from their PHP 5 equivalents. The location of the running UNIX socket has changed from /var/run/php5-fpm.sock to /var/run/php/php7.0-fpm.sock, as did the PID files.

Updating the PHP-FPM configuration

Running WordPress using FPM (Fast Process Manager) and NGINX, requires turning off the path translation in php.ini file. This can be done by uncommenting the line cgi.fix_pathinfo=0 found in the configuration file /etc/php/7.0/fpm/php.ini. Again these files have moved from the old location. After you’ve done this remember to restart the FPM service using the new systemd utilities: sudo systemctl restart php7.0-fpm.

Updating the NGINX configuration and Solving the Blank Response

This is the tricky part. After updating my NGINX configurations to the new UNIX socket path, and restarting NGINX, I found that I got blank PHP responses. Everything else worked, expect that any PHP page would not render. And not render by not rendering any content in the body of the responses. That led me down a few rabbit holes, and researching how to re-architecture my setup using Docker. Eventually I stumbled across a blog entry with the solution to the blank PHP response issue.

In a nutshell, with the NGINX upgrade one of the parameters needed for FastCGI went missing namely the fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; bit. Oddly this parameter appears in /etc/nginx/fastcgi.conf and not the /etc/nginx/fastcgi_params file that I normally include in my NGINX configs. Anyways after adding this line and restarting NGINX using sudo systemctl restart nginx everything worked correctly. Below I’ve included a sample NGINX configuration that should work.

Sample NGINX Configuration /etc/nginx/site-enabled/example-site.conf

“`
server {
server_name .example-site.com;
index index.php;
root /srv/www/location_of_wordpress_install;
listen 80;

access_log      /var/logs/nginx/site-access.log;
error_log       /var/logs/nginx/site-error.log info;

location / {
    try_files       $uri $uri/ /index.php$args;
}

location ~ \.php$ {                        
    fastcgi_split_path_info     ^(.+\.php)(/.+)$;
    fastcgi_pass    unix:/var/run/php/php7.0-fpm.sock;
    fastcgi_index   index.php;
    fastcgi_param   SCRIPT_FILENAME    $document_root$fastcgi_script_name;
    include         fastcgi_params;
}

location ~* \.(js|css|png|jpg|jpeg|gof|ico)$ {
    expires         max;
    log_not_found   off;
}

}
“`

Selling Off Some Domains

Posted on by

I was going to write about why needing an app isn’t a good way to try get into business. (Hint: It is very expensive, if you can’t code it yourself.) However something else popped up, namely I am trying to sell off two domains: justcheckers.org and justcheckers.net that I own. These were for the justCheckers project, that I’m essentially shutting down. Anyways if you are interested in buying them off me, feel free to contact me.

I will say that the experience so far has been interesting. Thankfully there are some good resources for selling a domain or site. I’ve looked at two companies that handle the auction and transfer of these things Flippa and Sedo. Both are quite legitimate, but there were some bad reviews and some people claiming to have been scammed. I’m trying out Flippa first since it was founded by the fine folks at SitePoint, who I quite like for other reasons. Anyways, I’m hoping that I can find a good home for the domains.