Reflections between PyCons – PyCon US 2018 + PyCon Canada 2018

Posted on by

I had the opportunity to go to PyCon US this year again. And I’ve been meaning to put down my thoughts about it, but I have managed to distract myself in a million different ways. However I feel I really ought to say what I learned:

PyCon US 2018

I enjoyed PyCon US this year, especially the “hallway” track, and I feel I learned from talking with vendors and going to a few of the open spaces. One of the best parts was meeting up with friends, who I only get to meet at PyCons. But you’d have to be there to understand and experience that. Also I wish I could of stayed for the coding sprints this year. While I did not watch the talks live, I am compiling a list of the talks that I enjoyed from PyCon US 2018 here. Some of the more interesting tidbits I learned:

LinkedIn is experimenting with an interesting alternative to microservices. The best way I can describe it is a single containerized codebase, with multiple entrypoints that allow for different services to run under different operational parameters. The engineering teams get to benefit from code co-location that you get from monolith. And the operations team can run the codebase as separate services that can be tweaked accordingly.

Mozilla through their experiments with web assembly is looking to running native applications in the browser. Dan Callahan’s PyCon keynote on getting Python into the browser was amazing. There is a lot of potential for web assembly, but there are a lot of unanswered questions behind it still.

I was surprised by the number of vendors who specialize in various aspects of e-commerce, who were out an exhibiting their SaaS offerings. If someone were to start a company like Points, you could do so with a smaller number of people and outsource a large number of the “non-core” elements of your business. Also Pythonistas sure love their Postgres databases, because there was quite a number of vendors working in that space too.

I got to meet and talk with the awesome Reuven Lerner, who host the amazing Freelancers’ Show. And I learned quite a bit about consulting from freelancers, and consultants who came out to Reuven’s open space. Positioning in the market is crucial, as I expected. What I didn’t expect was that I seemed to have gained quite a bit of knowledge from listening to the podcast and reading various books. Alas, it is all still theoretical rather than practical in many cases.

Python’s killer application is data science, and there is a lot of interest and work in that area. Which means while web development in Python will stay around, I believe there will be less and less emphasis on it both in the community and in PyCon. There are a lot of new and interesting coming into the field from coding schools, so only time will tell how that works out. But if you are into analysis and data science, Python will help you shine.

PyCon Canada 2018

Speaking of PyCons, Elaine Wong, who is the chair of PyCon Canada 2018, convinced me to help run the sprints for PyCon Canada 2018. This should be interesting, since I’ve never organized an event like that. But I look forward to the challenge. If you are in Toronto, and interested in helping sponsor, volunteer at or host the sprints, please contact me!

Rookeries v0.9.0 out – New UI and Live Editing of Sites + Juggling JSON with jq Book Update

Posted on by

It has been quite a while since my last update. I apologize for the long silence. I wanted to focus on getting Rookeries up and running, to the degree that I can host a website. Namely I was hoping to update the Amber Penguin Software website and the Juggling JSON with jq book companion website to use Rookeries. Unfortunately I had to re-architecture the UI and bring the CouchDB management of the site and pages up to spec. However I am happy with the overall results:

Interactive Editing of Menus

Rookeries v0.9.0 introduces support for adding, removing and reordering menu items. It took a bit to find a good simple implementation of a drag and drop able list.

]4 Version 0.9.0 introduces interactive editing of menus.

Interactive Editing of Site Headers and Footers

Rookeries v0.9.0 also extends support of editing pages to site headers and footers. It uses the same inline editor for Markdown and HTML, that the pages currently enjoy.

]5 You can now edit site headers in Rookeries

So what else changed?

  • The migration of the UX over to a sliding sidebar is done.
  • Internally redux was replaced with MobX, which is far more maintainable for me.
  • The addition of saving sites and the current selected page.

There is still a lot to do on Rookeries, such as making the CMS more robust and flexible. Adding and removing pages is still outstanding. So is selection of pages via URLs. Marking whether or not changes have been made, to prompt the user to save. Also to check that the site is not in an invalid state. However things are starting to look up.

A few people have inquired about the state of the Juggling JSON with jq book, and when will a sample chapter and table of contents will be available. I apologize for not replying directly to those emails (I will do so shortly), but I hope to get those out in the next few days, and hopefully before the end of the year. Work on Rookeries has essentially ate up all the time that I would of used for working on the book. I think going forward I will work on both in parallel.

Using Blender for Cover Design

Posted on by

One of the fun parts of writing Juggling JSON with jq, is that I can experiment with various things. From the technical writing side, working with Sphinx has forced me to learn the ins-and-outs of that technology. Also very likely I will need to get into working with LaTeX for more of the advanced PDF parts.

The book cover provided me with an excuse to work on my graphic design skills. I decided that I wanted to do something more than just use a stock photo or old-timey engraving (a.k.a. O’Reilly’s book covers). Instead I decided to use Blender to create a compelling cover image. Yes, it may sound like overkill, however using Blender harkens to past times. Before I settled on studying computer science at the University of Toronto, and working as a programmer, I considered becoming a 3D graphics artist. I played with 3D Studio Max in high school, and learned about drafting and animation. When faced with the reality of being proficient but not amazing at drawing, and the very real possibility of competing with many more talented and experienced artists in the market I decided against that career path. However this decision did not dissuade me from taking drawing in university or enthusiastically learning Maya to make an animation for a visual computing course.

So I took the excellent Youtube video tutorial by Blender Guru. After a few hours of watching the tutorial I came up with a final image like so:

Blender Doughnut scene

] Doughnut and cup scene created in Blender. Created on August 14, 2017

And boy did I learn a lot from the tutorial: navigating Blender’s confusing UI, modeling, texturing, lighting and rendering a final image.

As for the book cover I am working on a cell-shaded, low-poly scene involving penguins and juggling. The process has been pretty fun so far, even though quite time consuming. However I have finished my low-poly model of a penguin, and added rigging (internal skeleton with joints for movement):

Low-poly model of a penguin

] Low-poly model of a penguin for the Juggling JSON with jq book. Created on August 23, 2017

Rendered pose of a juggling penguin

] Rendered pose of the juggling penguin for the Juggling JSON with jq book. Created on August 23, 2017.

Juggling JSON with jq is Out Today

Posted on by

Apologies for the late post but it has been a busy day. As of today you can buy the early edition of Juggling JSON with jq on Gumroad. I am very excited since this is my first attempt at self publishing a book. Naturally the journey is still continuing as I work towards the final release of the book. All buyers of the book will get updates including the final version when it comes out.

I plan sending out an update on the book and the companion site/API that I am working on.

Getting Started with Writing a Technical ebook

Posted on by

The early release of my ebook Juggling JSON with jq comes out tomorrow! However this post is more about the process of writing the book itself.

Getting started on an technical ebook, (such as Juggling JSON with jq), requires a bit of upfront setup. On the ebook side, I decided to go the route of writing the book in Markdown, and generating the various formats using Sphinx. While I feel most comfortable using Markdown, and yet Sphinx uses reStructedText by default. So I had to coax Sphinx to accept Markdown by using a project called m2r. Generating the PDF version of the ebook took a bit to get working. Sphinx uses LaTeX to generate PDFs, and LaTeX while powerful can be clunky to work with. I wrapped everything up with an invoke script, and now I can quickly generating new versions of ebook in the various formats I want to support.

Something unique to writing technical books, is the need to have actual working examples. You can learn by reading, but working through exercises and examples re-enforces that learning. In the case of Unjumbling JSON with jq, I needed an example REST API that readers play with. I searched for some nice open APIs, but nothing seemed very compelling. Many of the open APIs require some form of user registration and non-trivial authentication method that would complicate the examples in the book. So I setup a simple demo API for the book. Thankfully with Docker and Flask, that isn’t a particularly daunting task. (Dockerizing most of my webapps definitely made my live easier overall.)

Finally using Gumroad made marketing and selling the book a lot more approachable. Getting everything setup for e-commerce is a daunting job, if you plan on doing it yourself. Thankfully for ebooks, and similar digital products, Gumroad solves most of the problems one can encounter. I definitely recommend using them if you are planning to do something similar.

Book Announcement: Unjumbling JSON with jq

Posted on by

jq is an amazing tool for querying and manipulating JSON in command-line, that I learned about from one of my good colleagues, Eric Olsen. And I feel that jq deserves a good book describing how to use this tool. Hence I am writing a book called Unjumbling JSON with jq on the topic.

As mentioned in a previous post, I originally planned on writing a single book on both jq and httpie. I divided the original book in two, because there is only a small overlap between the two. I wanted to show examples of grabbing a REST API response via httpie, and parsing the JSON output with jq. However basic querying a REST API is something that could be covered in a short section. By writing the books separately, I will be able to release them faster, and the books will be much more focused.

I plan on selling early drafts of the ebook on August 10th. Buyers of the ebook will get regular versions of the evolving drafts of the ebook, and a free upgrade to the final version of the book. I want to release the early drafts to get early reader feedback. In addition readers of the book will have access to the REST API that accompanies the book.

You can order the early version of Unjumbling JSON with jq from here].

WordPress Password Resets via MySQL

Posted on by

I have not completed the switch away from WordPress completely, (as this site and the justCheckers site show). I might switch to using something like Django CMS or Wagtail. Unfortunately I have not had as much time as I’d like to, to work on Rookeries, my own Flask based CMS

Still in the meantime, I managed to accidentally lock myself out of my own WordPress sites. Thankfully there are a few ways to reset passwords in WordPress. The most surefire way I found was to reset the password using MySQL. (Using the wp-cli tools looks interesting, but I didn’t feel like setting up get another PHP tool.) Most of the time you don’t need to do this. However if you’ve managed to forget your only admin password… well this will get you out of that problem. So what does resetting a WordPress password via MySQL, you ask?

  1. Log into your server via SSH (or to MySQL via a SSH tunnel if you have that enabled)
  2. Login into MySQL: mysql -u $MY_DB_USER -p
  3. Get the username if you have not already. You can look those up in the ${MY_WP_SUFFIX}_wp_users table.
  4. Finally reset the password, using the MD5() function and an update SQL statement: UPDATE "wp_users" SET "user_pass" = MD5('new_password') WHERE "wp_users"."user_login"= "username";
  5. Log in with the user using the regular WordPress login (e.g. https://my-wordpress.example.com/wp-login.php). Remember to change the password, as that will use a stronger hash than MD5 internally and is more secure.

And there you go!

Notify Me when Done “X” in KDE

Posted on by

One of the few Java webapps I work on at work, has a very long startup time. Unfortunately since the server startup code is proprietary and owned by the vendor, there is not much I can do about that. However it is easy to forget to check if the server has started up, I decided to that I needed a way for my computer to notify me that the webapp was up. Here is how I came up with a simple and quick way to do just that in KDE.

So my webapp has an health endpoint that can be easily queried via HTTP. With httpie the HTTP query was very easy, however to script httpie to keep querying until the result came back, meaning the server was up. At first I tried do a while with negation of the return code, and then I found on StackOverflow that the bash until command will do just that. (Without needing to figure out the appropriate negation).

The second part was figuring how to create notifications in KDE via the console. Turns out that kdialog will create both notifications and general popup alerts.

Putting the two together I came up:

until http :8080/my_health_endpoint; do echo 'Waiting...'; sleep 10; done; kdialog --passivepopup "Ready to go!" 10`

I added a sleep in there, to throttle the number of times that httpie would run. The second parameter on the kdialog dictates how long the notification popup will be around. Alternatively I could of used --msgbox if I wanted a dialog that I had to press ‘OK’ on.

Resources

Approaching Inbox Zero in Gmail

Posted on by

Earlier this year (yes I meant to send this out much earlier) I went to a meetup hosted by my local PyLadies group. There Tracy Osborn of Wedding Lovely and Hello Web App fame gave an amazing talk about marketing for developers. I was truly inspired by the talk, and I feel it was very relevant for me, especially as I try to launch my first product and startup Amber Penguin Software. I could write a series of blog posts just on the content from this one meetup alone, and I probably will over time. But today I’ll focus on one thing in particular that I’ve learned from Tracy.

Getting to Inbox Zero

Nowadays I try to not organize my day to day tasks, by either my inbox or even one of my many, many Trello boards. Rather I try to bite off a few urgent and important tasks each day. Still I end up spending time on tasks initiated from emails in my email inbox. Unfortunately my inboxes seems to fill up faster than I can manage them at times. I would love to have a clean inbox as in Inbox Zero technique but more importantly I want to be much more responsive to the emails I get. Also opening up my email can be overwhelming when I see the number of emails in my inbox.

Using Multiple Inboxes in Gmail

One of the things I learned after the meetup, while browsing Tracy’s site was a technique to get my inbox under control at least in Gmail. In essence, you need to enable the “Multiple Inboxes” lab experiment in the Gmail Labs settings. Then you need to write a few filters such as is:drafts || label:follow-up (which happens to be my filter for follow-up emails) for each particular inbox. Et voilà! You have a much more manageable inbox that is subdivided into categories, and the actions you need to take.

Where it works and where it doesn’t work

Unfortunate this technique only works in Gmail at the moment. Some other webmail providers maybe have a similar multiple inbox solution, but unfortunately ProtonMail does not but it is a suggested feature. So my ProtonMail will probably lag behind in terms of how quickly I respond, unless I or someone else implements the multiple inbox feature in ProtonMail.

However where I can use multiple inboxes like in my Amber Penguin Software email (managed by Gmail) it has drastically improved my email experience and my own responsiveness. My Gmail still needs some love to get everything under control, but once I do I will be much better at replying to emails. Ultimately this technique helps you become more confident in categorizing your email, and then acting up on it when the time comes.

Post PyCon, ebook and Freelancing Plans

Posted on by

PyCon US 2017 has come and gone, and the past few weeks have been busy with renovations and family related activities. So I have not been able to write as much as I’d like to. But now life is starting to gain some normality, and I look forward to more regular blogging.

I really enjoyed PyCon this year. I ended up spending a lot of time in the so-called hallway track, meeting old friends, new folks and learning about interesting projects and companies. I still have quite a large queue of talks to watch on Youtube on the PyCon 2017 channel. I did get to talk to Guido, and ask him about how one should approach development with the new async/await functionality in Python 3. In general, I am floored how friendly everyone was, and how many amazing conversations I had.

I stuck around for the coding sprints, as I try to for every PyCon. I worked on APIStar (a cool new simple Python 3 web framework that uses type annotations), Paramiko and Ansible, and managed to be fairly productive. A bunch of PRs were addressed in the docker_* modules for Ansible. Paramiko finally got some Flake8 love. And APIStar now has better support for reverse routing.)

Aside from meeting with people, I enjoyed spending time in and around Portland. I do have to hand it to Portlanders, they have a lot of great restaurants and micro-breweries. Some of my friends involved in Fabric fact got me on the Untappd app just to keep track of the various brews I got a chance to try out. Another pleasant surprise was the swing dancing scene in Portland, especially everyone involved with Stumptown Dance. But definitely the most fun I had was visiting some friends I made in Portland, and also hiking outside of Portland. (Namely going to the Pacific Coast, and the various falls around Multnomah Falls.)

I do miss Portland, and I do hope to be back there sometime in the near future. In the meantime, I plan on trying something new. While my attempts at building a product have been informative, they have stalled simply because of the amount of work and long time of getting to market. So I’ll be trying something new: I’m open to working freelance on mobile (Kivy), Python web app and API projects through my startup: Amber Penguin Software. Feel to contact me if you have an interesting projects that you’d like me to work on.

More important I will be working on an ebook about two modern web dev tools that I use: (jq and httpie). If there is interest in this ebook, then I have ideas for other ones on Kivy, Ansible and Docker. If you are interested, please subscribe to the mailing list using the form below and I’ll keep you up to date on my progress. I will also be doing this through my startup just to keep things organized, and to enable me to bring on people as needed.