On Increasing Creative Output

This week I am juggling a number of things related to PyCon Canada, and life
in general. Also I came to the realization in the past few weeks, that if I
want to effectively work on long term projects like Rookeries, writing
about tooling, working on my side adventure, et cetera, without burnning
out I need to have the time, space, and health to do so effectively in the long
run.

Still it felt wrong to break my writing/publishing schedule, just because I am
figuring things out. After I want to work on my output consistency while still
performing my normal day-to-day activities. Rather than force myself to rush
through the more technical post on Rookeries, I will something simpler.
Today I want to discuss some guidelines that have I adopted that help me get
better at creative work. Since both technical writing and programming are
ultimately creative work, the ideas I want to look at apply to them as well.

Schedules and Commitments

Creative types (e.g. writers and coders) often avoid schedules like the plague.
Yet I would agree you need both schedules and commitments (i.e. deadlines).
These push you to perform and ship real work. The more often you ship, publish,
release, so on, the more experience you gain and the better you become as a
professional creative. You need that feedback from your audience, client or
end-user, and you can only gain that by releasing your work to them.

Often you need external motivation to get something out there. Otherwise
you may engage endlessly in either procrastination or polishing something that
is already ready. Consistent commitments (like my own weekly journal post) help
you build that habit of shipping regularly. I would argue that regular shipping
is what differentiates a true professional creative and a wannabe. So embrace
those schedules and commitments!

Good Workflows and Environments Help You Get into the Flow

Creative output is the greatest when you are in the flow of things. Output
becomes natural, and you become more productive. However distractions can
easily pop you out of flow. If your tools, workflow or work environment
become the reason for distractions, then you will have a rough time creating.

Investing in a good workflow–tools and process that make you effective–pays
large dividends. In my case I figured out that Markdown and a good text editor,
makes me the most productive when writing. The ultimate reason for my creating
Rookeries is to create a better workflow for me when I work on websites.

Also finding the right environment to work improves creative output.
Environment includes both the physical and mental space. Hence putting on
the right music gets me into the flow than listening to the ambient noise of
an open office. (Actually I have a whole rant on why open office spaces are
the worst for creative work but that will be for another time.) Unfortunately
I do not currently have the luxury of adapting my work environment too much,
but if you have that option you should take it.

Stay Healthy

Staying healthy is one area that I often neglect to think about as a creative.
I get too caught up in the work, to remember to care of myself as well as I
ought to. The truth is that you can not consistently crank out great work when
you are sick, tired, unfocused, under the influence, or distracted by other
issues. Even if you try to push through say a cold while doing coding, you
generally find the results less than satifactory after the fact.

Physical fitness is important since you are human and you have a physical
component. That is why regular exercise, healthy food, regular rest, etc.
help with creative productivity. If you are not healthy, you will not have the
good enough energy levels for creative work.

Also as a human creative, you need to work on your mental and spirtual fitness
as well. Very few creatives can work outside of a supportive environment. I
could not imagine being productive without the support of my family, friends,
and colleagues. Also the less worries you have, the easier it will be to focus
on creative work.

Summary

I will summarize simply saying that if you want to maintain a creative
profession (writing, art, music, programming, etc.), you need to stick to
schedules and commitments to ship work regularly. Investing in a good
environment to work in will pay huge dividends. And stay healthy so that you
can continue to do the work you love for many happy years to come.

Rookeries v0.14.0 Released – Tech Preview of Plugins

I’m excited to announce today’s new release of Rookeries: version 0.14.0. This release previews support for plugins!

After quite a lot of experimentation, and I was able to create a tech preview of a plugin architecture. The server can now find and load frontend plugins. And the frontend plugins now extend the markdown editor, and provide new web components on a page. While the current plugins are essentially tech demonstrators, and the server and admin console need work, I am confident that this architecture will enable Rookeries to move to the next level.

What is the next level? I want to make Rookeries the most usable, extendable Markdown based CMS, targeted toward more technical users such as developers, and technical documentation writers. Rookeries will focus on enabling the creation of web site by writing Markdown, styling with CSS and using plugins for more complex additions: such as SEO, custom markup, image resizing, et cetera.

As for my next steps, I will be working in 3 areas:
* Reducing the size of the Rookeries Docker image (currently at a massive 4 GB).
* Refactoring the main app and the admin console to take advantage of plugins.
* Creating an experience for using plugins that rivals WordPress.

There is still a long way to go, however I am confident that Rookeries will be usable for production (outside of hosting its own site) before the end of the year.

Mailing List Migrations

Hello dear reader!

I hope you are having a wonderful year so far! You may have noticed a small change in the sidebar menu, where there used to be 2 sign-up forms. I removed those as there are dedicated pages to the signups now, and the sidebar was just getting quite busy. (I plan on doing a major redesign of the site this year. Stay tuned for details.)

Also you may have noticed that I have not posted a update recently. This is because I dedicated the end of last year and the beginning of the new year to organizing myself better. I hope to write more this year, maybe even on a regular schedule. While I might not be able to emulate Jonathan Stark’s one update a day for his mailing list, as I dear reader, unlike Jonathan I still have a regular full-time day job that I need to plan around. I do plan on updating my mailing lists for Rookeries and Juggling JSON with jq e-course once every 2 weeks. If I can spare the time, I want to return to releasing a blog entry on a more regular basis.

If you are wondering how I can manage all of this, I will say that Drip makes email list management painless. (It even managed to save me from some junk e-mail addresses which Gumroad did not…) Like with any sort of automation, it took me some time to set it up and understand how everything fit together. I hoped that more people would of migrated over from my Gumroad mailing list, but there I don’t know how much of those emails were junk. Most of them were, which gave me a lot of false hope about having an overwhelming crowd lining up to buy my e-course on jq. Drip gives me a much more realistic view. But I am still optimistic. Overall I am quite happy. What would make me even happier, dear reader, is if you signed up for one of my organic, free-range, and informative mailing lists! But no sweat!

Surfacing after PyCon Canada 2018

Uff! What a weekend! For the past few weeks, I did not write any updates because I helped organize PyCon Canada 2018! And present a talk… because apparently life is not exciting enough to do one or the other… Actually these next 2 days I am coordinating the coding sprint portion of the conference which I very much look forward to!

It was totally worth it. Even though I could of used more sleep, the conference went off without any real problems. I met a number of very interesting people, and while I didn’t get to see many of the talks, I thoroughly enjoyed helping out. Big thanks to all the organizers, sponsors and volunteers for making PyCon Canada 2018 a hit!

And I’ll be back to a (semi-)regular schedule of blogging in the next few days.

Qt + QML Experiments

At the end of last year, I took a bit of a hiatus from blogging and my other projects. I found that I needed a break from my current projects, and to work on something totally unrelated. One of the things I try to do each year is to challenge myself to learn a new skill. Early last year, I took to learning Javascript properly. Through my work at Points (especially on the redesign of the SouthWest Rapid Rewards inline Buy app) and on Rookeries, I got significantly better at Javascript and frontend web development. I also briefly played around with writing server-side apps in Node.

So I challenged myself in December to learn to develop apps using C++ and Qt. Using the Game Programming in Qt book and Cherno’s Learning C++ YouTube series, I managed to learn the basics of Qt and C++. In fact I was very pleasantly surprised. Learning C++ can be hard but I got it. Maybe as the cppcon 2015 talk about learning C++ points out, that learning C++ is hard, because learning old K&R style C first and then getting into C++ is hard. (Pointers are hard, especially if you get into pointer to pointer to array of pointers hell.) But it doesn’t have to be that way. The Qt book and working on visual and projects that I never imagined I’d ever work at, really motivated me to learn. Below I’ve included screenshots of some of the projects I worked on:

While I am far from calling myself an expert in Qt. C++ can be needlessly complex when use the old idioms and the community seems to be full of overly opinionated people, I am so glad I took the effort to learn C++. Lots of the memory management and macros in Rust, now make sense. Thanks to be Qt book, I understand whyy developers prefer embedding Javascript or Lua over Python in the game engines and embedded devices. (Writing a Python to C++ two-way translation layer is tedious. And this is without worrying about getting the Python standard library working as well.) Also I am pretty convinced that Qt with QML is the state of the art when it comes to UX development. (React and Kivy are far cries in comparison, not that either should be discounted as important technologies.)

Now I am inspired to delve more into native development. I plan on bringing back justcheckers as a Qt/QML based game. I will see if I can help out with some aspects of Qt, Rust and CPython. I am excited not only because I essentially achieved a New Years resolution in the first 2 months or so. But also because actual native development is now something I can do and understand.

Reflections between PyCons – PyCon US 2018 + PyCon Canada 2018

I had the opportunity to go to PyCon US this year again. And I’ve been meaning to put down my thoughts about it, but I have managed to distract myself in a million different ways. However I feel I really ought to say what I learned:

PyCon US 2018

I enjoyed PyCon US this year, especially the “hallway” track, and I feel I learned from talking with vendors and going to a few of the open spaces. One of the best parts was meeting up with friends, who I only get to meet at PyCons. But you’d have to be there to understand and experience that. Also I wish I could of stayed for the coding sprints this year. While I did not watch the talks live, I am compiling a list of the talks that I enjoyed from PyCon US 2018 here. Some of the more interesting tidbits I learned:

LinkedIn is experimenting with an interesting alternative to microservices. The best way I can describe it is a single containerized codebase, with multiple entrypoints that allow for different services to run under different operational parameters. The engineering teams get to benefit from code co-location that you get from monolith. And the operations team can run the codebase as separate services that can be tweaked accordingly.

Mozilla through their experiments with web assembly is looking to running native applications in the browser. Dan Callahan’s PyCon keynote on getting Python into the browser was amazing. There is a lot of potential for web assembly, but there are a lot of unanswered questions behind it still.

I was surprised by the number of vendors who specialize in various aspects of e-commerce, who were out an exhibiting their SaaS offerings. If someone were to start a company like Points, you could do so with a smaller number of people and outsource a large number of the “non-core” elements of your business. Also Pythonistas sure love their Postgres databases, because there was quite a number of vendors working in that space too.

I got to meet and talk with the awesome Reuven Lerner, who host the amazing Freelancers’ Show. And I learned quite a bit about consulting from freelancers, and consultants who came out to Reuven’s open space. Positioning in the market is crucial, as I expected. What I didn’t expect was that I seemed to have gained quite a bit of knowledge from listening to the podcast and reading various books. Alas, it is all still theoretical rather than practical in many cases.

Python’s killer application is data science, and there is a lot of interest and work in that area. Which means while web development in Python will stay around, I believe there will be less and less emphasis on it both in the community and in PyCon. There are a lot of new and interesting coming into the field from coding schools, so only time will tell how that works out. But if you are into analysis and data science, Python will help you shine.

PyCon Canada 2018

Speaking of PyCons, Elaine Wong, who is the chair of PyCon Canada 2018, convinced me to help run the sprints for PyCon Canada 2018. This should be interesting, since I’ve never organized an event like that. But I look forward to the challenge. If you are in Toronto, and interested in helping sponsor, volunteer at or host the sprints, please contact me!

Rookeries v0.9.0 out – New UI and Live Editing of Sites + Juggling JSON with jq Book Update

It has been quite a while since my last update. I apologize for the long silence. I wanted to focus on getting Rookeries up and running, to the degree that I can host a website. Namely I was hoping to update the Amber Penguin Software website and the Juggling JSON with jq book companion website to use Rookeries. Unfortunately I had to re-architecture the UI and bring the CouchDB management of the site and pages up to spec. However I am happy with the overall results:

Interactive Editing of Menus

Rookeries v0.9.0 introduces support for adding, removing and reordering menu items. It took a bit to find a good simple implementation of a drag and drop able list.

]4 Version 0.9.0 introduces interactive editing of menus.

Interactive Editing of Site Headers and Footers

Rookeries v0.9.0 also extends support of editing pages to site headers and footers. It uses the same inline editor for Markdown and HTML, that the pages currently enjoy.

]5 You can now edit site headers in Rookeries

So what else changed?

  • The migration of the UX over to a sliding sidebar is done.
  • Internally redux was replaced with MobX, which is far more maintainable for me.
  • The addition of saving sites and the current selected page.

There is still a lot to do on Rookeries, such as making the CMS more robust and flexible. Adding and removing pages is still outstanding. So is selection of pages via URLs. Marking whether or not changes have been made, to prompt the user to save. Also to check that the site is not in an invalid state. However things are starting to look up.

A few people have inquired about the state of the Juggling JSON with jq book, and when will a sample chapter and table of contents will be available. I apologize for not replying directly to those emails (I will do so shortly), but I hope to get those out in the next few days, and hopefully before the end of the year. Work on Rookeries has essentially ate up all the time that I would of used for working on the book. I think going forward I will work on both in parallel.

Using Blender for Cover Design

One of the fun parts of writing Juggling JSON with jq, is that I can experiment with various things. From the technical writing side, working with Sphinx has forced me to learn the ins-and-outs of that technology. Also very likely I will need to get into working with LaTeX for more of the advanced PDF parts.

The book cover provided me with an excuse to work on my graphic design skills. I decided that I wanted to do something more than just use a stock photo or old-timey engraving (a.k.a. O’Reilly’s book covers). Instead I decided to use Blender to create a compelling cover image. Yes, it may sound like overkill, however using Blender harkens to past times. Before I settled on studying computer science at the University of Toronto, and working as a programmer, I considered becoming a 3D graphics artist. I played with 3D Studio Max in high school, and learned about drafting and animation. When faced with the reality of being proficient but not amazing at drawing, and the very real possibility of competing with many more talented and experienced artists in the market I decided against that career path. However this decision did not dissuade me from taking drawing in university or enthusiastically learning Maya to make an animation for a visual computing course.

So I took the excellent Youtube video tutorial by Blender Guru. After a few hours of watching the tutorial I came up with a final image like so:

Blender Doughnut scene

] Doughnut and cup scene created in Blender. Created on August 14, 2017

And boy did I learn a lot from the tutorial: navigating Blender’s confusing UI, modeling, texturing, lighting and rendering a final image.

As for the book cover I am working on a cell-shaded, low-poly scene involving penguins and juggling. The process has been pretty fun so far, even though quite time consuming. However I have finished my low-poly model of a penguin, and added rigging (internal skeleton with joints for movement):

Low-poly model of a penguin

] Low-poly model of a penguin for the Juggling JSON with jq book. Created on August 23, 2017

Rendered pose of a juggling penguin

] Rendered pose of the juggling penguin for the Juggling JSON with jq book. Created on August 23, 2017.

Getting Started with Writing a Technical ebook

The early release of my ebook Juggling JSON with jq comes out tomorrow! However this post is more about the process of writing the book itself.

Getting started on an technical ebook, (such as Juggling JSON with jq), requires a bit of upfront setup. On the ebook side, I decided to go the route of writing the book in Markdown, and generating the various formats using Sphinx. While I feel most comfortable using Markdown, and yet Sphinx uses reStructedText by default. So I had to coax Sphinx to accept Markdown by using a project called m2r. Generating the PDF version of the ebook took a bit to get working. Sphinx uses LaTeX to generate PDFs, and LaTeX while powerful can be clunky to work with. I wrapped everything up with an invoke script, and now I can quickly generating new versions of ebook in the various formats I want to support.

Something unique to writing technical books, is the need to have actual working examples. You can learn by reading, but working through exercises and examples re-enforces that learning. In the case of Unjumbling JSON with jq, I needed an example REST API that readers play with. I searched for some nice open APIs, but nothing seemed very compelling. Many of the open APIs require some form of user registration and non-trivial authentication method that would complicate the examples in the book. So I setup a simple demo API for the book. Thankfully with Docker and Flask, that isn’t a particularly daunting task. (Dockerizing most of my webapps definitely made my live easier overall.)

Finally using Gumroad made marketing and selling the book a lot more approachable. Getting everything setup for e-commerce is a daunting job, if you plan on doing it yourself. Thankfully for ebooks, and similar digital products, Gumroad solves most of the problems one can encounter. I definitely recommend using them if you are planning to do something similar.

Book Announcement: Unjumbling JSON with jq

jq is an amazing tool for querying and manipulating JSON in command-line, that I learned about from one of my good colleagues, Eric Olsen. And I feel that jq deserves a good book describing how to use this tool. Hence I am writing a book called Unjumbling JSON with jq on the topic.

As mentioned in a previous post, I originally planned on writing a single book on both jq and httpie. I divided the original book in two, because there is only a small overlap between the two. I wanted to show examples of grabbing a REST API response via httpie, and parsing the JSON output with jq. However basic querying a REST API is something that could be covered in a short section. By writing the books separately, I will be able to release them faster, and the books will be much more focused.

I plan on selling early drafts of the ebook on August 10th. Buyers of the ebook will get regular versions of the evolving drafts of the ebook, and a free upgrade to the final version of the book. I want to release the early drafts to get early reader feedback. In addition readers of the book will have access to the REST API that accompanies the book.

You can order the early version of Unjumbling JSON with jq from here].