Packaging up a Rust Binary for Linux

Prologue

I should of written an update for quite some time. While I’ve been experimenting with marketing analytics, learning about data science, business development, doing DevOps with GitLab CI and various other things, I wanted to write up my learning when I had a chance to internalize everything. However what made me decide to write an update is this tweet from Chris Krycho. Chris runs the amazing New Rustacean podcast, which is a must listen for anyone interested in learning about programming in Rust.

How does one package a Rust app?

Chris asked about finding a good way to distribute Rust binaries across Linux distros:

Interestingly enough, I recently figured out how to package a small Rust CLI utility that I’m working on. My response was:

This post elaborates on what I meant with my reply.

Building snap packages using snapcraft

The fine folks at Canonical (the makers of Ubuntu Linux), have created something called snap packages. These packages and associated package manager help developers distribute applications (desktop, cloud, etc.) in a safe, isolated manner. I currently have slack installed this way. snaps isolate apps by having the package maintainer declare the capabilities an app requires (network access, access to system files, the GPU, home directory, etc.), and then ensuring the apps can not escape this sandbox.

Basic Setup

Getting setup with packaging a Rust crate was not too hard:

  1. Install snapcraft: sudo snap install snapcraft --classic
  2. Create a snap template inside your crate project: snap init
  3. Edit the generated snap/snapcraft.yml as per the documentation.
  4. Build the snap using snapcraft.
  5. Install the resulting snap with sudo snap install my-cool-rust-bin_x.y.z.snap --devmode --dangerous (this assuming you are experimenting with building a snap)
  6. Add you should be able to distribute your app as a snap now. (See the caveats below.)

Caveats working with snaps

Now there a bunch of caveats when working with snaps. And for my own Rust utility, I found these too taxing and I decided to go with creating a standard Debian package instead. However if I wanted to target multiple distributions and my app didn’t have a very unorthodox setup (my app relies on using the Chrome WebDriver to control a networked device managed by dd-wrt), I would probably have gone with a snap instead:

  • You need to know what capabilities your app needs: file access, network, etc. and you need to declare the appropriate interfaces in your snapcraft.yml
  • Using something other than my local system (be it a Docker based build or using a different base like base18), failed terribly at least for Rust.
  • Whether or not I’d have more success if my base system was the recommended Ubuntu 16.04 and not 18.04 is an outstanding mystery.
  • The snap confinement, even on the much more liberal devmode, works very well. No amount of coaxing on my part, let me use system paths when trying to spawn a process. This could just be me though, as not declaring network access did not block my app.
  • The docs could of been clearer about what was the latest recommended approach. (Still way clearer than the documentation for creating a DEB or RPM from scratch.)
  • Knowing which libraries (for the type of base system) your app needs takes a bit of experimentation. (e.g. I needed libssl1.0 for some builds and libssl1.1)
  • I have no idea how the classical confinement should work, and it is not recommended either way.

The end result for me was a working snap package, but an app that would not work when called from an installed snap. However I think snap packages are probably the way to go moving forward (or a similar format like flatpak). Since I only plan on targeting local Ubuntu 18.04 installs, I ended up creating a Debian package instead.

Building a Debian package with Cargo

I found a nice utility for creating a deb out of a Rust crate, called cargo-deb. After installing the crate with cargo: cargo install cargo-deb, I simply ran cargo deb and I was done. cargo-deb looked into my Cargo.toml for the metadata, ran a build and a few moments later I was the proud owner of a Debian package. Since my app relies on the chromium-browser and chromium-chromedriver packages, I added a small section in my Cargo.toml as so:

[package.metadata.deb]
depends = "$auto, chromium-browser, chromium-chromedriver"

The $auto is something that the Debian packaging mechanism needs, and that is the comma separated format that DEBs use.

Building a RPM from a DEB package

Now this the part that I didn’t do this time around. However I figured out how to create RPM packages from DEB packages a few months ago. The trick is to use the alien utility to create a RPM out of a DEB:

sudo alien --to-rpm --scripts --verbose my-cool-rust-bin.deb

For the record, I did not try to improve or debug the resulting RPM. (This entire effort was for a product that failed to launch.) However as part of my tests I was able install it and run it from on CentOS VM.

Epilogue

Anyways, I would recommend the cargo-deb and alien approach, if you are not planning to distribute a Rust app across a multitude of Linux distros. I would recommend dipping into snap if you plan on distributing something more commercial and wide-spread like a slack or kubectl. And I hope that helps you on your Rust app packaging for Linux journey!

To Make or Not to Make – Using cargo make for Rookeries v0.12.0

I was pleasantly surprised when my last blog post about migrating to Rust’s integration tests really took off on Twitter. I did not quite expect that much interest. 🙂

Using cargo-make

I recently continued with my exploration of Rust through Rookeries (my attempt at a static site generator/backing API server). This time I worked on switching over from using invoke and GNU make to using a nice build system called cargo-make. Overall I am quite happy with the result. To give you a taste of how cargo-make describes build tasks and environment variables, I included a short snippet from the Rookeries Makefile.toml below:

<br />
[env]
SERVER_PORT = "5000"

[tasks.build-js]
description = "Build JS frontend."
command = "npm"
args = ["run", "build"]

[tasks.run-dev-server]
description = "Run a Rookeries dev server."
command = "cargo"
args = ["run", "--", "run", "${SERVER_PORT}"]
dependencies = ["build-js"]

Much like invoke and Make, you can specify dependencies. Unlike Make, there is no weird tab-based syntax, or implicit behaviour that requires a look up in the GNU Make manual. Just a simple TOML file. Environment variables can be specified in standalone .env files, passed in or specified in the Makefile.toml itself, and allow for variables specified during a run to override defaults, etc. cargo-make also lets you use conditional logic but I did not need to use that for my purposes.

For Rookeries, I ended up creating locally running tasks, and versions for the Dockerized build setup that I have in CircleCI. Also I ended up using the scripting support that cargo-make provides for preparing the database for, and running the integration tests.

My Thoughts on CouchDB for Rookeries and Rust

I feel I cheated a bit towards the end especially by using a few curl commands to instantiate the CouchDB database, and with running the API integration tests. Maybe I should of wrote a proper CouchDB library and utility. However I decided against it, simply because I want move away from CouchDB for Rookeries. So I may not want to invest that much time in maintaining a proper database client if I’ll end up switching over to using Postgresql via diesel.rs Originally, I started using CouchDB because I use it heavily at my work at Points. As time goes on, I find myself more and more reluctant with using CouchDB, just because of all the issues I’ve experienced at work. That said, if someone wants me to maintain a CouchDB Rust library/CLI based on the code I have for Rookeries, then please reach out to me. The CouchDB crates I saw in crates.io did not inspire me, so I wrote my own CouchDB module in Rust. But I don’t want to maintain that code just for myself, so if anyone actually needs a proper CouchDB crate and wants me to maintain it, please reach out to me.

Update – Sofa Library

After I wrote this blog post originally, I did not know about this nice CouchDB crate called sofa. If I had know about it, I would of not bothered with my own implementation.

New Release of Rookeries – Now 100% Rust

Finally since this migration was a bit of work, so I decided to release a new version of Rookeries. Since the invoke setup was the last Python code, Rookeries is now 100% completely written in Rust. (And the JS frontend naturally.) The Rookeries Docker image is available on Docker Hub. The project is still quite in flux, and what I will work on next depends on what I need to do to hook up Rookeries to one of my Gatsby powered sites.