Introducing axoupdater

Cargo-dist 0.12.0 introduced a new feature that users have been asking for for awhile: an updater program, which makes it easy for your users to keep your software up to date. While we support distribution via two different package managers (Homebrew and npm), which have builtin ways to keep software up to date, we recognize that a large number of users still primarily get their software installed via the shell or PowerShell installers — and for those users it's easy to get stuck on older versions, and inconvenient to upgrade. Axoupdater, our new updater tool, was designed to make it as easy as possible to ship a way for your users to keep up to date. It also fulfills the prophecy that every tool a package manager nerd works on will eventually become a package manager.

We had two conflicting goals which we worked to square with each other:

• We wanted to provide an installer which was as easy as possible, requiring zero configuration or work from app developers beyond the choice to package the updater at all; but
• We wanted to provide a way for app developers to control every aspect of the process, and to incorporate updater logic directly into their application.

To satisfy both needs, we settled on two different approaches: a fully standalone updater executable, and a Rust library which exposes a convenient API with all of the updater's functionality.

The standalone updater user experience

Since the intent was to make the standalone version of the updater as simple as possible, I set out with a few basic design goals that ensured that it would be useful to a wide variety of projects.

• The updater should be non-interactive and require no options;
• The updater's output should be both as clear and as generic as possible.

If an app developer chooses to enable the updater in their cargo-dist config, then users who run the shell or PowerShell installer will receive one new binary alongside the program. It's installed under the name YOURPROGRAM-update (for example axolotlsay-update), and simply running it performs an update which looks like this:

$ axolotlsay-update
Checking for updates...
downloading axolotlsay 0.2.114 aarch64-apple-darwin
installing to /Users/mistydemeo/.cargo/bin
  axolotlsay
  axolotlsay-update
everything's installed!
New release 0.2.114 installed!

No configuration, no arguments: just run, and it performs an update and then reports on the results. It's as simple as can be and, for most projects, I believe it handles exactly what they need — no more or less.

While this version of the updater is standalone, it does integrate with one kind of app: programs which support custom subcommands the way that git and cargo do.¹ Programs following this informal standard treat any program with the right file name as one of its own subcommands, and axoupdater has been designed to be installed under a compatible file name.

Before the updater: cargo-dist prerequisites

Before we even started on the updater, I knew cargo-dist was going to need some new features to support it. Specifically, packages installed via the shell and PowerShell installers were missing something crucial that the "true" package manager installers have: the ability to tell a) the software that was installed, b) what version is installed, c) where it's installed, d) what files were installed, and e) where new versions should be fetched from. Each of these gives us something we need for the updater to do its job:

• By knowing what software was installed, we can ensure we're upgrading the right thing.
• By knowing what version was installed, we can compare it against the latest version to see if we even need to do an upgrade at all.
• By knowing where it was installed, we can make sure the new version gets installed to the same location.
• By knowing what files were installed, we can clean up after ourselves.
• By knowing where the last version was installed, we know what source to fetch new releases from. This is important since cargo-dist can publish to multiple sources: just knowing the software was installed from somewhere doesn't tell us if the new version is located on GitHub or Axo releases, or the specific repository to look it up from.

A package manager, like Homebrew or npm, tracks all of that information by default: it needs to know these things in order to handle uninstallation, upgrades, and other tasks that are core parts of the package management experience. Because the shell and PowerShell installers aren't package managers, they haven't needed to know this information and, consequently, they haven't been tracking it.

In preparation for the updater itself, I first added a feature in cargo-dist 0.8.0 to start creating install receipts prior to actually shipping the updater itself. Every installation of program via cargo-dist's shell or PowerShell installer since 0.8.0 places a JSON receipt in a known configuration path². Its contents look like this:

{  "binaries": [    "axolotlsay"  ],  "install_prefix": "/Users/mistydemeo/.cargo/bin",  "provider": {    "source": "cargo-dist",    "version": "0.12.0-prerelease.1"  },  "source": {    "app_name": "axolotlsay",    "name": "cargodisttest",    "owner": "mistydemeo",    "release_type": "github"  },  "version": "0.2.114"}

The structure is intentionally simple and easy to read, even just by eye. Each piece of data provides enough data to satisfy the five questions I asked above, allowing the updater to reason about what to do and how to do it. It's my hope that this information might also be useful to other programs that might, for whatever reason, want to query information about cargo-dist installed software³ — or even users who may want to write their own uninstallers or other utilities!

The standalone updater

The actual implementation of axoupdater is quite simple; by making use of support code in cargo-dist and existing functionality, we were able to get everything we needed done in a lean 800 lines of Rust. The update process runs through the following steps:

• The updater determines what program it's intended to upgrade and loads the install receipt for that program;
• Using the information about where to query for updates in the receipt, it checks for the latest version number.
• If the running version isn't the latest version, it fetches the shell or PowerShell installer and runs that.

The install process, in other words, isn't that different from what a user does when they choose to upgrade your software; we're just doing it for them.

One of the decisions I made while writing the updater was to reuse our existing installers as-is. Our standalone updaters are written in shell (macOS, Linux) and PowerShell (Windows) for portability; it keeps us from having to ship multiple copies of the installer for different OSs and CPU architectures. The result, however, is that the installer isn't Rust code we can just embed or call from the updater as a library. Instead of totally replicating that logic within the updater, where it could drift away from other implementations or develop unique bugs, simply reusing the existing installers kept things easier. It also keeps the updater closer to the platonic ideal of "exactly what your user would do by hand, just automated".

Relying on the existing installers was mostly a seamless process, but we did run into one problem: choosing the installation location. Cargo-dist allows developers to customize the type of installation path the user may want, and the path the installer chooses can be influenced by one of a few environment variables, depending on how exactly the installer was built. If either the developer has changed the installation location for the new version, or the user's environment has changed, the new installer might pick a different location than before. This actually pointed to a separate weakness of the installer as written: there was no way to force it to install somewhere. Sometimes, no matter what configuration says, you want to just tell the software what to do and have it listen for once in your life. We resolved this by adding and making use of the new CARGO_DIST_FORCE_INSTALL_DIR; as a bonus, it's now available for other users who also have exacting requirements for the installer.

Now, you might be wondering if adding a new native code artifact to your releases is going to increase its build time. The good news is: no, it certainly won't! When designing it, I adopted an automatic self-configuration design that means we can fetch a prebuilt binary from the official axoupdater repository. Axoupdater knows what app it's built for based on its filename. Since we always install it under the name YOURAPP-update, axoupdater can read its own filename and, using that, decide what app it's been configured for. Any prebuilt axoupdater binary is the same; the only magic comes from how it was installed.

App sources

As mentioned above, one of the things the updater needs to know is where to fetch releases from. For most of our users, that's GitHub, but that's not guaranteed: Axo Releases, our own releases hosting service, is in private beta now. In the future, we may add support for other release hosting options as well.

Axoupdater makes sure that both users and developers get a completely seamless experience regardless of which hosting backend is in use. For developers, we've also made sure that the updater library gates its support for GitHub and Axo Releases behind feature flags, allowing you to turn off whichever release system you don't use and keep the size of the dependency tree smaller.

The updater library

While I expect many developers to be happy with the standalone updater, I wanted to make sure we provided opinionated developers with the tools they need to create an updater which works exactly the way they'd like. axoupdater, the library, is the code that powers our updater executable. The standalone updater is a thin binary which simply uses the same code we make available to third parties, ensuring we eat our own dogfood and keep it working well. (It also means we had to package it as the dreaded binary-and-library crate.)

While I wrote the updater executable first, the decision to keep this use case in mind paid off; keeping pretty much 100% of the implementation in the library meant I was already 95% of the way to something that developers would need, and the remainder of the work focused on customization options.

When writing the updater library, I also added support for more advanced updater options in order to allow developers integrating it to finetune the update strategy and enable making choices such as whether to allow prereleases, or to upgrade to a specific named version instead of just whatever the most recent version is.

In particular, since the standalone executable prints its outputs to the terminal, the original implementation of the updater library didn't provide any feedback to the caller on what updates had been performed — a clear oversight, and information that callers would need to better communicate things to their users. That was expanded to provide the following information following a successful upgrade:

• The detected old version (pre-upgrade)
• The installed new version (post-upgrade)
• The new version's tag, which may or may not differ from the new version number
• The path to which the new version was installed

A sample third-party app leveraging the axoupdater library for its own updater subcommand is Astral's uv, which was able to implement a self-updater in only 115 lines using axoupdater's API. We also make use of it in cargo-dist itself.

The fun part - quirks and platform-specific issues

As simple as we kept the implementation, we ran into our share of fun issues simply because of the nature of what we were doing. Replacing an actively-running executable without interruptions can be fraught, and we had to come up with various platform-specific fixes in order to support what we're doing.

On Windows, it's not possible to write to or delete an executable while it's running. Obviously, this posed issues for us when writing a feature that does exactly that. In order to work around this, we had to be a bit cleverer. On Windows, axoupdater creates a temporary directory and relocates the running executable to that path. This is valid; Windows doesn't mind moving an actively-running executable, just writing to it or deleting it. We then write the new executable to the path where the old version used to be, and delete the temporary directory when the process completes.

Our initial implementation created a directory for this inside the system temporary path, but this presented an interesting issue: a user tried to run an updater for a program installed on a dev drive, which is a separate logical filesystem from the temporary path. We can't, of course, move an executable to a separate filesystem, so we had to go back to the drawing board and reconsider. We settled on instead creating the temporary directory inside the directory where the binary is present, allowing us to be absolutely certain they'll be located on the same filesystem. Since we always clean up the temporary path when the update is complete, we can be sure that we don't clutter the user's filesystem permanently.

On Linux, we ran into a separate issue. Ovewriting the actively running executable is legal, but with a catch: it has to be done cleanly. Our shell installer originally copied binaries to their destinations via cp, but this happens via a potentially-messy series of writes that can cause errors. We swapped to moving binaries to their destinations using mv, an atomic operation that's safe for our usecase.

Axoupdater today

Axoupdater is available to all cargo-dist users who distribute their software using v0.12.0 or later; while we've marked it as experimental, I believe it's in good shape for projects to try out today. I'd also love to hear your feedback on how it works for you — or doesn't work.