cargo-dist for any language

cargo-dist was born to distribute Rust software — "cargo" is right in the name, after all! But the longer development has continued, and the more we've seen it in action, the clearer it's become that a lot of what it does is useful for all kinds of software. If we could just teach cargo-dist how to build software in other languages, we'd have so many new opportunities to use its features like CI generation, release creation, and so on.

As I've become more familiar with the code, I've come to realize just how much of cargo-dist isn't Cargo-specific. Although a few parts of the codebase assumed that a project was a Cargo project, there was already a clean division between the build stage and the rest of the project. Finishing the separation was surprisingly straightforward and it's opened quite a few new doors for us.

How we taught cargo-dist to build other projects

If you're not interested in the implementation, and just want to see it in action, skip ahead to the next section.

Central to managing this separation is our axoproject crate. We already used this to obtain essential metadata about projects and, since it's used by other Axo tools like Oranda, it was already designed with basic support for non-Cargo projects. That gave us a good spot to add support for other kinds of projects.

Axoproject exposes structured data to the caller with 1) fundamental data about the project, and 2) a judgment call of what kind of project it is. The former was already normalized between the two types of projects axoproject supported, cargo and npm, which meant we could easily slot in new kinds of projects.

Since generic projects could be any kind of project, in any kind of language, with any sort of buildsystem, we needed some kind of metadata source to unify them. For Cargo projects, we use the standard Cargo.toml metadata and the guppy crate to parse it in a Cargo-compatible way. For a project that could be anything, there's no standard format that already exists and so, to make our job easier, we decided to define a very Cargo-like format.

Our format, dist.toml, is very similar to the Cargo.toml format used by Rust, and is used to provide some basic metadata about projects along with some new information that cargo-dist needs about how to build your program. Fields that are in common with Cargo.toml use the same names and same values, so Rust developers will immediately feel at home and the official Cargo documentation can act as clarifying documentation to our own. We only had to add a few extra fields — while Cargo is able to provide us implicit information about what binaries and libraries a package produces, and we already know that cargo build is what we have to run, these could be anything for a generic build. We've defined configuration fields to allow these to be specified manually.

Until now, cargo-dist has largely ignored the "project kind" field, but it gave us the tooling we needed to introduce some branching. cargo-dist's build process is already segmented into a number of smaller jobs, so we were able to easily cordon off only a small number of jobs that were Cargo-specific and introduce alternatives for non-Cargo builds.

Let's try out cargo-dist generic builds!

Let's take a look at what distributing a simple non-Rust program with cargo-dist looks like. This example is based on our public C sample program which you can view to see it all together.

First, let's write our program. We'll keep it simple by writing a basic hello world.

#include <stdio.h>int main() { puts("Hello, cargo-dist!"); }

To build this, we'll use a Makefile. You won't be surprised, but this is maybe the hardest part of the whole process! Here's a simple example which calls gcc to build our one executable.

CC := gccEXEEXT :=ifeq ($(OS), Windows_NT)    EXEEXT := .exeendifall: main$(EXEEXT)main$(EXEEXT):    $(CC) main.c $(CFLAGS) -o main$(EXEEXT).PHONY: all

Don't worry too much about what this Makefile means; what's important is just that we're able to generate a binary by running make. Once we have these two files, we create a dist.toml manifest for our program. For our sample program, we'll want to focus on these two fields:

binaries = ["main"]build-command = ["make"]

The binaries field tells cargo-dist what binaries your program contains that cargo-dist needs to be able to install, and build-command tells cargo-dist what to run in order to produce them. cargo-dist normally gets those values straight from Cargo, and since we don't have Cargo we have to specify them manually instead. The whole dist.toml, with the rest of the app metadata, looks like this:

[package]name = "cargo-dist-c-example"description = "A test of a C program for cargo-dist"version = "0.1.0"license = "MIT OR Apache-2.0"repository = "https://github.com/axodotdev/cargo-dist-c-example"binaries = ["main"]build-command = ["make"]

With these three files in place, our repo is just about done. From here, everything else we need to do is exactly the same as with a Cargo-based app. Let's walk through it anyway, just to see how it's done. Run cargo dist init, and answer the questions however you like. As soon as that's done, it's finished: this app is ready for cargo-dist! You can push the changes to GitHub and watch CI do its job, or you can run a local build and see how it works. To create a local build, try running cargo dist build --artifacts=local --target=aarch64-apple-darwin. (You can change the "target" to anything else you like if you're not on a Mac.) As soon as it's finished, you should see a new target/distrib directory has been created, and packaged inside it: a nice copy of your app, app packaged up and ready to go.

I'm excited to see what users start distributing with this new tooling. We've created sample projects using C and JavaScript, and I'm personally using it to distribute projects written in Objective-C. If you need more detail on how to use it, take a look at the official docs. Let us know if there's anything more you find yourself wanting from this feature, and until then, happy hacking!