BinaryBuilder.jl

The purpose of the BinaryBuilder.jl Julia package is to provide a system for compiling 3rd-party binary dependencies that should work anywhere the official Julia distribution does. In particular, using this package you will be able to compile your large pre-existing codebases of C, C++, Fortran, Rust, Go, etc... software into binaries that can be downloaded and loaded/run on a very wide range of machines. As it is difficult (and often expensive) to natively compile software packages across the growing number of platforms that this package will need to support, we focus on providing a set of Linux-hosted cross-compilers. This package will therefore set up an environment to perform cross-compilation for all of the major platforms, and will do its best to make the compilation process as painless as possible.

Note that at this time, BinaryBuilder itself runs on Linux x86_64 and macOS x86_64 systems only, with Windows support under active development. On macOS and Windows, you must have docker installed as the backing virtualization engine. Note that Docker Desktop is the recommended version; if you have Docker Machine installed it may not work correctly or may need additional configuration.

Project flow

Suppose that you have a Julia package Foo.jl which wants to use a compiled libfoo shared library. As your first step in writing Foo.jl, you may compile libfoo locally on your own machine with your system compiler, then using Libdl.dlopen() to open the library, and ccall() to call into the exported functions. Once you have written your C bindings in Julia, you will naturally desire to share the fruits of your labor with the rest of the world, and this is where BinaryBuilder can help you. Not only will BinaryBuilder aid you in constructing compiled versions of all your dependencies, but it will also build a wrapper Julia package (referred to as a JLL package) to aid in installation, versioning, and build product localization.

The first step in the BinaryBuilder journey is to create a build recipe, usually named build_tarballs.jl. The Julia community curates a tree of build recipes, Yggdrasil, that already contains many examples of how to write a build_tarballs.jl file. These files contain information such as the name, version and source locations for a particular build, as well as the actual steps (in the form of a bash script) and the products that should be generated by the build.

The result of a successful build is an autogenerated JLL package, typically uploaded to the JuliaBinaryWrappers github organization. Binaries for each version of every build are uploaded to the GitHub releases page of the relevant JLL package. Finally, a registration request is opened against the General Julia registry, so that packages such as the aforementioned Foo.jl can simply pkg> add libfoo_jll to download the binary artifacts as well as the autogenerated Julia wrapper code.

To generate a build_tarballs.jl file (explained in greater detail in Building Packages), one can clone Yggdrasil, copy an existing build recipe, modify it, and submit a new PR, or use the BinaryBuilder Wizard to do all of that automatically. See also the FAQ, build tips, build troubleshooting and tricksy gotchas for help with common problems.

Wizard interface

While constructing your own build_tarballs.jl is certainly possible, BinaryBuilder.jl supports a more interactive method for building the binary dependencies and capturing the commands used to build it into a build_tarballs.jl file: the Wizard interface. To launch it, run BinaryBuilder.run_wizard(), and follow the instructions on-screen.

How does this all work?

BinaryBuilder.jl wraps a root filesystem that has been carefully constructed so as to provide the set of cross-compilers needed to support the wide array of platforms that Julia runs on. This RootFS is then used as the chroot jail for a sandboxed process which runs within the RootFS as if that were the whole world. The workspace containing input source code and (eventually) output binaries is mounted within the RootFS and environment variables are setup such that the appropriate compilers for a particular target platform are used by build tools.