The purpose of the
BinaryBuilder.jl Julia package is to provide a system for compiling 3rd-party binary dependencies should work anywhere the official Julia distribution does. In particular, using this package you will be able to compile your large pre-existing codebases of C, C++ and Fortran software into binaries that can be downloaded and loaded/run on a very wide range of machines. As it is difficult (and often expensive) to natively compile software packages across the growing number of platforms that this package will need to support, we focus on Linux-hosted cross-compilers. This package will therefore set up an environment to perform cross-compilation for all of the major platforms, and will do its best to make the compilation process as painless as possible.
Note that at this time, BinaryBuilder runs on Linux x86_64 systems only, with macOS support under active development, although it builds binaries for all major platforms.
Suppose that you have a Julia package
Foo.jl for which you want to build binary dependencies. With BinaryBuilder, you will create a GitHub repository
FooBuilder that contains a
build_tarballs.jl script to build the libraries and other binaries needed by
Foo.jl. It is very common that you will want these binaries to be built by Travis CI automatically, so most builder repositories will want to create a
.travis.yml file as well to automatically run
build_tarballs.jl and upload ("deploy") the resulting binaries ("build artifacts") to a release in the
The result of a successful build is the set of built binaries as well as a
build.jl file that can be used within
Foo.jl (with the
BinaryProvider package) to download and make available these binaries for Julia packages. Much of this process can be done for you interactively through the
run_wizard function, though that wizard currently only runs on Linux and MacOS with supported hardware.
build_tarballs.jl file is a Julia script of the form
using BinaryBuilder; build_tarballs(ARGS, "Foo", sources, script, platforms, products, dependencies). See below for a detailed description of the arguments, but essentially you provide: a list of
sources to download (source code tarballs or git repositories, for example) and any
dependencies required to build the sources, a bash
script to build and install your code, a list of
platforms to build on, and a list of build
products (e.g. libraries) to save and upload. This is the script that will be ultimately run when you run
julia build_tarballs.jl locally, or on a Travis CI server.
In order to build on Travis, you must have a Travis CI account and enable Travis for your
FooBuilder repository that is hosted on Github. This will allow Travis to automatically run your script whenever changes are pushed to the repository. The
.travis.yml file describes what Travis should run, and for
BinaryBuilder this file should have a particular form that is automatically generated for you by the
run_wizard function, and can also be created by the
setup_travis function. Alternatively, you can copy it from one of the provided examples, changing a couple of lines for the name of the repo and the OAuth token.
In particular, Travis uploads the build products to a GitHub release whenever you push a new tag or create a new release in your
FooBuilder repository. To do this, Travis needs to authenticate itself to the Github server, and it does this using an encrypted OAuth token stored in your
.travis.yml file. This token is created for you automatically by the
setup_travis functions, and it can also be created manually via
travis setup releases using the
travis command-line program.
Once these files are created and uploaded to GitHub, and Travis is enabled for your FooBuilder repository, then simply creating a release in your repo will trigger Travis to run, using cross-compilers to generate binaries for all platforms simultaneously, and finally to automatically attach the binaries to the release, along with a
build.jl file that is used by client Julia packages that wish to download and install this package.
See also the FAQ, build tips, and tricksy gotchas for help with common problems.
BinaryBuilder.jl build script (what is often referred to as a
build_tarballs.jl file) looks something like this:
using BinaryBuilder src_tarball = "<path to source tarball>" src_hash = "sha256 hash of the source tarball" sources = [ (src_tarball, src_hash), ] script = raw""" make make install """ products(prefix) = [ LibraryProduct(prefix, "libfoo", :libfoo), ExecutableProduct(prefix, "fooifier", :fooifier), ] dependencies =  # Build 'em! build_tarballs( ARGS, "libfoo", sources, script, platforms, products, dependencies, )
This bare-bones snippet (an adapted form of the
libfoo test within this repository) first identifies the sources to download and compile (there can be multiple sources listed here), then lists the
bash commands to actually build this particular project. Next, the
products are defined. These represent the output of the build process, and are how
BinaryBuilder.jl knows that its build has succeeded. Finally, we pass this information off to
build_tarballs(), which takes it all in and runs the builds, placing output tarballs into the
The bash commands contained within
script will be executed for each
platform that is passed in, so if there are platform differences that need to be addressed in the build script, using
if statements and the
$target environment variable can be a powerful tool. See the OpenBLASBuilder build script for an example showcasing this.
build_tarballs() method completes, it will have written out a
build.jl file to download and install the generated tarballs. This file is what will be used in Julia packages that need to use your built binaries, and is typically included within the tagged release uploads from a builder repository. Here is an example release from the IpoptBuilder repository, containing built tarballs as well as a
build.jl that can be used within
While constructing your own build script is certainly possible,
BinaryBuilder.jl supports a more interactive method for building the binary dependencies and capturing the commands used to build it into a
build_tarballs.jl file; the Wizard interface.
BinaryBuilder.jl contains a wizard interface that will walk you through constructing a
build_tarballs.jl file. To launch it, run
BinaryBuilder.run_wizard(), and follow the instructions on-screen.
How does this all work?
BinaryBuilder.jl wraps a root filesystem that has been carefully constructed so as to provide the set of cross-compilers needed to support the wide array of platforms that Julia runs on. This _RootFS_ is then used as the chroot jail for a sandboxed process which runs within the RootFS as if that were the whole world. The workspace containing input source code and (eventually) output binaries is mounted within the RootFS and environment variables are setup such that the appropriate compilers for a particular target platform are used by build tools.