Edit on GitHub

Frequently Asked Questions

I'm having trouble compiling <project name here>

First, make sure that you can compile that project natively on whatever platform you're attempting to compile it on. Once you are assured of that, search around the internet to see if anyone else has run into issues cross-compiling that project for that platform. In particular, most smaller projects should be just fine, but larger projects (and especially anything that does any kind of bootstrapping) may need some extra smarts smacked into their build system to support cross-compiling. Finally, if you're still stuck, try reaching out for help on the #binarybuilder channel in the JuliaLang slack.

How do I use this to compile my Julia code?

This package does not compile Julia code; it compiles C/C++/Fortran dependencies. Think about that time you wanted to use IJulia and you needed to download/install libnettle. The purpose of this package is to make generated tarballs that can be downloaded/installed painlessly as possible.

What is this I hear about the macOS SDK license agreement?

Apple restricts distribution and usage of the macOS SDK, a necessary component to build software for macOS targets. Please read the Apple and Xcode SDK agreement for more information on the restrictions and legal terms you agree to when using the SDK to build software for Apple operating systems. As usual, you should not take legal advice from FAQs on the internet, but in an effort to distill that large document down a bit, it is a breach of the license agreement to use the SDK to compile macOS binaries on a machine that is itself not a macOS machine. Although this toolkit is designed to primarily run on Linux machines, it would not be breaking the license agreement to run this toolkit within a virtualized environment on a macOS machine, whereas it would be breaking the license agreement to run this toolkit on, for example, an Amazon AWS machine running Linux. The Docker runner implements the virtualization approach on macOS machines. BinaryBuilder.jl, by default, will not automatically download or use the macOS SDK on non-apple host operating systems, unless the BINARYBUILDER_AUTOMATIC_APPLE environment variable is set to true.

Are there other environment variables I can use?

Yes, take a look.

Absolutely! There's nothing Julia-specific about the binaries generated by the cross-compilers used by BinaryBuilder.jl. Although the best interface for interacting with this software will always be the Julia interface defined within this package, you are free to use these software tools for other projects as well. Note that the cross-compiler image is built through a multistage bootstrapping process, see this repository for more information. Further note the macOS SDK license agreement tidbit above.

At line XXX, ABORTED (Operation not permitted)!

Some linux distributions have a bug in their overlayfs implementation that prevents us from mounting overlay filesystems within user namespaces. See this Ubuntu kernel bug report for a description of the situation and how Ubuntu has patched it in their kernels. To work around this, you can launch BinaryBuilder.jl in "privileged container" mode. BinaryBuilder should auto-detect this situation, however if the autodetection is not working or you want to silence the warning, you can set the BINARYBUILDER_RUNNER environment variable to privileged. Unfortunately, this involves running sudo every time you launch into a BinaryBuilder session, but on the other hand, this successfully works around the issue on distributions such as Arch linux.

I have to build a very small project without a Makefile, what do I have to do?

What BinaryBuilder needs is to find the relevant file (shared libraries, or executables, etc...) organised under the $prefix directory: libraries should go to ${libdir}, executables to ${bindir}. You may need to create those directories. You are free to choose whether to create a simple Makefile to build the project or to do everything within the build_tarballs.jl script. When the script completes, BinaryBuilder expects to find at least one artifact built for the expected architecture in either ${libdir} or ${bindir}. Remember also that you should use the standard environment variables like CC, CXX, CFLAGS, LDFLAGS as appropriate in order to cross compile. See the list of variables in the Tips for Building Packages section.

I love the wizard, but now I want to break free: can I build the tarballs without it?

The build_tarballs.jl script can be used as a command line utility, it takes a few options and as argument the list of triplets of the targets. You can find more information about the syntax of the script in the Command Line section or by running

julia build_tarballs.jl --help

Instead of using the wizard, you can adapt for a new library a build_tarballs.jl script originally written for another library. Then, you can build the tarballs with

julia --color=yes build_tarballs.jl --debug --verbose

The --debug option will drop you into the BinaryBuilder interactive shell if an error occurs. If the build fails, after finding out the steps needed to fix the build you have to manually update the script in build_tarballs.jl. You should run again the above command to make sure that everything is actually working.

Since build_tarballs.jl takes as argument the comma-separated list of targets for which to build the tarballs, you can select only a few of them. For example, with

julia --color=yes build_tarballs.jl --debug --verbose aarch64-linux-musl,arm-linux-musleabihf

you'll run the build script only for the aarch64-linux-musl and arm-linux-musleabihf targets.

If you decide to use this workflow, however, you will need to manually open pull requests for Yggdrasil.

Can I open a shell in a particular build environment for doing some quick tests?

Yes! You can use BinaryBuilder.runshell(platform) to quickly start a shell in the current directory, without having to set up a working build_tarballs.jl script. For example,

julia -e 'using BinaryBuilder; BinaryBuilder.runshell(Windows(:i686))'

will open a shell in a Windows 32-bit build environment, without any source loaded. The current working directory of your system will be mounted on ${WORKSPACE} within this BinaryBuilder environment.

Can I publish a JLL package locally without going through Yggdrasil?

You can always build a JLL package on your machine with the --deploy flag to the build_tarballs.jl script. Read the help (--help) for more information.

A common use case is that you want to build a JLL package for, say, Libfoo, that will be used as dependency to build Quxlib, and you want to make sure that building both Libfoo and Quxlib will work before submitting all the pull requests to Yggdrasil. You can prepare the build_tarballs.jl script for Libfoo and then build and deploy it with

julia --color=yes build_tarballs.jl --debug --verbose --deploy="MY_USERNAME/Libfoo_jll.jl"

replacing MY_USERNAME with your GitHub username: this will build the tarballs for all the platforms requested and upload them to a release of the MY_USERNAME/Libfoo_jll.jl, where the JLL package will also be created. As explained above, you can pass argument the list of triplets of the platforms for you which you want to build the tarballs, in case you want to compile only some of them. In the Julia REPL, you can install this package as any unregistered package with

]add https://github.com/MY_USERNAME/Libfoo_jll.jl.git

or develop it with

]dev https://github.com/MY_USERNAME/Libfoo_jll.jl.git

Since this package is unregistered, you have to use the full PackageSpec specification to add it as dependency of the local builder for Quxlib:

    Dependency(PackageSpec(; name = "Libfoo_jll",  uuid = "...", url = "https://github.com/MY_USERNAME/Libfoo_jll.jl.git"))

You can of course in turn build and deply this package with

julia --color=yes build_tarballs.jl --debug --verbose --deploy="MY_USERNAME/Quxlib_jll.jl"

Can I install packages in the build environment?

Yes, but it's unlikely that you'll need to. The build environment is based on Alpine Linux (triplet: x86_64-linux-musl) so you can use apk to install packages in it. However, if you need runtime libraries or programs for the target system these packages won't help you. The package manager is useful only to install utilities, tools or libraries that are needed exclusively at compile time on the build system.

What are those numbers in the list of sources? How do I get them?

The list of sources is a vector of BinaryBuilder.AbstractSources. What the hash is depends on what the source is:

  • For a FileSource or an ArchiveSource, the hash is a 64-character SHA256 checksum. If you have a copy of that file, you can compute the hash in Julia with

    using SHA
open(path_to_the_file, "r") do f
     bytes2hex(sha256(f))
end

    where path_to_the_file is a string with the path to the file. Alternatively, you can use the command line utilities curl and shasum to compute the hash of a remote file:

    $ curl -L http://example.org/file.tar.gz | shasum -a 256

    replacing http://example.org/file.tar.gz with the actual URL of the file you want to download.

  • For a GitSource, the hash is the 40-character SHA1 hash of the revision you want to checkout. For reproducibility you must indicate a specific revision, and not a branch or tag name, which are moving targets.