Support cross-building aarch64 unikernels on macOS

[samoht]

I'd like to build aarch64 Solo5/MirageOS unikernels on an Apple Silicon Mac, instead of going through a Linux VM.

This adds a Darwin case to configure.sh that builds the target toolchain and bindings with clang, ld.lld and llvm-objcopy, so solo5-cross-aarch64 installs on macos/arm64. It declares the llvm and lld depexts, and a CI job builds it on every push.

The toolchain is pure LLVM: no libgcc is bundled. The downstream unikernel link needs none. clang inlines the aarch64 atomics, and once ocaml-solo5's nolibc stopped pulling long double soft-float (mirage/ocaml-solo5#171), the toolchain wrapper stopped passing -lgcc. I verified it by building a full aarch64 cross-compiler and linking a unikernel on macOS with no libgcc present.

The rest of the stack:

• Support building the aarch64 cross-compiler on macOS mirage/ocaml-solo5#171 builds the aarch64 cross-compiler on macOS
• Install a relocatable gmp.pc mirage/ocaml-gmp#29 installs a relocatable gmp.pc
• Archive libgmp.a with the cross ar/ranlib when cross-compiling mirage/ocaml-gmp#30 archives libgmp.a with the cross ar/ranlib
• Honor OCAML* environment variables in configure ocaml/Zarith#173 honours the OCAML* environment variables in configure

[dinosaure]

It seems that this PR is attempting to resolve several issues, including the one relating to the Solo5 version (#642). I have therefore deliberately ignored the diffs relating to the VERSION variable.

Furthermore, this patch introduces the libgcc.a library into the Solo5 distribution, which is highly problematic (what happens if the library comes from an old version of clang and the user is now using a new version? Should we keep track of clang versions? Why should we be responsible for distributing such a library that we do not build ourselves Can’t tools external to Solo5 obtain libgcc.a themselves? etc.).

In our view, the direction of this PR is unsatisfactory.

More generally, this raises the question of support for MacOS when building unikernels. It should be noted that we currently lack the (hardware) resources to approve such support (and CI alone is not sufficient). This is already more or less the case with Xen (even though I have a computer configured with it), and adding MacOS would place further strain on our maintenance work (given the complexity of the toolchain available on MacOS).

This must, of course, be weighed against the benefits that such a pull request might bring, and it does not seem to me that MacOS has been an actively pursued goal in the history of Solo5.

[samoht]

What would you recommend for running solo5 on macOS? I have a working prototype of a hvf backend (ie. using macOS hypervisor.framework to run unmodified hvt uniknernels) but if I cannot compile them on macOS this makes the UX a bit convoluted.

Is such a backed even weclome to the project? If not let me know how you want me to proceed (a separate package/project for the tender for instance ?)

[dinosaure]

Is such a backed even weclome to the project? If not let me know how you want me to proceed (a separate package/project for the tender for instance ?)

As I have just mentioned, we have very limited resources and do not have the hardware resources to support macOS (in order to test and maintain it in the long term). It is therefore difficult for us to commit to such support if we cannot test (and improve) it locally.

The example of Xen mentioned above is a perfect illustration of this: it is difficult for us to test and improve it, as we have to do so with the limited time we have and the single machine at our disposal for local testing (and to reproduce any bugs that Xen users might encounter) - it is, in fact, in precisely these cases that CI becomes ineffective.

These issues are open to debate, but the basic proposal cannot be accepted in its current form, particularly with regard to libgcc.a (and the issues surrounding the inclusion of this archive in the distribution).

[hannesm]

My take on this:

• if there's a macOS hvt tender, that would be great and very welcome
• a question is about the CI system, since as mentioned, there's no Apple hardware at the disposal at the current maintainers

Now, I don't understand much of macOS, but this PR seems that your proposed path is to use the cross-aarch64 infrastructure (developed for GNU/Linux to build aarch64 binaries on a x86_64 host). Taking a potential tender into consideration, it is likely this would be built using the normal host toolchain!? Now, for building unikernels, as you mention, there's some need to use some cross-compilers installed via 3rd party utilities. This is fine. I'm sceptical about copying a libgcc.a from aarch64-elf-gcc, and don't quite understand which symbols are missing, and why now a libgcc should be used, when the rest of the toolchain/linker/objcopy is from llvm. Plus can't you inject the path to the libgcc.a (-L/where/it/is) and -lgcc?

But as said, I'd appreciate some more elaboration on why "libgcc.a" is needed (in terms of which symbols are missing), and why can't a llvm-based library be used?

And maybe mixing solo5-cross-aarch64 for both cross-architecture builds and cross-macho-elf builds isn't the best idea? Maybe a new solo5-macos is the path to go - with both tender and cross-compilation setup for building elf binaries on macOS?

[samoht]

I think the only symbols that need libgcc are __extenddftf2 and __trunctfdf2 for the long double / double soft-float conversion pulled in by nolibc's vfprintf. Maybe a simpler patch would be to remove the long double from there completely? This seems to be truncated anyway...

edit: so yes, long double -> double in vprintf.c make all of this works with libgcc! I can open a PR with this if you think that's acceptable.

[samoht]

So mirage/ocaml-solo5#171 removes libgcc deps completely (on all platforms I think?) that makes the x-compilation to macOS much easier and so make this PR uneeded.

Apologies for the mess of PRs. I'll try to clean them up once I find something working properly, so feel free to either give an opinion on the direction of just skip those PRs for now.

[samoht]

Rebased and simplified per your feedback (force-pushed, so it's now a single commit).

The bundled libgcc is gone, which was the main objection, and it turned out to be unnecessary: downstream (mirage/ocaml-solo5#171) clang inlines the aarch64 atomics and nolibc no longer pulls long double soft-float, so the unikernel link references nothing from libgcc and the toolchain wrapper stopped passing -lgcc. I verified it by building a full aarch64 cross-compiler and linking a unikernel on macOS with no libgcc present. So Solo5 now ships a pure-LLVM toolchain (clang / ld.lld / llvm-objcopy) and no GNU runtime.

I also dropped the version.h / $SOLO5_VERSION changes (that was #642, now closed).

[samoht]

I am moving this PR to draft mode as @shym has kindly taken over the patch sets.