Nuget package 6.0.0-beta1 missing LLVM C API symbols on Linux

[tritao]

As the title says, the latest Nuget package for libllvm is missing C API symbols.

joao@joao-HP:~/dev/dotnet/corert/packages/libllvm$ llvm-nm-6.0 5.0.1/runtimes/linux-x64/native/libLLVM.so | grep LLVMModuleCreateWithName
0000000000838d20 T LLVMModuleCreateWithName
0000000000838db0 T LLVMModuleCreateWithNameInContext
joao@joao-HP:~/dev/dotnet/corert/packages/libllvm$ llvm-nm-6.0 6.0.0-beta1/runtimes/linux-x64/native/libLLVM.so | grep LLVMModuleCreateWithName
joao@joao-HP:~/dev/dotnet/corert/packages/libllvm$ 

This breaks CoreRT under Linux.

[mjsabby]

Thanks @tritao. We'll get this squared away this week, I also noticed a similar break in CoreRT. FYI @morganbr

[KodrAus]

@mjsabby is there anything we can do to help fix this up?

[KodrAus]

I'm not sure how helpful it would be to anyone, but I'm working around this in corert right now by building my own libLLVM and patching it in (since corert unpacks packages and links the native objects directly).

On OSX

Clone the llvm project:

$ git clone -b release_60 https://github.com/llvm-mirror/llvm

Configure LLVM to build as a single static binary:

$ mkdir llvm-build
$ cd llvm-build
$ cmake -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_LINK_LLVM_DYLIB=ON -DBUILD_SHARED_LIBS=OFF ../llvm

Build LLVM:

$ make

Go for a long walk.

If everything goes well, you should find llvm-build/lib/libLLVM.dylib.

[mjsabby]

@KodrAus thanks. Working on this.

[KodrAus]

Thanks @mjsabby!

I imagine it wouldn't need to change all that much, but do you think it would be worth setting up a repository for libLLVM that just contains an Azure Pipeline or something for fetching/building an LLVM binary on the supported platforms, bundling those up into a nupkg and publishing to nuget.org? That might give us some more visibility into how the package is produced?

[mjsabby]

@KodrAus Yup I so desperately want to do that :)

Happy to create a Microsoft sponsored account if you wanted to take a stab at it.

[KodrAus]

Sure 👍 I'm keen to try spike something out. I'll just spin up a repo and see where it goes. I'll open an issue or send a ping if I get anywhere 😃

[KodrAus]

So I've actually been able to make some progress on this. Building for OSX and Linux seems to work out ok, but I'm having some trouble getting a single .dll out of the Windows build.

@mjsabby I'm guessing that's what the vcxproj in this repo is all about? Are you able to share some details about how you build LLVM for Windows at the moment?

[mjsabby]

Yes the vcxproj is for that. In fact, you need to have the code also be built on Linux and macOS. Because that's what the PInvokes call into since neither platform exposes all the arches.

Are you using a hosted Azure Pipeline instance? Let me know the details and we can put all the relevant info in the repo.

Really appreciate you taking this!

[KodrAus]

Ah right. Sure! The repo I'm working on is here, with the current Azure Pipeline looking something like this. It's still proof-of-concept quality stuff, but does manage to get the native bits produced in some form on Windows, OSX and Linux.

[KodrAus]

I wasn't able to successfully link up a Windows build using the vcxproj from this repo on my machine, but LLVM takes a really long time to build locally on my poor old Windows box (~3hrs vs ~1hr on the build agent), so I might just try running it in the Windows pipeline after LLVM's build and see how I go getting it to link up there.

[mjsabby]

@KodrAus do you mind if we copy over your azure pipeline yml over to an MSFT Azure Pipelines? I'll then get beefier VMs for this task.