fails to link neovim with lto on RELDEBUG

[kada49]

I have been using wild to link my nightly build of neovim and typst for a while. When lto started being supported (--features=plugins) I tried to compile wild with that flag and to try to link neovim with it. It fails. Tried on 2 different machines.

My setup:

Fedora Linux 43 x86_64
latest wild commit (tried a couple revisions, always the same error)
latest neovim (tried a couple revisions, always the same error)

How to reproduce:

1. Clone neovim - git clone https://github.com/neovim/neovim
2. Clone wild - git clone httpd://github.com/wild-linker/wild
3. compile wild:

cd wild/wild
cargo install cargo install --locked --path . --features=plugins

4. put wild as ld in PATH
5. compile neovim

cd neovim
make CMAKE_BUILD_TYPE=RelWithDebInfo

6. Enjoy link error:

-- Using NVIM_VERSION: v0.12.0-dev-2477+g6beb1fe0cd
[677/682] Linking C executable bin/nvim
FAILED: [code=1] bin/nvim
: && /usr/lib64/ccache/cc -O2 -g -DRELDEBUG -flto=auto -fno-fat-lto-objects -Wl,--export-dynamic -rdynamic src/nvim/CMakeFiles/nvim_bin.dir/__/__/test/unit/fixtures/multiqueue.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/__/test/unit/fixtures/vterm_test.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/autocmd.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/buffer.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/command.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/deprecated.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/events.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/extmark.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/options.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/private/converter.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/private/dispatch.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/private/helpers.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/private/validate.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/tabpage.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/ui.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/vim.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/vimscript.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/win_config.c.o src/nvim/CMakeFiles/nvim_bin.dir/api/window.c.o src/nvim/CMakeFiles/nvim_bin.dir/arabic.c.o src/nvim/CMakeFiles/nvim_bin.dir/arglist.c.o src/nvim/CMakeFiles/nvim_bin.dir/autocmd.c.o src/nvim/CMakeFiles/nvim_bin.dir/base64.c.o src/nvim/CMakeFiles/nvim_bin.dir/buffer.c.o src/nvim/CMakeFiles/nvim_bin.dir/buffer_updates.c.o src/nvim/CMakeFiles/nvim_bin.dir/bufwrite.c.o src/nvim/CMakeFiles/nvim_bin.dir/change.c.o src/nvim/CMakeFiles/nvim_bin.dir/channel.c.o src/nvim/CMakeFiles/nvim_bin.dir/charset.c.o src/nvim/CMakeFiles/nvim_bin.dir/clipboard.c.o src/nvim/CMakeFiles/nvim_bin.dir/cmdexpand.c.o src/nvim/CMakeFiles/nvim_bin.dir/cmdhist.c.o src/nvim/CMakeFiles/nvim_bin.dir/context.c.o src/nvim/CMakeFiles/nvim_bin.dir/cursor.c.o src/nvim/CMakeFiles/nvim_bin.dir/cursor_shape.c.o src/nvim/CMakeFiles/nvim_bin.dir/debugger.c.o src/nvim/CMakeFiles/nvim_bin.dir/decoration.c.o src/nvim/CMakeFiles/nvim_bin.dir/decoration_provider.c.o src/nvim/CMakeFiles/nvim_bin.dir/diff.c.o src/nvim/CMakeFiles/nvim_bin.dir/digraph.c.o src/nvim/CMakeFiles/nvim_bin.dir/drawline.c.o src/nvim/CMakeFiles/nvim_bin.dir/drawscreen.c.o src/nvim/CMakeFiles/nvim_bin.dir/edit.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/buffer.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/decode.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/deprecated.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/encode.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/executor.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/fs.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/funcs.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/gc.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/list.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/typval.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/userfunc.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/vars.c.o src/nvim/CMakeFiles/nvim_bin.dir/eval/window.c.o src/nvim/CMakeFiles/nvim_bin.dir/event/libuv_proc.c.o src/nvim/CMakeFiles/nvim_bin.dir/event/loop.c.o src/nvim/CMakeFiles/nvim_bin.dir/event/multiqueue.c.o src/nvim/CMakeFiles/nvim_bin.dir/event/proc.c.o src/nvim/CMakeFiles/nvim_bin.dir/event/rstream.c.o src/nvim/CMakeFiles/nvim_bin.dir/event/signal.c.o src/nvim/CMakeFiles/nvim_bin.dir/event/socket.c.o src/nvim/CMakeFiles/nvim_bin.dir/event/stream.c.o src/nvim/CMakeFiles/nvim_bin.dir/event/time.c.o src/nvim/CMakeFiles/nvim_bin.dir/event/wstream.c.o src/nvim/CMakeFiles/nvim_bin.dir/ex_cmds.c.o src/nvim/CMakeFiles/nvim_bin.dir/ex_cmds2.c.o src/nvim/CMakeFiles/nvim_bin.dir/ex_docmd.c.o src/nvim/CMakeFiles/nvim_bin.dir/ex_eval.c.o src/nvim/CMakeFiles/nvim_bin.dir/ex_getln.c.o src/nvim/CMakeFiles/nvim_bin.dir/ex_session.c.o src/nvim/CMakeFiles/nvim_bin.dir/extmark.c.o src/nvim/CMakeFiles/nvim_bin.dir/file_search.c.o src/nvim/CMakeFiles/nvim_bin.dir/fileio.c.o src/nvim/CMakeFiles/nvim_bin.dir/fold.c.o src/nvim/CMakeFiles/nvim_bin.dir/fuzzy.c.o src/nvim/CMakeFiles/nvim_bin.dir/garray.c.o src/nvim/CMakeFiles/nvim_bin.dir/getchar.c.o src/nvim/CMakeFiles/nvim_bin.dir/grid.c.o src/nvim/CMakeFiles/nvim_bin.dir/hashtab.c.o src/nvim/CMakeFiles/nvim_bin.dir/help.c.o src/nvim/CMakeFiles/nvim_bin.dir/highlight.c.o src/nvim/CMakeFiles/nvim_bin.dir/highlight_group.c.o src/nvim/CMakeFiles/nvim_bin.dir/indent.c.o src/nvim/CMakeFiles/nvim_bin.dir/indent_c.c.o src/nvim/CMakeFiles/nvim_bin.dir/input.c.o src/nvim/CMakeFiles/nvim_bin.dir/insexpand.c.o src/nvim/CMakeFiles/nvim_bin.dir/keycodes.c.o src/nvim/CMakeFiles/nvim_bin.dir/linematch.c.o src/nvim/CMakeFiles/nvim_bin.dir/log.c.o src/nvim/CMakeFiles/nvim_bin.dir/lua/api_wrappers.c.o src/nvim/CMakeFiles/nvim_bin.dir/lua/base64.c.o src/nvim/CMakeFiles/nvim_bin.dir/lua/converter.c.o src/nvim/CMakeFiles/nvim_bin.dir/lua/executor.c.o src/nvim/CMakeFiles/nvim_bin.dir/lua/secure.c.o src/nvim/CMakeFiles/nvim_bin.dir/lua/spell.c.o src/nvim/CMakeFiles/nvim_bin.dir/lua/stdlib.c.o src/nvim/CMakeFiles/nvim_bin.dir/lua/treesitter.c.o src/nvim/CMakeFiles/nvim_bin.dir/lua/xdiff.c.o src/nvim/CMakeFiles/nvim_bin.dir/main.c.o src/nvim/CMakeFiles/nvim_bin.dir/map.c.o src/nvim/CMakeFiles/nvim_bin.dir/map_glyph_cache.c.o src/nvim/CMakeFiles/nvim_bin.dir/mapping.c.o src/nvim/CMakeFiles/nvim_bin.dir/mark.c.o src/nvim/CMakeFiles/nvim_bin.dir/marktree.c.o src/nvim/CMakeFiles/nvim_bin.dir/match.c.o src/nvim/CMakeFiles/nvim_bin.dir/math.c.o src/nvim/CMakeFiles/nvim_bin.dir/mbyte.c.o src/nvim/CMakeFiles/nvim_bin.dir/memfile.c.o src/nvim/CMakeFiles/nvim_bin.dir/memline.c.o src/nvim/CMakeFiles/nvim_bin.dir/memory.c.o src/nvim/CMakeFiles/nvim_bin.dir/menu.c.o src/nvim/CMakeFiles/nvim_bin.dir/message.c.o src/nvim/CMakeFiles/nvim_bin.dir/mouse.c.o src/nvim/CMakeFiles/nvim_bin.dir/move.c.o src/nvim/CMakeFiles/nvim_bin.dir/msgpack_rpc/channel.c.o src/nvim/CMakeFiles/nvim_bin.dir/msgpack_rpc/packer.c.o src/nvim/CMakeFiles/nvim_bin.dir/msgpack_rpc/server.c.o src/nvim/CMakeFiles/nvim_bin.dir/msgpack_rpc/unpacker.c.o src/nvim/CMakeFiles/nvim_bin.dir/normal.c.o src/nvim/CMakeFiles/nvim_bin.dir/ops.c.o src/nvim/CMakeFiles/nvim_bin.dir/option.c.o src/nvim/CMakeFiles/nvim_bin.dir/optionstr.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/dl.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/env.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/fileio.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/fs.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/input.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/lang.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/mem.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/proc.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/pty_proc_unix.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/shell.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/signal.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/stdpaths.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/time.c.o src/nvim/CMakeFiles/nvim_bin.dir/os/users.c.o src/nvim/CMakeFiles/nvim_bin.dir/path.c.o src/nvim/CMakeFiles/nvim_bin.dir/plines.c.o src/nvim/CMakeFiles/nvim_bin.dir/popupmenu.c.o src/nvim/CMakeFiles/nvim_bin.dir/profile.c.o src/nvim/CMakeFiles/nvim_bin.dir/quickfix.c.o src/nvim/CMakeFiles/nvim_bin.dir/regexp.c.o src/nvim/CMakeFiles/nvim_bin.dir/register.c.o src/nvim/CMakeFiles/nvim_bin.dir/runtime.c.o src/nvim/CMakeFiles/nvim_bin.dir/search.c.o src/nvim/CMakeFiles/nvim_bin.dir/sha256.c.o src/nvim/CMakeFiles/nvim_bin.dir/shada.c.o src/nvim/CMakeFiles/nvim_bin.dir/sign.c.o src/nvim/CMakeFiles/nvim_bin.dir/spell.c.o src/nvim/CMakeFiles/nvim_bin.dir/spellfile.c.o src/nvim/CMakeFiles/nvim_bin.dir/spellsuggest.c.o src/nvim/CMakeFiles/nvim_bin.dir/state.c.o src/nvim/CMakeFiles/nvim_bin.dir/statusline.c.o src/nvim/CMakeFiles/nvim_bin.dir/strings.c.o src/nvim/CMakeFiles/nvim_bin.dir/syntax.c.o src/nvim/CMakeFiles/nvim_bin.dir/tag.c.o src/nvim/CMakeFiles/nvim_bin.dir/terminal.c.o src/nvim/CMakeFiles/nvim_bin.dir/testing.c.o src/nvim/CMakeFiles/nvim_bin.dir/textformat.c.o src/nvim/CMakeFiles/nvim_bin.dir/textobject.c.o src/nvim/CMakeFiles/nvim_bin.dir/tui/input.c.o src/nvim/CMakeFiles/nvim_bin.dir/tui/terminfo.c.o src/nvim/CMakeFiles/nvim_bin.dir/tui/termkey/driver-csi.c.o src/nvim/CMakeFiles/nvim_bin.dir/tui/termkey/driver-ti.c.o src/nvim/CMakeFiles/nvim_bin.dir/tui/termkey/termkey.c.o src/nvim/CMakeFiles/nvim_bin.dir/tui/tui.c.o src/nvim/CMakeFiles/nvim_bin.dir/ugrid.c.o src/nvim/CMakeFiles/nvim_bin.dir/ui.c.o src/nvim/CMakeFiles/nvim_bin.dir/ui_client.c.o src/nvim/CMakeFiles/nvim_bin.dir/ui_compositor.c.o src/nvim/CMakeFiles/nvim_bin.dir/undo.c.o src/nvim/CMakeFiles/nvim_bin.dir/usercmd.c.o src/nvim/CMakeFiles/nvim_bin.dir/version.c.o src/nvim/CMakeFiles/nvim_bin.dir/viml/parser/expressions.c.o src/nvim/CMakeFiles/nvim_bin.dir/viml/parser/parser.c.o src/nvim/CMakeFiles/nvim_bin.dir/vterm/encoding.c.o src/nvim/CMakeFiles/nvim_bin.dir/vterm/keyboard.c.o src/nvim/CMakeFiles/nvim_bin.dir/vterm/mouse.c.o src/nvim/CMakeFiles/nvim_bin.dir/vterm/parser.c.o src/nvim/CMakeFiles/nvim_bin.dir/vterm/pen.c.o src/nvim/CMakeFiles/nvim_bin.dir/vterm/screen.c.o src/nvim/CMakeFiles/nvim_bin.dir/vterm/state.c.o src/nvim/CMakeFiles/nvim_bin.dir/vterm/vterm.c.o src/nvim/CMakeFiles/nvim_bin.dir/window.c.o src/nvim/CMakeFiles/nvim_bin.dir/winfloat.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/cjson/fpconv.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/cjson/lua_cjson.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/cjson/strbuf.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/mpack/conv.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/mpack/lmpack.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/mpack/mpack_core.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/mpack/object.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/mpack/rpc.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/xdiff/xdiffi.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/xdiff/xemit.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/xdiff/xhistogram.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/xdiff/xpatience.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/xdiff/xprepare.c.o src/nvim/CMakeFiles/nvim_bin.dir/__/xdiff/xutils.c.o -o bin/nvim  /home/kada49/Desktop/neovim/.deps/usr/lib/libluv.a  /home/kada49/Desktop/neovim/.deps/usr/lib64/liblpeg.a  /home/kada49/Desktop/neovim/.deps/usr/lib64/libtree-sitter.a  /home/kada49/Desktop/neovim/.deps/usr/lib64/libutf8proc.a  /home/kada49/Desktop/neovim/.deps/usr/lib64/libunibilium.a  /home/kada49/Desktop/neovim/.deps/usr/lib/libluajit-5.1.a  -fstack-protector-strong  -lm  -lutil  /home/kada49/Desktop/neovim/.deps/usr/lib/libuv.a  -ldl  -lrt && :
wild: error: Failed copying from /tmp/cc3dzI8H.debug.temp.o (47360 (185/0)) to output file
  Caused by:
    Tried to copy a symbol in a section we didn't load. symbol `multiqueue.c.492e1003` (74722 local=6) in file #47360 (185/0) (/tmp/cc3dzI8H.debug.temp.o) (NON_INTERPOSABLE | DIRECT)

collect2: error: ld returned 255 exit status
ninja: build stopped: subcommand failed.
make: *** [Makefile:102: nvim] Error 1

P.S.: I don't know whether there is a better way to get logs, so this is what I got. If you know pls let me know

[davidlattimore]

Thanks. From this and other bug reports, it seems like I should change the status of LTO from supported-but-disabled-by-default to experimental-still-work-in-progress.

[fahdfady]

I'm investigating this now & will give it a shot. Would like to be assigned on this.

[fahdfady]

It seems that the issue is happening during writing debug symbols, not related to LTO, eventhough the linking passes if we don't pass LTO which is weird.
Why would i suggest the issue is writing debug symbols

1. the error mentions a .debug.temp.o file wild: error: Failed copying from /tmp/ccJCrrIE.debug.temp.o
2. tried to use --strip-all in linker arguments and didn't reproduce the error

make CMAKE_BUILD_TYPE=RelWithDebInfo CC=gcc CXX=g++ ENABLE_LTO=ON CMAKE_EXTRA_FLAGS="-DCMAKE_EXE_LINKER_FLAGS=-Wl,--strip-all"

3. used --strip-debug to make sure it's specifically about debug symbols, and it surely is by now.

make CMAKE_BUILD_TYPE=RelWithDebInfo CC=gcc CXX=g++ ENABLE_LTO=ON CMAKE_EXTRA_FLAGS="-DCMAKE_EXE_LINKER_FLAGS=-Wl,--strip-debug"

[fahdfady]

Additionally, I tried clang instead of gcc and it compiled with no problems at all.

make CMAKE_BUILD_TYPE=RelWithDebInfo CC=clang CXX=clang++

even explicitly enabling LTO to make sure it's not the problem here:

make CMAKE_BUILD_TYPE=RelWithDebInfo CC=clang CXX=clang++ ENABLE_LTO=ON

I suspect that the problem will be with how gcc handles debug symbols before reaching our linker, possibly only happens with LTO enabled.

[fahdfady]

The error comes from the function that writes elf debug symbols. see: https://github.com/wild-linker/wild/blob/main/libwild/src/elf_writer.rs#L1604

I still don't have any hint about LTO till now. Still investigating.