RFC: #[export_visibility = ...] attribute.

[anforowicz]

This RFC proposes to add #[export_visibility = …] attribute, which seems like a reasonable way to address the following issues:

• Private #[no_mangle] symbols are exported from a cdylib rust#98449
• Ability from the top-level of the compilation not to mark #[no_mangle] items exported from shared library rust#73958

This RFC complements the -Zdefault-visibility=... command-line flag, which is tracked in rust-lang/rust#131090

This PR replaces the Major Change Proposal (MCP) at rust-lang/compiler-team#881
(/cc @bjorn3, @ChrisDenton, @chorman0773, @joshtriplett, @mati865, @workingjubilee, and @Urgau who have kindly provided feedback in the Zulip thread associated with that MCP)

/cc @tmandry from rust-lang/rust-project-goals#253, because one area where this RFC seems needed is FFI tooling

Rendered

[bjorn3]

Should only be the case for libraries. For binaries all functions are already made not-exported.

[bjorn3]

I would have expected all of Chromium to use a single rust allocator rather than use a different one for each DSO. Why is that not the case?

[anforowicz]

I would have expected all of Chromium to use a single rust allocator rather than use a different one for each DSO. Why is that not the case?

Is that really a requirement if foo.so doesn't export any functions that return pointers to Rust-related objects? I would expect in such a case that which Rust allocator / standard library / etc is used would be an internal implementation detail of foo.so. IIUC this detail leaks out only because of an unintentional export of a cxx-generated, internal symbol.

But to try to answer the question - the same allocator is statically linked into Chromium binaries. This means that an executable and an .so may end up with a separate copy of the same global data structures of the allocator.

[bjorn3]

Is that really a requirement if foo.so doesn't export any functions that return pointers to Rust-related objects?

It is not a requirement. I'm just surprised that Chromium copies the entire rust standard library between the dylib and executable rather than using the copy from the dylib in the executable to save space.

[anforowicz]

Is that really a requirement if foo.so doesn't export any functions that return pointers to Rust-related objects?

It is not a requirement. I'm just surprised that Chromium copies the entire rust standard library between the dylib and executable rather than using the copy from the dylib in the executable to save space.

That is indeed a bit unfortunate. I think this is to some extent based on the following:

• Chromium requirement to use an external linker
• Assumption that only rlibs / static_libs can be linked by an external linker, and that an external linker wouldn't be able to handle dylibs

But thank you for bringing this up - maybe this should indeed be treated as an alternative fix for https://crbug.com/418073233. I am not sure what the next steps should be for this aspect:

• Maybe I should open a bug (either under https://github.com/rust-lang/rust/issues, or under https://crbug.com) to move this discussion elsewhere?
• Maybe before opening a bug I should first learn more about dylibs...

[bjorn3]

For most use cases rather than specifying the exact symbol visibility (which may not even be supported by the object file format, like interposable on pe/coff or (with the default two-level namespaces) mach-o) I think having just a way to force SymbolExportLevel::Rust rather than the default SymbolExportLevel::C would be a better idea. This causes it to still be exported from rust dylibs (as necessary to avoid linker errors depending on the exactly when rustc decides to codegen functions), but prevents it from being exported from cdylibs. It doesn't work for staticlibs currently, but for those if you want to limit symbol visibility you have to specify your own version script during linking anyway to prevent exporting all rust mangled symbols too.

[anforowicz]

For most use cases rather than specifying the exact symbol visibility (which may not even be supported by the object file format, like interposable on pe/coff or (with the default two-level namespaces) mach-o) I think having just a way to force SymbolExportLevel::Rust rather than the default SymbolExportLevel::C would be a better idea. This causes it to still be exported from rust dylibs (as necessary to avoid linker errors depending on the exactly when rustc decides to codegen functions), but prevents it from being exported from cdylibs.

The fact that you are distinguishing between dylibs and cdylibs makes me think that you assume that linking is driven by rustc. If so, then this may not apply to Chromium, which uses an external linker.

It doesn't work for staticlibs currently, but for those if you want to limit symbol visibility you have to specify your own version script during linking anyway to prevent exporting all rust mangled symbols too.

That's not 100% correct - instead of using a version script, one may also use -Zdefault-visibility=hidden.

[bjorn3]

The fact that you are distinguishing between dylibs and cdylibs makes me think that you assume that linking is driven by rustc. If so, then this may not apply to Chromium, which uses an external linker.

The Chromium case is effectively equivalent to using staticlibs, not to using rust dylibs/cdylibs.

That's not 100% correct - instead of using a version script, one may also use -Zdefault-visibility=hidden.

That doesn't apply to the standard library unless you go out of your way using unstable features to recompile the standard library.

[anforowicz]

The fact that you are distinguishing between dylibs and cdylibs makes me think that you assume that linking is driven by rustc. If so, then this may not apply to Chromium, which uses an external linker.

The Chromium case is effectively equivalent to using staticlibs, not to using rust dylibs/cdylibs.

Ack / agreed.

That's not 100% correct - instead of using a version script, one may also use -Zdefault-visibility=hidden.

That doesn't apply to the standard library unless you go out of your way using unstable features to recompile the standard library.

Thank you for bringing up this point. This probably should be explicitly addressed by the RFC (*), but I am not sure if I agree with your conclusions so far. This is because:

• Chromium does in fact compile the standard library within Chromium's build system (e.g. see build/rust/std/rules/BUILD.gn auto-generated from standard library's Cargo.toml files)
• But using a pre-built standard library doesn't necessarily make #[export_visibility = ...] less useful, because there may be other actions that can be taken to change the behavior of the standard library. For example, the RFC discusses changing the behavior of #[export_name = ...] and/or #[no_mangle] (in a future Rust language edition) so that in the future these attributes imply #[export_visibility = "inherit"] rather than #[export_visibility = "interposable"]. So maybe a similar change can/should be applied to #[rustc_std_internal_symbol]? And while users of #[no_mangle] and/or #[export_name = ...] may actually want the public-export behavior of these attributes, I think this is not the case for standard library symbols (so maybe the change to inherit behavior could even be done within the current language edition; or maybe trigerred by -Zdefault-visibility although this then would get quite close to the -Zdefault-visibility-for-c-exports=... alternative from the RFC).

(*) I am not sure what the right process is here. Should I add commits to the RFC as we keep discussing here? Should I first give people an opportunity to review the first draft?

[petrochenkov]

I think this is a good opportunity to expand the design space (and documentation) of "various levels of exportendess" a bit, even if the resulting proposal for export_visibility specifically stays more or less the same.

There are multiple attributes that targets this similar space (export_visibility, linkage, used, rustc_std_internal_symbol), so it would be good to somehow target them together properly.

What I'd like to see is a table of "levels of exportedness" combined with the kinds of end artifacts, and how we can users can express all those levels with the attributes listed above.

• a symbol only visible inside an object file
• a symbol visible outside of an object file but not outside of a cdylib/dylib/executable
• a symbol visible outside of a cdylib
• a symbol visible outside of a rust dylib
• a symbol visible outside of an executable
• a symbol visible outside of a cdylib/dylib/executable but not some other crate type
• a symbol visible outside of an object file inside a rlib/staticlib, but not outside of it
• a symbol visible outside of an X but only for LTO, after that it is only visible outside of Y (I think I've seen some issue about this in the tracker)
• where is the information about the exportedness level stored? in which case it can be stored in the target's object format (e.g. ELF, COFF or even archive metadata) and in which cases it needs to be stored in rmeta and require rustc for interpreting it (e.g. rustc must be used for linking).
• if a symbol is visible outside of X, does it mean that it is used in some sense? who can optimize that used symbol away in each case?
• if a symbol is used(X), does it also mean that it is visible outside of Y?
• which of the case combinations above make sense?

In particular, one of my requirements is that #[rustc_std_internal_symbol] should be expressible as an alias to several more fine-grained and single-purpose attributes available to users. IIRC, it had some LTO-related visibility requirement in particular.
Also, all symbols hard-coded in the compiler by name (there were such symbols in the past, not sure about now, some were migrated to rustc_std_internal_symbol) should be expressible by the same fine-grained attributes as well.

[bjorn3]

a symbol only visible inside an object file

This is something only rustc must be allowed to do (other than for symbols defined in inline asm called from within the same inline asm block). Only rustc knows if all callers will end up in the same object file as the definition and it doesn't provide any guarantees around when this happens. So exposing this to the user is a stability hazard.

where is the information about the exportedness level stored? in which case it can be stored in the target's object format (e.g. ELF, COFF or even archive metadata) and in which cases it needs to be stored in rmeta and require rustc for interpreting it (e.g. rustc must be used for linking).

For regular functions and #[rustc_std_internal_symbol] we have to store it in the rmeta and use a version script as at compile time we don't yet know if the object file ends up in a rust dylib or cdylib.

a symbol visible outside of an object file inside a rlib/staticlib, but not outside of it

For rlib this doesn't make sense. There is no way to make rlibs a symbol export boundary without introducing an expensive link/object file rewrite step for each individual rlib. For staticlib it would be nice to have a symbol export boundary, but unfortunately we don't have one right now even for SymbolExportLevel::Rust (which really shouldn't be exported from staticlibs and for which we already support not exporting them from cdylibs) except I believe when we do (fat?) LTO as in that case all object files in the staticlib get optimized together allowing them to be internalized in the output object.

a symbol visible outside of a cdylib

This makes sense to me. See the end of my comment.

a symbol visible outside of a rust dylib

This has to always be the case if it is visible outside of the object file. The very point of rust dylibs is that rust code in a separate DSO can call any public function, which thanks to cross-crate inlining can call effectively every function that rustc wouldn't make private to the current object file. And again, rustc doesn't provide any guarantees when this happens, so allowing you to not export symbols from a rust dylib is a stability hazard.

if a symbol is visible outside of X, does it mean that it is used in some sense? who can optimize that used symbol away in each case?

Yes.

if a symbol is used(X), does it also mean that it is visible outside of Y?

No

Also, all symbols hard-coded in the compiler by name (there were such symbols in the past, not sure about now, some were migrated to rustc_std_internal_symbol) should be expressible by the same fine-grained attributes as well.

rust_eh_personality should be the only remaining symbol with a hard coded name once rust-lang/rust#141061 lands (which removes the unmangled __rust_no_alloc_shim_is_unstable in favor of a mangled __rust_no_alloc_shim_is_unstable_v2). We unfortunately can't mangle it's name as LLVM hard codes it.

IIRC, it had some LTO-related visibility requirement in particular.

Not really aside from the visibility information we already tell the linker (export from rust dylib, don't export from cdylib).

Currently rustc internally works with three different symbol export levels:

• Not exported from the object file. This is done using internal symbol linkage.
• SymbolExportLevel::Rust. This exports from a rust dylib, but not a cdylib. This is for #[rustc_std_internal_symbol] and regular rust functions that are not #[no_mangle] that aren't made private to the object file either
• SymbolExportLevel::C. This exports from all crate types (for bin only when -Zexecutable-export-symbols is passed). This is enabled using #[no_mangle].

It makes sense to me to allow SymbolExportLevel::Rust for #[no_mangle] symbols for C/C++ code that ends up getting linked into the same cdylib.

[anforowicz]

It makes sense to me to allow SymbolExportLevel::Rust for #[no_mangle] symbols for C/C++ code that ends up getting linked into the same cdylib.

I think this probably should be captured somehow as one of the alternatives in the RFC. Is there a specific syntax that you have in mind here? I guess one option would be to have a #[symbol_export_level = "rust"] or maybe #[no_c_level_symbol_export] (or #[no_cdylib_symbol_export]?), although maybe the names could be improved somehow.

[bjorn3]

I think this probably should be captured somehow as one of the alternatives in the RFC.

👍

#[symbol_export_level = "rust"] or maybe #[no_c_level_symbol_export]

I don't think this is a good name as it is still meant to be usable from C, just not outside of the linked DSO.

#[no_cdylib_symbol_export]

This would be an option, although ideally if we manage to stop exporting all symbols from staticlibs, I would like the same attribute to be usable to prevent export from both cdylib and staticlib, so it should probably not mention cdylib in the name. I don't have suggestions for a better name though.

[chorman0773]

Frankly, if we have #[no_cdylib_symbol_export], I'd like the inverse for imports at least, so that it's possible to export symbols defined in C (or another language) from a cdylib.