How to work with the missing LD_LIBRARY_PATH after setcap?

[kc-eos]

Hello:

I am trying to run my project with libvma as non-root user, it is fine to run the sample sockperf with libvma after setting up the necessary permission with setuid and setcap mentioned in the user guide.

However, when I try to run it with my project, I found that LD_LIBRARY_PATH is missing / ignored by LD linker as it is running in a "secure-execution" mode after granting the capabilities to the executable. My project is in a structure where the executable has to link with several inhouse-built library, e.g.

 my_gateway -> libmy-core.so , libmy-network.so, libmy-util.so
 my_engine -> libmy-core.so , libmy-network.so, libmy-util.so
...

Then, without the LD_LIBRARY_PATH, I cannot start up my application due to ... : error while loading shared libraries: ....

I wonder

1. is there any recommendation to workaround this security-protection; and
2. will it be possible to run libvma with a wrapper script such that I don't need to run setcap on all the executables

PS1. Tried to bind rpath during compilation, but it is not perfect as the same .so will be deployed to different account without a consistent path.
PS2. It is running on RHEL 7.4

[igor-ivanov]

Hello,
try to load libvma using dlopen. See https://docs.nvidia.com/networking/display/vmav972lts/vma+configuration#src-106008374_VMAConfiguration-LoadingVMADynamically

[kc-eos]

Hi @igor-ivanov , thanks for the reply.

I want to confirm my understanding.

If we do dlopen, does libvma still need certain Linux capabilities to bypass the network stack? If so, how can we startup the process without doing setcap on the executable (which cause LD_LIBRARY_PATH not effective).

I have tried to setup the capabilities programmatically (via libcap), but it's also not working because, by design, Linux capabilities can only be dropped, but never be raised at runtime.

[igor-ivanov]

Related permissions are still needed in dlopen case too.

[kc-eos]

To share with anyone facing similar problem, my team has evaluated a number of options and finally decided to go with capsh , which has been available since RHEL8.

According to the man-page on capabilities: https://man7.org/linux/man-pages/man7/capabilities.7.html, by setting up the ambient capabilities, the capabilities set will be preserved to the child process and this will not trigger the ld.so secure-execution mode.

 Ambient (since Linux 4.3)
       This is a set of capabilities that are preserved across an
       execve(2) of a program that is not privileged.  The
       ambient capability set obeys the invariant that no
       capability can ever be ambient if it is not both permitted
       and inheritable.

       The ambient capability set can be directly modified using
       prctl(2).  Ambient capabilities are automatically lowered
       if either of the corresponding permitted or inheritable
       capabilities is lowered.

       Executing a program that changes UID or GID due to the
       set-user-ID or set-group-ID bits or executing a program
       that has any file capabilities set will clear the ambient
       set.  Ambient capabilities are added to the permitted set
       and assigned to the effective set when execve(2) is
       called.  If ambient capabilities cause a process's
       permitted and effective capabilities to increase during an
       execve(2), this does not trigger the secure-execution mode
       described in ld.so(8)

Then one can easily run the target executable in a wrapper script. (PS: runtime sudo privilege is still needed.)

#!/bin/sh
CMD_LINE="sudo capsh --user=`whoami` --caps='cap_setpcap+p cap_net_raw,cap_net_admin+ip' --addamb='cap_net_raw,cap_net_admin' -- -c '$@'"
eval $CMD_LINE

[kc-eos]

@igor-ivanov , you may consider closing this ticket Thanks for following up.

[igor-ivanov]

thank you for sharing solution details that could be useful for someone else.