What is rpath linux

Creating relocatable Linux executables by setting RPATH with $ORIGIN

Jul 27, 2019 · 2 min read

Lots of modern C/C++ projects leverage Autotools to create GNU building system e.g. generate make files based on the platform. Executable files (binaries) are generated during the make/compile process, and can be executed locally on the machine where the compilation is performed. However, if the same executable were moved onto a different machine, or simply a different folder on the same machine, the “library not found” error might be encountered while running the executable.

What is RPATH and $ORIGIN

RPATH stands for run-time s earch path. According to Wikipedia, “rpath designates the run-time search path hard-coded in an executable file or library. Dynamic linking loaders use the rpath to find required libraries.” Dynamic linking is a sort of “lazy” linking of required shared libraries not during the stage of compiling but the later stage of running one executable. A path to the shared libraries will be encoded into the header of the executable if rpath is set, overriding or supplementing the system default shared library searching paths, similar as one extends the chain of PATH system variable.

$ORIGIN is a special variable that indicate actual executable filename. It is resolved to where the executable is at run-time, and can be quite useful when setting RPATH.

How to check the value of RPATH

There are various ways of checking the RPATH value for an executable or library. objdump, readelf and chrpath are 3 of the frequently used utilities.

$ objdump -x path/to/executable | grep RPATH

$ readelf -d path/to/executable | head -20

Don’t be surprised if one executable has no RPATH value set.

Below screenshot provides an example of readelf output with RUNPATH and $ORIGIN:

Why and How to set RPATH

The aim of RPATH setting is to change the library search path so that one executable or library is able to link the desired dependent libraries during run-time.

There are 2 different stages RPATH could be set:

During compilation time

will tell the linker to build and run the executable under the specified library path, usually used to override the default library paths.

After compilation before execution

Above command could fail if no rpath was set previously for the executable.

Try below command with patchelf utility, which won’t complain about an unset rpath, and will get RUNPATH set to achieve similar target.

$ patchelf — set-rpath ‘$ORIGIN/path/to/library’

Conclusion

After setting RPATH to the directory where dependent libraries locate, with the help of $ORIGIN, an Linux executable could be run successfully either in place, or be moved around to different directories or even hosts, achieving the desired properties of relocatability and flexibility.

Источник

What is rpath linux

Recently I’ve been working on the Comedi ebuild in my Gentoo overlay, wrestling with the following error:

While tracking this down the source of this error, I learned a lot about dynamic linking on Linux, so here’s the condensed version.

RPATH , RUNPATH , and LD_LIBRARY_PATH . The current state of affairs is well summarized on the Debian wiki, which lists the library search path:

1. the RPATH binary header (set at build-time) of the library causing the lookup (if any)
2. the RPATH binary header (set at build-time) of the executable
3. the LD_LIBRARY_PATH environment variable (set at run-time)
4. the RUNPATH binary header (set at build-time) of the executable
5. /etc/ld.so.cache
6. base library directories ( /lib and /usr/lib )

There was a big dust-up between Debian and libtool back in 1999 when libtool-generated RPATH s caused problems during the libc5 to libc6 transition. The mailing list discussion makes for amusing and informative reading, if you’ve got a spare hour or two ;). If not, you should at least read the opening post, the description of competing contracts, and Buddha Buck’s description of how linking works, although I imagine things might have changed since then. The Debian / libtool compromise (don’t set RPATH by default for directories in the dynamic linker search path) was implemented in libtool 1.5.2 (released in 2004, see the Debian wiki), so this is not as big an issue as it once was.

By the way, it looks like RUNPATH was added since 1999 as a version of RPATH that did not override LD_LIBRARY_PATH , which is good, since LD_LIBARY_PATH gives you a way to link against libraries in, say, /var/tmp/portage/sci-libs/comedilib-9999/work/comedilib-9999/lib/.libs to test your executable before installation.

Anyhow, issues with rpaths persist. Since it both hard to predict all installation configurations at compile time, and tools to change rpaths later on (i.e. chrpath and patchelf) aren’t able to increase the size of the rpath string on Linux (they can on Solaris, because Solaris leaves a bit of padding at the end of the dynamic string table in the compiled ELF file). This means you will have trouble moving a library out of the standard library path into some out-of-the-way location. However, in the more common case of installing a library into the standard library path, chrpath is the tool you need, and it solved my comedilib QA issue.

Along the way, I ran across two other interesting posts by Diego PettenГІ about not bundling libraries.

Setting RPATH and RUNPATH

Most of the time, you’ll want to avoid setting RPATH and RUNPATH and just use libraries in your usual linking path. However, sometimes they are useful. For example, SimulAVR depends on the AVR-specific libbfd , which is hopefully not in your usual path. On Gentoo, crossdev installs under /usr/lib/binutils/ :

When you link against this library, you’ll want to set RUNPATH so you don’t have to remember to use LD_LIBRARY_PATH every time you run simulavr . Of course, if you switch to a different binutils version (e.g. not git ), you’ll need to fix the RUNPATH to point to the new target (or just rebuild simulavr against the new version).

Since you’re probably not calling the linker directly when you build simulavr , you’ll want to set some linker flags at configure time:

The relevant linker flags are -rpath and —enable-new-dtags . Without —enable-new-dtags , you’ll just set the RPATH flag, which is probably not what you want. With —enable-new-dtags , you’ll set both RPAH and RUNPATH to the same value. From ld(1) :

The DT_RPATH entries are ignored if DT_RUNPATH entries exist.

so setting both is the same as just setting RUNPATH (except for tools like chrpath which are only designed to handle a single tag). You can use readelf to see if the tags were set:

On Gentoo, —enable-new-dtags has been the default since 2005, but explicitly including the flag can’t hurt ;).

Источник

@rpath what?

In short: dynamic linking happened. Dynamic linking — what it is? It’s an operation that happens when part of the code spreads across different files (called libraries), and the binary content of the library is loaded in runtime.

A dynamic linker (which is a system tool) finds a symbol (e.g., function), located in the dynamic library. Next, loads the code into memory and assign memory address with the symbol. This way running program can find the implementation of the symbol stored in an external library (shared library). Action happens at the beginning of the program execution process before the application code starts executing (may happen later if symbols are lazily loaded)

It means that while we develop an application, we can miss it until the program runs.

If linking is not properly set up for the executable, this is the wake-up call we’ll notice at the very end of the development process:

• while executing an App binary from App.app bundle
• dyld — the dynamic linker tool from macOS system
• can’t load Allthethings binary from the Allthethings.framework bundle
• due to image not found when a search for @rpath/Allthethings.framework/Allthethings binary

The short answer is: because the file is not there.

Where exactly is there? the path is @rpath/Allthethings.framework/Allthethings. This value was set while building the application. It was passed to the linker (not the compiler).

@rpath stands for Runpath Search Path.

• In the Xcode, it’s set with LD_RUNPATH_SEARCH_PATH setting.
• In ld command tool it’s set with -rpath parameter when linking.
  So it’s a search path for the linker. Runtime Search Path instructs the dynamic linker to search a list of paths in order, to locate the dynamic library.

The value of the parameter may be an absolute path (or multiple paths) to a directory, e.g.: /usr/private/lib or @executable_path/Frameworks .

For the relative path, we can use one of two substitution symbols:

• @loader_path — resolves with the path to the directory containing the Mach-O binary which contains the load command. Thus, in every binary, it’s resolved to a different path, that said it’s the path to the library doing the loading of a given library.
  It’s not necessary the same library. Imagine that library libc.dylib loads another library libqq.dylib . In this case, loader_path will point to where libc.dylib is located (because this is caller is located).
• @executable_path — resolves to the absolute path of the executable, eg. /private/var/mobile/Containers/Bundle/Application/0F2C2461-A68B-4ABA-A604-B88E6E9D1BB1/App.app/

At runtime, dyld uses the runpath when searching for dynamic libraries whose load path begins with @rpath.

How the path @rpath/Allthethings.framework/Allthething from the error message is resolved then? We don’t know yet. To solve that, we need to learn about possible rpath for the binary. In this example the binary to investigate is locate at /private/var/mobile/Containers/Bundle/Application/0F2C2461-A68B-4ABA-A604-B88E6E9D1BB1/App.app/App .

Runpath Search Path is stored as a part of the binary, as an LC_RPATH command. To read the value of the section, we can use the command line tool otool -l 0F2C2461-A68B-4ABA-A604-B88E6E9D1BB1/App.app/App and search for LC_RPATH in the output:

there are one or more entries like this.

Now we can substitute @rpath with the found path and verify why the file is missing.

To fix the initial problem we’ll need to either:

• copy binary to the expected directory
• add another Runpath Search Path by modifying the value of LD_RUNPATH_SEARCH_PATH (or -rpath )

I’ve said the @rpath is part of the binary and is set during the compilation process (linking phase) with a given value of the parameters. However, if we need to modify the @rpath manually, e.g., as a part of installation phase — there’s an app for that: install_name_tool

install_name_tool

install_name_tool changes dynamic shared library install names and manipulate Runpaths.

To add new path:

To delete added path (we can only delete path added with -add_rpath ):

To change the existing path:

Embedded Binaries

We just mastered dynamic linking problem-solving. The meaning of the Embedded Binaries section from the Xcode is clear:

Xcode copies files from «Embedded Binaries» section to the place where LD_RUNPATH_SEARCH_PATH points to.

Controling linker with environment variables

The behaviour of dyld may be controlled by environment variables, however If System Integrity Protection is enabled, environment variables are ignored when executing binaries protected by System Integrity Protection — that is most of the time.

PS. it’s different on Linux because different loader is used.

Источник

• RpathIssue

Background

There is quite a lot of information on the RPATH issue in Debian, however there is no document which consistently explains what the problem actually is, why the use of RPATH is discouraged in Debian and what are the preferred ways to correct upstream builds that use this feature. This page was created by JurijSmakov with the intention of collecting all the relevant information on the subject.

The Problem

Dynamic linking

The job of the dynamic linker and loader (ld.so) is to resolve the executable’s dependencies on shared libraries and to load the required ones at run-time. It relies on the executable’s NEEDED headers to find the shared libraries with a matching SONAME (which includes the library name and ABI version).

The dynamic linker will look for a matching library in the following locations, in this order, which can be changed (see the footnotes below):

1. the DT_RPATH dynamic section attribute of the library causing the lookup
2. the DT_RPATH dynamic section attribute of the executable
3. the LD_LIBRARY_PATH environment variable, unless the executable is setuid/setgid.
4. the DT_RUNPATH dynamic section attribute of the executable

base library directories (/lib and /usr/lib)

Footnotes (the small print. ):

Regarding steps 1 and 2: The DT_RPATH attribute is ignored if the DT_RUNPATH attribute is found. Then, LD_LIBRARY_PATH is searched first!

Regarding step 3: LD_LIBRARY_PATH can be overridden by calling the dynamic linker with the option --library-path (e.g. /lib/ld-linux.so.2 --library-path $HOME/mylibs myprogram
Regarding steps 5 and 6: If the executable was linked with -z nodeflib linker option, /lib and /usr/lib are skipped at step 5 and 6.

Regarding all steps: If the dynamic linker is called using --inhibit-rpath LIST, the objects in LIST are ignored.

Before library searching takes place, the libraries in LD_PRELOAD are loaded

Why RPATH is an issue

A problem arises when binary A defines a NEEDED dependency on libraries B.so.1 and C.so.2, while library B.so.1 depends on library C.so.1. This means parts of the code will use one version of a library and other parts another. The many versions of a library scenario is needed to deal with gradual migrations, but maintainers shuffle libraries around when dealing with such a situation and packages with RPATH could end up finding the wrong version of a dependent library, one with incompatible dependencies.

Since RPATH is set at build-time, it can only be overridden with a rebuild or by setting the LD_LIBRARY_PATH variable and this turns packages using it into management problems:

LD_LIBRARY_PATH can’t be used because it has its own problems: it’ll be inherited by all processes generated by the parent and is therefore also discouraged for distribution-wide use for its possible side-effects.

Needing a rebuild means that all dependencies of the affected library would need to be rebuilt during any move, which is an unacceptable scenario for a distribution.

This situation would be better dealt with by the dynamic linker, since maintainers would have a central place to inform all dependent packages where to find libraries during transitional periods.

libtool’s role

Historically, GNU libtool made extensive use of the RPATH feature, including the hardcoded library path information by default into all executables and shared libraries, which meant that this issue used to appear on many libtool based packages whose libtool used RPATH as default and didn’t specifically implement workarounds for it.

Since libtool 1.5.2 (released 2004-01-25), on Linux libtool no longer sets RPATH for any directories in the dynamic linker search path, so this should no longer be an issue unless upstream used a really old version of libtool when creating their distribution tarball.

Debian’s Stance

While there’s no policy dictating the accepted use of RPATH, it’s been a general consensus that RPATH use is discouraged, given the interactions between the above reasons and Debian’s way of dealing with libraries and package dependencies.

Currently, the only generally accepted use of this feature in Debian is to add non-standard library path (like /usr/lib/

) to libraries that are only intended to be used by the executables or other libraries within the same source package.

Possible solutions

Upstream changes

Ideally, upstream should implement a build-system that supports turning RPATH usage off during configuration.

If the build-system is based on pre-1.5.2 libtool, suggest upstream upgrade to a newer version.

Patching the build-system

Alternatively, the build-system can be patched to avoid RPATH at build-time.

Autotools with libtool

Following example (taken from configure.ac in wmaker) illustrates it:

Another possibility (similar by the spirit) is presented by Matthew Garrett:

Or more generally, it's also possible to patch the configure generated libtool script, the autoconf generated configure script or the raw acloca.m4 script via debian/rules with:

NOTE: if by need or choice the aclocal.m4, configure.in or configure.ac files were patched, it's necessary to re-run autoconf in order to generate an up-to-date configure script.

CMake

You can stop cmake adding the flags for rpath-setting by adding

to the properties on the relevant binary in CMakeLists.txt e.g.

Ideally this change would be upstreamed.

Using chrpath

The package chrpath can be used during debian/rules' install target to strip the RPATH header from the generated binary.

rpath issue on amd64

Fedora (and possibly some other distros) patched their libtool 1.5 packages so that on 64 bit Linux platforms it calculates the list of directories ld.so searches as /lib64 and /usr/lib64 plus anything in ld.so.conf (rather than /lib and /usr/lib plus anything in ld.so.conf which is what vanilla upstream libtool does).

Debian use /lib and /usr/lib on all architectures, so if upstream built their tarballs on a Fedora 10 or earlier box, then an rpath would end up set in Debian packages for 64 bit architectures. The libtool packages in Fedora 11 and later (which are of libtool 2.2.6) have a revised patch which adds /lib64 and /usr/lib64 while keeping /lib and /usr/lib, so this problem is now resolved unless upstream uses a no longer supported version of Fedora (or hasn't made a release in a while).

Источник

Читайте также:  Скорость сетевой карты линукс