Linux library search path

How to show all shared libraries used by executables in Linux?

I’d like to know which libraries are used by executables on my system. More specifically, I’d like to rank which libraries are used the most, along with the binaries that use them. How can I do this?

13 Answers 13

1. Use ldd to list shared libraries for each executable.
2. Cleanup the output
3. Sort, compute counts, sort by count

To find the answer for all executables in the «/bin» directory:

Change «/bin» above to «/» to search all directories.

Output (for just the /bin directory) will look something like this:

Edit — Removed «grep -P»

I didn’t have ldd on my ARM toolchain so I used objdump:

This works better than ldd when the executable uses a non default loader

to learn what libraries a binary uses, use ldd

You’d have to write a little shell script to get to your system-wide breakdown.

Check shared library dependencies of a program executable

To find out what libraries a particular executable depends on, you can use ldd command. This command invokes dynamic linker to find out library dependencies of an executable.

> $ ldd /path/to/program

Note that it is NOT recommended to run ldd with any untrusted third-party executable because some versions of ldd may directly invoke the executable to identify its library dependencies, which can be security risk.

Instead, a safer way to show library dependencies of an unknown application binary is to use the following command.

$ objdump -p /path/to/program | grep NEEDED

readelf -d recursion

redelf -d produces similar output to objdump -p which was mentioned at: https://stackoverflow.com/a/15520982/895245

But beware that dynamic libraries can depend on other dynamic libraries, to you have to recurse.

Choose one, and repeat:

/maps for running processes

This is useful to find all the libraries currently being used by running executables. E.g.:

shows all currently loaded dynamic dependencies of init (PID 1 ):

This method also shows libraries opened with dlopen , tested with this minimal setup hacked up with a sleep(1000) on Ubuntu 18.04.

On OS X by default there is no ldd , objdump or lsof . As an alternative, try otool -L :

In this example, using which openssl fills in the fully qualified path for the given executable and current user environment.

On UNIX system, suppose binary (executable) name is test. Then we use the following command to list the libraries used in the test is

With ldd you can get the libraries that tools use. To rank the usage of libraries for a set of tool you can use something like the following command.

(Here sed strips all lines that do not start with a tab and the filters out only the actual libraries. With sort | uniq -c you get each library with a count indicating the number of times it occurred.)

You might want to add sort -g at the end to get the libraries in order of usage.

Источник

Shared Libraries: Understanding Dynamic Loading

In this post, I will attempt to explain the inner workings of how dynamic loading of shared libraries works in Linux systems. This post is long — for a TL;DR, please read the debugging cheat sheet.

This post is not a how-to guide, although it does show how to compile and debug shared libraries and executables. It’s optimized for understanding of the inner workings of how dynamic loading works. It was written to eliminate my knowledge debt on the subject, in order to become a better programmer. I hope that it will help you become better, too.

What Are Shared Libraries?

A library is a file that contains compiled code and data. Libraries in general are useful because they allow for fast compilation times (you don’t have to compile all sources of your dependencies when compiling your application) and modular development process. Static Libraries are linked into a compiled executable (or another library). After the compilation, the new artifact contains the static library’s content. Shared Libraries are loaded by the executable (or other shared library) at runtime. That makes them a little more complicated in that there’s a whole new field of possible hurdles which we will discuss in this post.

Example Setup

To explore the world of shared libraries, we’ll use one example throughout this post. We’ll start with three source files:

main.cpp will be the main file for our executable. It won’t do much — just call a function from a random library which we’ll compile:

The random library will define a single function in its header file, random.h :

It will provide a simple implementation in its source file, random.cpp :

Note: I’m running all of my examples on Ubuntu 14.04.

Compiling a Shared Library

Before compiling the actual library, we’ll create an object file from random.cpp :

In general, build tools don’t print to the standard output when everything is okay. Here are all the parameters explained:

• -o random.o : Define the output file name to be random.o .
• -c : Don’t attempt any linking (only compile).
• random.cpp : Select the input file.

Next, we’ll compile the object file into a shared library:

The new flag is -shared which specifies that a shared library should be built. Notice that we called the shared library librandom.so . This is not arbitrary — shared libraries should be called lib .so for them to link properly later on (as we’ll see in the linking section below).

Compiling and Linking a Dynamic Executable

First, we’ll create a shared object for main.cc :

This is exactly the same as before with random.o . Now, we’ll try to create an executable:

Okay, so we need to tell clang that we want to use librandom.so . Let’s do that 1 :

Hmmmmph. We told our compiler we want to use a librandom file. Since it’s loaded dynamically, why do we need it in compile time? Well, the reason is that we need to make sure that the libraries we depend on contain all the symbols needed for our executable. Also note that we specified random as the name of the library, and not librandom.so . Remember there’s a convention regarding library file naming? This is where it’s used.

So, we need to let clang know where to search for shared libraries. We do this with the -L flag. Note that paths specified by -L only affect the search path when linking — not during runtime. We’ll specify the current directory:

Great. Now let’s run it!

This is the error we get when a dependency can’t be located. It will happen before our application even runs one line of code, since shared libraries are loaded before symbols in our executable.

This raises several questions:

• How does main know it depends on librandom.so ?
• Where does main look for librandom.so ?
• How can we tell main to look for librandom.so in this directory?

To answer these question, we’ll have to go a little deeper into the structure of these files.

ELF — Executable and Linkable Format

The shared library and executable file format is called ELF (Executable and Linkable Format). If you check out the Wikipedia article you’ll see that it’s a hot mess, so we won’t go over all of it. In summary, an ELF file contains:

• ELF Header
• File Data, which may contain:
  • Program header table (a list of segment headers)
  • Section header table (a list of section headers)
  • Data pointed to by the above two headers

The ELF header specifies the size and number of segments in the program header table and the size and number of sections in the section header table. Each such table consists of fixed size entries (I use entry to describe either a segment header or a section header in the appropriate table). Entries are headers and they contain a “pointer” (an offset in the file) to the location of the actual body of the segment or section. That body exists in the data part of the file. To make matters more complicated — each section is a part of a segment, and a segment can contain many sections.

In effect, the same data is referenced as either part of a segment or a section depending on the current context. sections are used when linking and segments are used when executing.

We’ll use readelf to… well, read the ELF. Let’s start by looking at the ELF header of main :

We can see that this is an ELF file (64-bit) on Unix. Its type is EXEC , which is an executable file — as expected. It has 9 program headers (meaning it has 9 segments) and 30 section headers (i.e., sections).

Next up — program headers:

Again, we see that we have 9 program headers. Their types are LOAD (two of those), DYNAMIC , NOTE , etc. We can also see the section ownership of each segment.

Finally — section headers:

I trimmed this one for the sake of brevity. We see our 30 sections listed with various names (e.g., .note.ABI-tag ) and types (e.g., SYMTAB ).

You might be confused by now. Don’t worry — it won’t be on the test. I’m explaining this because we’re interested in a specific part of this file: In their Program Header Table, ELF files can have (and shared libraries in particular must have) a segment header that describes a segment of type PT_DYNAMIC . This segment owns a section called .dynamic which contains useful information to understand dynamic dependencies.

Direct Dependencies

We can use the readelf utility to further explore the .dynamic section of our executable 2 . In particular, this section contains all of the dynamic dependencies of our ELF file. We only specified librandom.so as a dependency, so we would expect there to be exactly one dependency listed:

We can see librandom.so , which we specified, but we also get four extra dependencies we didn’t expect. These dependencies seem to appear in all compiled shared libraries. What are they?

• libstdc++ : The standard C++ library.
• libm : A library that contains basic math functions.
• libgcc_s : The GCC (GNU Compiler Collection) runtime library.
• libc : The C library: the library which defines the ‘system calls’ and other basic facilities such as open , malloc , printf , exit , etc.

Okay — so we know that main knows it depends on librandom.so . So why can’t main find librandom.so in runtime?

Runtime Search Path

ldd is a tool that allows us to see recursive shared library dependencies. That means we can see the complete list of all shared libraries an artifact needs at runtime. It also allows us to see where these dependencies are located. Let’s run it on main and see what happens:

Right off the bat, we see that librandom.so is listed — but not found. We can also see that we have two additional libraries ( vdso and ld-linux-x86-64 ). They are indirect dependencies. More importantly, we see that ldd reports the location of the libraries. Consider libstdc++ . ldd reports its location to be /usr/lib/x86_64-linux-gnu/libstdc++.so.6 . How does it know?

Each shared library in our dependencies is searched in the following locations 3 , in order:

1. Directories listed in the executable’s rpath .
2. Directories in the LD_LIBRARY_PATH environment variable, which contains colon-separated list of directories (e.g., /path/to/libdir:/another/path )
3. Directories listed in the executable’s runpath .
4. The list of directories in the file /etc/ld.so.conf . This file can include other files, but it is basically a list of directories — one per line.
5. Default system libraries — usually /lib and /usr/lib (skipped if compiled with -z nodefaultlib ).

Fixing our Executable

Alright. We validated that librandom.so is a listed dependency, but it can’t be found. We know where dependencies are searched for. We’ll make sure that our directory is not actually on the search path by using ldd again:

I trimmed the output since it’s very… chatty. It’s no wonder our shared library is not found — the directory where librandom.so is located is not in the search path! The most ad-hoc way to solve this is to use LD_LIBRARY_PATH :

It works, but it’s not very portable. We don’t want to specify our lib directory every time we run our program. A better way is to put our dependencies inside the file.

Enter rpath and runpath .

rpath and runpath

rpath and runpath are the most complex items in our runtime search path “checklist”. The rpath and runpath of an executable or shared library are optional entries in the .dynamic section we reviewed earlier 4 . They are both a list of directories to search for.

The only difference between rpath and runpath is the order they are searched in. Specifically, their relation to LD_LIBRARY_PATH — rpath is searched in before LD_LIBRARY_PATH while runpath is searched in after. The meaning of this is that rpath cannot be changed dynamically with environment variables while runpath can.

Let’s bake rpath into our executable and see if we can get it to work:

The -Wl flag passes the following, comma-separated, flags to the linker. In this case, we pass -rpath . . To set runpath instead, we would also have to pass —enable-new-dtags 5 . Let’s examine the result:

The executable runs, but this added . to the rpath , which is the current working directory. This means it won’t work from a different directory:

We have several ways to solve this. The easiest way is to copy librandom to a directory that is in our search path (such as /lib ). A more complicated way, which, obviously, is what we’re going to do — is to specify rpath relative to the executable.

$ORIGIN

Paths in rpath and runpath can be absolute (e.g., /path/to/my/libs/ ), relative to the current working directory (e.g., . ), but they can also be relative to the executable. This is achieved by using the $ORIGIN variable 6 in the rpath definition:

Notice that we need to escape the dollar sign (or use single quotes), so that our shell won’t try to expand it. The result is that main works from every directory and finds librandom.so correctly:

Let’s use our toolkit to make sure:

Runtime Search Path: Security

If you ever changed your Linux user password from the command line, you may have used the passwd utility:

The password hash is stored in /etc/shadow , which is root protected. How then, you might ask, your non-root user can change that file?

The answer is that the passwd program has the setuid bit set, which you can see with ls :

It’s the s (the fourth character of the line). All programs that have this permission bit set run as the owner of that program. In this example, the user is root (third word of the line).

“What does that have to do with shared libraries?”, you ask. We’ll see with an example.

We’ll now have librandom in a libs directory next to main and we’ll bake $ORIGIN/libs 7 in our main ’s rpath :

If we run main , it works as expected. Let’s turn on the setuid bit for our main executable and make it run as root :

Alright, rpath doesn’t work. Let’s try setting LD_LIBRARY_PATH instead:

What’s going on here?

For security reasons, when running an executable with elevated privileges (such as setuid , setgid , special capabilities, etc.), the search path list is different than normal: LD_LIBRARY_PATH is ignored, as well as any path in rpath or runpath that contains $ORIGIN .

The reason is that using these search path allows to exploit the elevated privileges executable to run as root . Details about this exploit can be found here. Basically, it allows you to make the elevated privileges executable load your own library, which will run as root (or a different user). Running your own code as root pretty much gives you absolute control over the machine you’re using.

If your executable needs to have elevated privileges, you’ll need to specify your dependencies in absolute paths, or place them in the default locations (e.g., /lib ).

An important behavior to note here is that, for these kind of applications, ldd lies to our face:

ldd doesn’t care about setuid and it expands $ORIGIN when it is searching for our dependencies. This can be quite a pitfall when debugging dependencies on setuid applications.

Debugging Cheat Sheet

If you ever get this error when running an executable:

You can try doing the following:

• Find out what dependencies are missing with ldd .
• If you don’t identify them, you can check if they are direct dependencies by running readelf -d | grep NEEDED .
• Make sure the dependencies actually exist. Maybe you forgot to compile them or move them to a libs directory?
• Find out where dependencies are searched by using LD_DEBUG=libs ldd .
• If you need to add a directory to the search:
  • Ad-hoc: add the directory to the LD_LIBRARY_PATH environment variable.
  • Baked in the file: add the directory to the executable or shared library’s rpath or runpath by passing -Wl,-rpath, (for rpath ) or -Wl,—enable-new-dtags,-rpath, (for runpath ). Use $ORIGIN for paths relative to the executable.
• If ldd shows that no dependencies are missing, see if your application has elevated privileges. If so, ldd might lie. See security concerns above.

If you still can’t figure it out — you’ll need to read the whole thing again 🙂

Sources

Note that we chose to dynamically link librandom.so to main . It’s possible to do this statically — and load all of the symbols in the random library directly into the main executable. ↩︎

The objdump executable can provide similar results. In this case for example: objdump -p librandom.so | grep NEEDED will print very similar output. ↩︎

The search path is different for setuid / setguid applications. See below for more details. ↩︎

rpath ’s type is DT_RPATH while runpath ’s type is DT_RUNPATH . ↩︎

Note that $ORIGIN is not an environment variable. If you export ORIGIN=/path it will have no effect. It is always the directory in which the executable is placed. ↩︎

Источник