When linking a shared library on linux, are all modules included?

Viewed 270

I'm porting a system of apps from AIX to linux, and all of those apps include a single shared library. I've got the shared library building on as a linux .so now - and I see at least one post here that describes how to specify what's exported from a shared library (as AIX does via a .exp file).

Just one silly question, though. On AIX, if a module in a shared library is not referenced by anything in the app that's linking to it, it is ignored by the linker. That doesn't seem to be the case on linux - but I want to make sure.

While testing my linux shared library, I left out one module with dependencies I wasn't ready to deal with yet (or more accurately, I provided a substitute module with dummy functions for all the entry points to that module, thinking that would allow it to link). So far, so good. But when I attempted to link that shared library into a trivial test app, the linker reported unresolved symbols for stuff referenced by another shared library module that is itself only referenced from within the module I replaced with dummies. I.e., I would have expeceted that module to simply be ignored...

In other words, this module is being considered by the linker as part of the final application even though nothing in the app references it. I tried the same experiment on AIX (replacing the same module with dummies and attempting to link a trivial app there). No complaints.

So, The AIX linker only attempts to resolve shared library module dependencies if those modules themselves are explicitly called in from the application. But the linux linker attempts to resolve dependencies for all shared library modules whether they're called in from the application or not.

Is this true? And if so, is there any way to override that behavior? Ultimately, when I port everything, all of the dependencies will resolve. But for now, it's hard to leave something out - even if it's not referenced...


Here's a minimal case:

main.c contains function main(), which calls function one().
one.c contains function one(), which does nothing.
two.c contains function two(), which calls function three().

There is no function three(), but libshared.so is built from 
modules one.c and two.c.  Program main is built from main.c and
links in libshared.so. 

The linker needs to resolve function one(), which is in the shared
library.  But that's all main.c requires.  Still, function two() in
the library references function three(), which doesn't exist.
The linker will complain about the undefined symbol 'three', even
though program main doesn't need it.
On AIX the linker will not complain and everything will work.

main.c:

    #include <stdio.h>
    int     one();
    int main()
    {
            one();
    }

one.c:

    #include <stdio.h>
    int one()
    {
            return 1;
    }

two.c:

    #include <stdio.h>
    int three();
    int two()
    {
            return three();
    }

build libshared.so with modules one.c and two.c:

    gcc -fPIC -shared one.c two.c -o libshared.so

Attempt to build main from main.c and libshared.so:

    gcc main.c -o main -L. -lshared
    ./libshared.so: undefined reference to `three'
    collect2: error: ld returned 1 exit status

The linker reports an undefined reference to 'three',
which is referenced from two() - but main() doesn't ever call two().
2 Answers

The actual answer: shared libraries are in fact shared objects: they are treated as a single object, not as a *.a library.

This shows that Linux (meaning: glibc/gcc/gold/ld) and AIX have different concepts regarding shared objects.

In Linux, when you link an executable, ld/gold checks the dependencies of the used shared objects as well -- Aix linker doesn't: it assumes that the shared objects are to be used as they are, their dependencies aren't part of the current linking. (At least this is the default behaviour.)

Here is a summary of my tests:

+----------------+--------------------+-------------------------------+
|                |    AIX             |    linux                      |
+----------------+--------------------+-------------------------------+
| libshared.so   |  only with option  |    yes                        |
| can be created |  -Wl,-berok        |                               |
+----------------+--------------------+-------------------------------+
| main           |    yes             |  only with option             |
| can be created |                    |  -Wl,--allow-shlib-undefined  |
+----------------+--------------------+-------------------------------+

Note: My random thoughts regarding AIX and linking: http://lzsiga.users.sourceforge.net/aix-linking.html

By default the GNU binutils linker, ld on Linux requires a symbol ref to be defined by some input file (i.e. object file or shared library) in the linkage if ref is referenced by the definition of any symbol def in any input file that the linkage needs. It doesn't matter whether def is referenced in turn.

Your program linkage needs libshared.so. libshared.so defines two, which refers to three, so three must be defined.

You can countermand this default behaviour to tolerate undefined references in shared libraries (but not in object files) as follows:

$ gcc main.c -o main -L. -lshared -Wl,--allow-shlib-undefined

--allow-shlib-undefined is documented in the ld manual

The notion of module in your language corresponds to translation unit at the compilation level and object file at the linkage level. It might be helpful to appreciate that an object file input to the linkage of a ELF program or shared library has no distinct existence in the program or shared library. It is cut into pieces and scattered around. So there is no sense in which it would be possible for a linkage:

$ gcc main.c -o main -L. -lshared ...

to ignore the unreferenced module two.(c|o) within libshared.so. There is no such thing. If that linkage did not need any definition provided by libshared.so then it would ignore the shared library altogether1. If it needs the shared library, then by default its references must be resolved.


[1] That is, on Debian-clan systems where gcc is built to invoke ld with the --as-needed option by default. On Redhat-clan systems GCC by default links shared libraries if they are input, needed or not.

Related